Accessibility best practices for virtual events

As everyone knows, most of our open source events have transformed from in-person to digital this year. However, due to the sudden change, not everything is accessible. We took this issue seriously and decided to work with one of our accessibility experts, Neighborhood Access, to share best practices for our community. We hope this will help you organize your digital events!

Bringing Live Transcribe's Speech Engine to Everyone

Earlier this year, Google launched Live Transcribe, an Android application that provides real-time automated captions for people who are deaf or hard of hearing. Through many months of user testing, we've learned that robustly delivering good captions for long-form conversations isn't so easy, and we want to make it easier for developers to build upon what we've learned. Live Transcribe's speech recognition is provided by Google's state-of-the-art Cloud Speech API, which under most conditions delivers pretty impressive transcript accuracy. However, relying on the cloud introduces several complications—most notably robustness to ever-changing network connections, data costs, and latency. Today, we are sharing our transcription engine with the world so that developers everywhere can build applications with robust transcription.

Those who have worked with our Cloud Speech API know that sending infinitely long streams of audio is currently unsupported. To help solve this challenge, we take measures to close and restart streaming requests prior to hitting the timeout, including restarting the session during long periods of silence and closing whenever there is a detected pause in the speech. Otherwise, this would result in a truncated sentence or word. In between sessions, we buffer audio locally and send it upon reconnection. This reduces the amount of text lost mid-conversation—either due to restarting speech requests or switching between wireless networks.

(From the Live Transcribe launch video)

Endlessly streaming audio comes with its own challenges. In many countries, network data is quite expensive and in spots with poor internet, bandwidth may be limited. After much experimentation with audio codecs (in particular, we evaluated the FLAC, AMR-WB, and Opus codecs), we were able to achieve a 10x reduction in data usage without compromising accuracy. FLAC, a lossless codec, preserves accuracy completely, but doesn't save much data. It also has noticeable codec latency. AMR-WB, on the other hand, saves a lot of data, but delivers much worse accuracy in noisy environments. Opus was a clear winner, allowing data rates many times lower than most music streaming services while still preserving the important details of the audio signal—even in noisy environments. Beyond relying on codecs to keep data usage to a minimum, we also support using speech detection to close the network connection during extended periods of silence. That means if you accidentally leave your phone on and running Live Transcribe when nobody is around, it stops using your data.

Finally, we know that if you are relying on captions, you want them immediately, so we've worked hard to keep latency to a minimum. Though most of the credit for speed goes to the Cloud Speech API, Live Transcribe's final trick lies in our custom Opus encoder. At the cost of only a minor increase in bitrate, we see latency that is visually indistinguishable to sending uncompressed audio.

Today, we are excited to make all of this available to developers everywhere. We hope you'll join us in trying to build a world that is more accessible for everyone.

By Chet Gnegy, Alex Huang, and Ausmus Chang from the Live Transcribe Team

iOS Accessibility Scanner Framework

At Google, we are committed to accessibility and are constantly looking for ways to improve our development process to discover, debug and fix accessibility issues. Today we are excited to announce a new open source project: Accessibility Scanner for iOS (or GSCXScanner as we lovingly call it). This is a developer tool that can assist in locating and fixing accessibility issues while an app is being developed.

App development can be a time consuming process, especially when it involves human testers. Sometimes, as in the case with accessibility testing, they are necessary. A developer can write automated tests to perform some accessibility checks, but GSCXScanner takes this one step further. When a new feature is being developed, often there are several iterations of code changes, building, launching and trying out the new feature. It is faster and easier to fix accessibility issues with the feature if they can be detected during this phase when the developer is working with the new feature.

GSCXScanner lives in your app process and can perform accessibility checks on the UI currently on the screen simply with the touch of a button. The scanner’s UI which is overlaid on the app can be moved around so you can use your app normally and trigger a scan only when you need it. Also, it uses GTXiLib, a library of iOS accessibility checks to scan your app, and you can author your own GTX checks and have them run along with scanner’s default checks.

Using the scanner does not eliminate the need for manual testing or automated tests, these are must haves for delivering quality products. But GCSXScanner can speed up the development process by showing issues in app during development.

Help us improve GSCXScanner by suggesting a feature or better yet, writing one.

By Sid Janga, Central Accessibility Team

Open sourcing GTXiLib, an accessibility test automation framework for iOS

Google believes everyone should be able to access and enjoy the web. We share guidance on building accessible tech over at Google Accessibility and we recently launched a dedicated disability support team. Today, we’re excited to announce that we’ve open sourced GTXiLib, an accessibility test automation framework for iOS, under the Apache license.

We want our products to be accessible and automation, with frameworks like GTXiLib, is one of the ways we scale our accessibility testing. GTXiLib can automate the process of checking for some kinds of issues such as missing labels, hints, or low contrast text.

GTXiLib is written in Objective-C and will integrate with your existing XCTests to perform all the registered accessibility checks before the test tearDown. When the checks fail, the existing test fails as well. Fixing your tests will thus lead to better accessibility and your tests can catch new accessibility issues as well.

• Reuse your tests: GTXiLib integrates into your existing functional tests, enhancing the value of any tests that you have or any that you write.
• Incremental accessibility testing: GTXiLib can be installed onto a single test case, test class or a specific subset of tests giving you the freedom to add accessibility testing incrementally. This helped drive GTXiLib adoption in large projects at Google.
• Author your own checks: GTXiLib has a simple API to create custom checks based on the specific needs of your app. For example, you can ensure every button in your app has an accessibilityHint using a custom check.

Do you also care about accessibility? Help us sharpen GTXiLib by suggesting a check or better yet, writing one. You can add GTXiLib to your project using CocoaPods or by using its Xcode project file.

We hope you find this useful and look forward to feedback and contributions from the community! Please check out the README for more information.

By Siddartha Janga, Google Central Accessibility Team 

A Summer of Accessibility

Tekla (formerly mEADL) is a collection of open hardware and open source applications that may be used to enable access to mobile devices for people with motor impairments. The idea for Tekla was the brainchild of the mobile accessibility team at the Inclusive Design Institute (IDI), which became a new mentoring organization for Google Summer of Code in 2010. Our goal has been to bridge some of the gaps currently preventing people with disabilities from using mobile devices.

Google Summer of Code student Eric Wan is a developer who employs accessibility switch access in his daily life. Having him on our team was incredibly valuable because he has a personal understanding of the practical issues involved in creating an accessibility solution. Google Summer of Code provided the means for Eric to "scratch his own itch" and now many more switch users may potentially benefit as the IDI is working with Komodo OpenLab to commercialize the technology.

The video above shows Eric using the “sip and puff” switches attached to his wheelchair to send commands to his Android handset in order to start the phone's SMS application and compose a text message - all without ever touching the phone!

How it works:
Switch events from Eric's wheelchair are sent to the phone via the Tekla shield, a bluetooth interface based on the Arduino open-hardware platform. Once on the phone, these switch events are used to navigate an on-screen keyboard provided by the Tekla open-source app, which is available for free from the Android market.

Challenges:
There is still some accessibility work to do on Android devices. For example, menus that appear when pressing the device MENU key and some pop-up windows will block the on-screen keyboard, sometimes locking switch users out of the phone. Also, some third-party application developers will disable access features in their interfaces, making their apps unusable with the Tekla shield. For example, Eric can sign in to Skype, but he cannot use the "call" button or the dialer due to limitations of the user interface. Luckily though, most functions and apps are accessible because developers generally have to go out of their way to make their apps inaccessible.

Acknowledgements:
Eric is an engineering grad student at the University of Toronto, and he helped develop Tekla with me (his mentor) and Zongyi Yang, another U of T student. In addition to funding from the 2010 Google Summer of Code program, development on Tekla has been funded by the Ontario Ministry of Research and Innovation. Tekla is also being developed as part of the AEGIS Project.

By Jorge Silva, Google Summer of Code Mentor for Inclusive Design Institute

Changing the Look of the Web with Stylebot

Stylebot editing the Open Source Blog in advanced mode

Stylebot editing the Open Source Blog in basic mode

Cascading Style Sheets (CSS) are a part of every web designer’s vocabulary when styling websites, and since its inception, the C (Cascading) in CSS was intended as a way to empower users to have the final say over how they perceive content on the web. But because creating user stylesheets generally requires programming, end users have not always been able to easily leverage this functionality. Stylebot, a new Google Chrome extension created as a Google Summer of Code project, hopes to finally unlock the power of the C in CSS by giving the end user final control on how web content is displayed.

At Google, the Accessibility Engineering team is very excited about the potential of extensions like Stylebot to improve the accessibility of the web, making it possible for users to customize the web to fit their needs. For example, a Stylebot user with special reading needs might change a webpage by removing images, picking new text and background colors, and even moving blocks of text around. And Stylebot saves the custom style they create, so the next time they access that page the changes will still be there. Even better, they can sync their saved styles across computers so that webpage will always appear with their preferred style.

We invited Ankit Ahuja, the Google Summer of Code student who worked on Stylebot, to write about his experiences creating the extension. While there’s still a lot of work to be done to make Chromium more accessible, extensions like Stylebot are a great step - giving users themselves the power to shape the way they interact with the web.

My name is Ankit Ahuja, and I successfully completed my Google Summer of Code project this year for Chromium. I was mentored by Rachel Shearer. My project is Stylebot, a Chromium extension that enables users to easily customize the web’s appearance. Ultimately, Stylebot aims to make the web more accessible and adaptable.

StyleBot screencast demo

One of the main objectives of the project was to allow users unfamiliar with CSS to be able to use this extension with ease. In the Basic mode, users launch Stylebot on a page, select an element and style it. Changes made by the users are automatically saved, so the next time they visit the page, their custom styles are already applied. This mode provides an easy-to-use GUI for the commonly used CSS properties. For the more advanced users, there is a separate mode in which they can write their own CSS.

A nontrivial problem was allowing the users to preview the changes instantly. So although a stylesheet is used to apply the custom CSS when the page is initially loaded, inline CSS is deployed while the user is in editing mode for a smooth, dynamic editing experience. Another important issue was determining the best way to position the Stylebot panel on the page. During testing, I found drag-and-drop to be slow. Instead, allowing the panel to be moved to a fixed left or right position felt the most user friendly. CSS parsing was required and luckily there already existed a CSS parser in JavaScript.

On my part, I’ve tried to make sure the Stylebot code is useful for other developers. I’ve kept the implementation of features like extension data synchronization, selection of elements, CSS selector generation, etc. separate, so that anyone can reuse the code easily. I’ve also used code from other open source projects. For example, the user interface is the CSS version of Cappuccino’s Aristo and the selection of elements is similar to Firebug’s implementation.

Take a look at a few examples we created using Stylebot. You can install the extension from the Google Chrome extensions gallery. We’re already receiving some positive and critical feedback, which is exciting!

I had an amazing experience participating in Google Summer of Code this year. I had a great time interacting with my mentor through the summer, who was very helpful and motivating. Finally, I would like to thank Google for sponsoring and making this project possible.

By Ankit Ahuja, 2010 Google Summer of Code Student and Rachel Shearer, Google Accessibility Engineering Team

London Open Source Jam 14

We held the 14th Google London Open Source Jam at our Victoria HQ on September 24th. The topic this time was "Video and Sound", and our Jammers had some real treats to share.

Steven Goodwin told us how his open source SGX 3D graphics engine deals with three key problems of other computer game engines. On a similar theme, Themis Bourdenas discussed the vine engine, a modular game engine for 2d and 3d games.

Borys Musielak presented Filmaster, an open source film recommendation engine. Neil Harris told us about an attempt by the Kendra Initiative to foster a common meta data format for content discovery on the semantic web.

In an Open Source Jam first, Jagannathan gave a performance of his Din software musical instrument. Din is designed for playing live Indian music, is based on Bezier curves and really has to be heard to be fully appreciated.

Sam Mbale gave us an update on his projects to help Africans build online communities using open source. Mike Mahemoff discussed some web tools frameworks for intranets, bookmarklets and trails in Scrumptious.

The UK government has plans to introduce a law to allow content-owners to force ISPs to disconnect the internet connection of users suspected of file sharing, without any proof. Glyn Wintle gave us an overview of how the proposed law will affect us, how the Open Rights Group is campaigning against it, and how we can help.

Douglas Squirrel talked about the difficulty blind people have in finding information on websites, and presented BlindPages.com - a new project to reformat the web in a screen-reader friendly way. He also demoed a prototype telephone interface to the service.

Much pizza was eaten and free beer drunk, and we all ended up in the pub next door to continue our discussions. A big thank you to all our speakers and attendees, and we hope to see you at the next Jam!


By Matt Godbolt, Mobile Engineering

TalkBack: An Open Source Screenreader For Android

Earlier this year, we blogged about project Eyes-Free — a collection of Android applications that enable efficient eyes-free interaction with your mobile phone. Since then, one of the questions we have received most often is about a complete access API to enable general purpose adaptive technologies such as screenreaders.

We are happy to announce the first version of such an API as part of the latest Android release (Donut). This new API is now available within the Android 1.6 SDK , and we welcome developer feedback. The Android Access framework generates android.view.accessibility.AccessibilityEvent in response to user interaction events; the event payload contains additional details about the event, e.g., the user interface control that received focus. This access framework enables the creation of general purpose screenreading applications that make all of Android's user interface, as well as native Android applications built with standard Android widgets usable without looking at the screen.

You can see this API in use within our Open Source Android screenreader TalkBack. With TalkBack installed, standard Android user interface elements such as ListView produce spoken feedback during user interaction. Applications SoundBack (for producing non-spoken auditory feedback) and KickBack (for producing haptic feedback) generate additional augmentative output and demonstrate how multiple access applications can be active simultaneously.

in response to user interaction events; the event payload contains additional details about the event, e.g., the user interface control that received focus. This access framework enables the creation of general purpose screenreading applications that make all of Android's user interface, as well as native Android applications built with standard Android widgets usable without looking at the screen.

What This Means For Developers

If you are interested in developing innovative access solutions on Android and have been eagerly waiting for our access APIs, the Donut SDK contains what you have been waiting for — including a set of free voices for English (US and UK), French, Italian, German and Spanish. You can use TalkBack, SoundBack and KickBack as a starting point for designing your own access innovations.

If you are an Android developer interested in making your applications more widely usable, you can use TalkBack and friends to quickly verify whether your applications remain usable when not looking at the screen. In this context, here are a few coding tips to ensure that your applications work out of the box with these tools:

1. Ensure that all visually drawn UI controls have meaningful textual labels.
2. Ensure that users can navigate to all controls in your application using the trackball.
3. Ensure that navigating controls in your application with the trackball results in a meaningful traversal order.

What This Means For End Users

End-users of Android 1.6 (Donut) can enable TalkBack, SoundBack and KickBack via the Accessibility section of the Settings menu. You need to do this only once i.e., once enabled, these access applications remain active across restarts. Note that depending on your Android device, you may need to install these applications from the Android Market; we will post videos that demonstrate step-by-step instructions for specific Android devices in the Eyes-Free channel on YouTube.

Providing Feedback

We (T. V. Raman, Charles L Chen, and Svetoslav Ganov) will be continuously improving the underlying APIs and access tools, and we look forward to your questions and feedback on the Android Developers Group.

By Charles Chen and Svetoslav Ganov, Software Engineering Team and T.V. Raman, Research Scientist

Talking RSS Reader for Android

Keeping informed in a fast moving world can be a challenge. What if you could use those moments when your body is busy but your mind is idle to catch up on the news? That's how I decided that I would get my Android phone to read the news to me, out loud. This is doubly useful for me, because I am blind.

The Talking RSS Reader application reads articles out loud using text-to-speech. The text of the sentence currently being spoken is colored on the screen. Speech and text scrolling are synchronized. The touchscreen buttons to skip articles are right at the bottom corners of the screen, where your fingers can find them on their own. Menus and dialogs are also spoken out, so that you can "star" an item or choose a different RSS feed without ever having to look at your phone.

The application integrates with the Google Reader service, which means that articles read on your phone need not be shown to you again when you use Google Reader on another device.

It is my hope that drivers, joggers and commuters will find this a helpful tool for keeping up with the news that concerns them.

The source code for the application is available on Google code, so that anyone wanting to develop a useful talking application for Android will benefit from what I learned. If you'd like to send feedback or have questions, drop by our discussion list. Happy hearing!

By Stephane Doyon, Software Engineering Team

Writing Talking Applications for Android

Toward the end of last year, we released an Android binding to eSpeak, an Open Source Text To Speech (TTS) engine that provides support for a large number of languages. Since then, we have seen significant interest among the developer community. Many have stepped up to using the Text To Speech service for a wide variety of end-user scenarios.

Based on our implementation experience, we've once again updated the TTS service and the update is available from Android Market as before. In addition to the source code, you can find extensive documentation, including an online tutorial on using the TTS API on the Eyes-Free project site. You might also want to subscribe to our Eyes-Free Android YouTube Channel or check out Marvin, the Eyes-Free Shell for Android. Marvin provides a useful launchpad for developers creating eyes-free applications using spoken output via the TTS API.

We'd like to thank the eSpeak developers for a versatile Open Source TTS engine, and look forward to turning Android into a productive eyes-free environment!

By Charles Chen, Software Engineering Team and T.V. Raman, Research Scientist

Announcing Eyes-Free Shell For Android

Project Eyes-Free aims to enable fluent eyes-free use of mobile devices running Android. Target uses range from eyes-busy environments like driving, to use by people who are unwilling or unable to look at the visual display. You can get a high-level overview of more potential use cases for Eyes-Free from this recent New York Times article. As described in the article, we are releasing components from project Eyes-Free as they become ready for end-user deployment.

Though the underlying source code has been available for some time from our repository on Google Code, we've now posted the first public release of the eyes-free shell on the Android Marketplace. Users of the eyes-free shell can conveniently launch talking applications. Along with this release, we've also made available a collection of applications to turn mobile devices running Android into eyes-free communication devices.

Each of these applications have been written to be useful both to end users and as a means of helping the developer community come up to speed quickly as they develop eyes-free applications for Android:

Talking Dialer

A key innovation is the use of the touch screen to enable one-handed, eyes-free dialing of phone numbers using the touch screen. The dialer comes with a talking phone-book that enables users to quickly select a desired contact using the touch screen.

Knowing Your Location

This mini-application announces your present location based on information acquired via GPS and the cell network. It speaks your current heading using the built-in magnetic compass, looks up the current location on Google Maps, and announces the location in terms of a nearby address and street intersection.

Device State

This mini-application announces useful information such as battery state, signal strength, and availability of WiFi networks.

Date And Time

This mini-application provides single-touch access to current date and time.

We will be uploading video tutorials demonstrating the use of these applications to YouTube over the next few weeks. Please see the Eyes-Free project home page for these links as they become available. As always, we welcome your feedback and look forward to hearing from you in our discussion group.

by Charles Chen, Software Engineering Team and T.V. Raman, Research Scientist

Open Source Developers @ Google Speaker Series: Charles Chen & T.V. Raman

By Leslie Hawthorn, Open Source Team

Want to learn more about creating accessible Web 2.0 applications from the creators of Fire Vox and Emacspeak? If you are nearby Google's Mountain View, California, USA Headquarters on Monday, July 14th, please join us for Charles Chen and T.V. Raman's presentation Enhancing Web 2.0 Accessibility via AxsJAX. They will take you through a hands on tutorial on Google-AxsJax, an Open Source framework for injecting usability enhancements into Web 2.0 applications. Among other topics, Charles and T.V. will cover an overview of AxsJAX's developer tools, enabling eyes-free interaction for web applications and iterative design processes for accessibility improvements. They will also let you know the secret to getting a cool t-shirt with the Google logo printed in Braille.

Like all sessions of the Open Source Developers @ Google Speaker Series, this session will be open to the public. Doors open at 6:30 PM and light refreshments will be served. All are welcome and encouraged to attend; guests should plan to sign in at Building 43 reception upon arrival. For those of you who cannot join us in person, the presentation will be taped and published along with all public Google Tech Talks.

For those of you who were unable to attend our last presentation, you can check out the video of Tim Ansell's presentation Gaming for Freedom.

AxsJAX: My Web 2.0 Application Is Feeling Accessible

By T.V. Raman, Research Scientist





Last November, we announced the release of the AxsJAX framework as an Open Source project hosted on Google Code. The convenience afforded by having HTTP access to the GoogleCode repository has meant that this project has neverneeded to make downloadable releases — as we build new enhancements, users automatically experience the benefits of what we create.

The AxsJAX project depends on a number of cutting-edge technologies — including the emerging browser support for W3C ARIA. Charles and I took advantage of the presence of over 4,000 attendees at the CSUN 2008 conference last week to expose as many end-users and developers as possible to the benefits of Web 2.0 technologies. Until now, these technologies have been mostly viewed as an obstacle to users with special needs; I therefore found it refreshing to turn things around and demonstrate positive benefits that emerge when the underlying web platform is used to solve accessibility challenges.

Those of you who ran into me or Charles at the conference probably noticed our AxsJAX shirts — thanks again to Google's Open Source Team for helping us spread the source! The last few months have seen the AxsJAX framework evolve rapidly, and in that process, the library has acquired significant functionality that drastically reduces the amount of code it takes to AxsJAX content-rich web applications. We would like to take this opportunity to open up the AxsJAX Challenge — we invite web developers to innovate on our work as you AxsJAX your favorite web applications. Let us know about the great work you've done and we will recognize your work with a cool t-shirt that sports the Google logo in Braille on it.