Android Accessibility Update Focuses on Facial Gestures

Google's latest Android accessibility features rolled out this week, aimed at people with severe motor and speech disabilities who can't always rely on voice or touch to navigate their handsets.

Based on feedback from alternative communication technology users, Android is letting facial gestures do the heavy lifting, thanks to the front-facing camera and machine learning software that detect face and eye movements.

Camera Switches

The latest addition to Switch Access—part of the Android Accessibility Suite—Camera Switches lets you use one or more gestures to complete on-screen tasks. Assign silent signals like smiling, raising your eyebrows, looking left, right, up, or opening your mouth to release the notification panel, launch quick settings, scroll forward or backward, or return to the home screen, among other actions.

"We heard from people who have varying speech and motor impairments that customization options would be critical," Google Research and Central Accessibility Team product managers Lisie Lillianfeld and Allen Nikka, respectively, wrote in a blog announcement. "With Camera Switches, you or a caregiver can select how long to hold a gesture and how big it has to be to be detected."

Additional settings include the ability to show a visual indication of how long you hold a gesture, play a sound when something on screen changes in response, and keep the screen on when Camera Switches is enabled (the feature cannot unlock the screen if it turns off). Camera Switches can be used in tandem with physical switches.

To get started, download the Android Accessibility Suite app and visit Switch Access > Android Accessibility.

Project Activate

New kid on the block Project Activate also employs facial gestures to trigger customized actions, like speaking a preset phrase, sending a text, making a phone call, or playing audio.

"Customization is built into all areas of the application—from the particular actions you'd like to trigger, to the facial gestures you want to use, to how sensitive the application is to your facial gestures," Google explained. "So whatever your facial mobility, you can use Project Activate to express yourself."

Recommended by Our Editors

Sony's New Lip-Reading Technology Could Boost Accessibility—or Invade Privacy

'Tap to Alexa' Brings Greater Accessibility to Echo Show

Apple Proposes Accessibility Emoji, Including Service Dog

Download the Project Activate app—available in English in the US, UK, Canada, and Australia—from Google's Play store.

Lookout

In 2019, Google launched Lookout for people who are blind or low-vision. Using a smartphone camera, it recognizes objects and text in the physical world and announces them aloud. Several modes made a variety of everyday tasks easier, from identifying food products to describing what's on the table in front of you.

An update to Documents mode, for example, lets your phone read handwritten text, reciting sticky notes and birthday cards scrawled in Latin-based languages. Plus, the expanded Currency mode now identifies Euros and Indian Rupees, in addition to US dollars.

Google Assistant Now Lets You Tighten Your Nikes With Your Voice