Google Rolls Out New AI-Powered Accessibility Features for Pixel, Android Devices

Google introduced the Guided Frame feature in Pixel Camera which will provide spoken assistance to get the face in the frame.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 14 August 2024 17:20 IST

Google Rolls Out New AI-Powered Accessibility Features for Pixel, Android Devices

Photo Credit: Google

Google has also added support for seven new languages in Live Captions feature

Highlights

Google has updated the Magnifier app with new AI features
Magnifier app can now look for specific text in real-world surroundings
New Live Transcribe features have been added for foldable devices

Google introduced new artificial intelligence (AI) accessibility features for Pixel smartphones and Android devices on Tuesday. There are four new features, out of which two are exclusive to Pixel smartphones, and two have a wider availability across Android devices. These features are aimed at people with low vision and vision loss, people who are deaf, and those with speech impairment. The features include Guided Frame, new AI features in the Magnifier app, as well as improvements in the Live Transcribe and Live Captions feature.

Google Adds AI-Powered Accessibility Features

In a blog post, the tech giant highlighted that it is committed to working with the disability community and is looking to bring new accessibility tools and innovation to make technology more inclusive.

The first feature is dubbed Guided Frame, and it is exclusive to the Pixel Camera. The feature provides spoken assistance to users to help them place their faces within the frame and find the right camera angle. This feature is aimed at those with low vision and vision loss. Google says the feature will prompt users to tilt their faces up or down, or pan left to right before the camera automatically captures the photo. Additionally, it will also tell the user when the lighting is inadequate so they can find a better frame.

Here Are All the New AI Features Announced for Google Pixel Devices

Earlier, the feature was available through Android's screen reader TalkBack, but Guided Frame has now been placed within the camera settings.

Another Pixel-specific feature is an upgrade to the Magnifier app. The app was introduced last year and it allowed users to use the camera to zoom into real-world surroundings to read sign boards and find items on a menu board. Now, Google has used AI to let users search for specific words in their surroundings.

This will allow them to look for information about their flight at the airport or find a specific item at a restaurant as the AI will auto-zoom on the word. Additionally, a picture-in-picture mode has been added which shows the zoomed-out image in a smaller window while the searched word is locked into the bigger window. Users can also switch the lenses of the camera for specific purposes. The app also supports the front-facing camera so it can be used as a mirror.

US Considers Breaking Up Google in Rare Antitrust Move

Live Transcribe is also getting a new upgrade which will be supported only on foldable smartphones. In dual-screen mode, it can now show each speaker their own transcriptions while using the feature. This way, if two people are sitting across a table, the smartphone can be placed in the middle and each half of the screen will show what that person has said. Google says it will make it easier for all participants to follow the conversation better.

The Live Captions feature is also getting an upgrade. Google has added support for seven new languages — Chinese, Korean, Polish, Portuguese, Russian, Turkish, and Vietnamese — to Live Captions. Now, whenever the device plays a sound, users will be able to get a real-time caption for it in these languages as well.

Google Pixel 9 Series to Go on Sale in India at These Walk-in Centres

These languages will also be available on-device for Live Transcribe, Google said. This takes the total number of languages to 15. When transcribing these languages, users will no longer require a connection to the Internet. However, if connected to the Internet, the feature works with 120 languages.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Google, Accessibility Features, AI, Artificial Intelligence, Pixel

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More