Google Launches Accessibility-Focused Project Gameface on Android to Offer Hands-Free Navigation

Project Gameface was first introduced in 2023 as an open-source hands-free gaming mouse that works with facial expressions.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 17 May 2024 17:39 IST

Google Launches Accessibility-Focused Project Gameface on Android to Offer Hands-Free Navigation

Photo Credit: Google/Project Gameface

Project Gameface tracks head movements and facial expressions with the smartphone’s camera

Highlights

Project Gameface adds a virtual cursor to Android’s interface
The project will let developers build apps with accessibility features
Project Gameface collaborated with the Indian organisation Incluzza

Google has expanded its Project Gameface, an open-source project aimed at making tech devices more accessible, to Android, and now it can be used to control the smartphone interface. The project was first introduced during Google I/O 2023 as a hands-free gaming mouse that can be controlled using head movements and facial expressions. These were designed for those who suffer from physical disabilities and cannot use their hands or voice to control devices. Keeping the functioning same, the Android version adds a virtual cursor to allow users to control their device without touching it.

In an announcement made on its developer-focused blog post, Google said, “We're open-sourcing more code for Project Gameface to help developers build Android applications to make every Android device more accessible. Through the device's camera, it seamlessly tracks facial expressions and head movements, translating them into intuitive and personalised control.” Further, the company asked developers to use the tools to add accessibility features to their apps as well.

Project Gameface collaborated with the Indian organisation Incluzza which supports people with disability. Using the collaboration, the project learned how its technologies can be expanded to different use cases such as typing a message, looking for jobs, and more. It used MediaPipe's Face Landmarks Detection API and Android's accessibility service to create a new virtual cursor for Android devices. The cursor moves following the user's head movement after tracking it using the front camera.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

The API recognises 52 facial gestures including raising an eyebrow, opening the mouth, moving the lips, and more. These 52 movements are used to control and map a wide range of functions on the Android device. One interesting feature is dragging. Users can use this to swipe the home screen. To create a drag effect, users will have to define a start and end point. It can be something like opening the mouth and moving the head, and once the endpoint is reached, closing the mouth again.

Notably, while this technology has been made available on GitHub, it is now up to developers to build apps using this option to make it more accessible to users. Apple also recently introduced a new feature that uses eye-tracking to control the iPhone.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Google, Project Gameface, Accessibility, Android

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More