Brain Computer Interface Turns Mental Handwriting Into Text on Screen, Could Help People With Paralysis

The system displayed the mental handwriting on-screen in real-time, meaning as and when the participant imagined them.

By Edited by Gadgets 360 Newsdesk | Updated: 13 May 2021 16:09 IST

Brain Computer Interface Turns Mental Handwriting Into Text on Screen, Could Help People With Paralysis

Photo Credit: F. Willett et al./Nature 2021/Erika Woodrum

In recent years, the research team has deciphered the neural activity related to speech

Highlights

Researchers have decoded brain activity related to writing by hand
In the experiment, the participant could type 90 characters per minute
The innovation could help people with paralysis type without using hands

A team of researchers at Stanford University have, for the first time, decoded brain activity related to writing by hand. To further explore this, the researchers collaborated with a participant with paralysis who had sensors implanted in his brain. The team then used an algorithm that identified the letters the participant imagined and attempted to write. The system then displayed the same text on-screen in real time, meaning as and when the participant imagined them. Krishna Shenoy, the study co-author and a Howard Hughes Medical Institute Investigator at the university, said the innovation, with more development, could well help people with paralysis rapidly type without using their hands.

In the experiment, the participant could type 90 characters per minute, which was more than double the typing record set by the previous “brain-computer interface,” the researchers said. Shenoy jointly supervised the study with Jaimie Henderson, a Stanford neurosurgeon.

Jose Carmena, a neural engineer at the University of California, Berkeley, who was not involved in the research says such technology had the potential to help people with all kinds of disabilities. "It's a big advancement in the field," he says.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

An injury or a disease may take away a person's ability to move but the brain's neural activity connected with walking, holding a cup of coffee or tea, or even speaking a sentence remains. And this activity is exactly what the scientific community can tap into to enable people with paralysis or amputations to express their thoughts or even carry out an activity.

In recent years, Shenoy's team has deciphered the neural activity related to speech, hoping to reproduce it. Not just that, his team had also developed a technology to help participants move a cursor on the screen with help of implanted sensors.

This helped them type at least 40 characters on the screen by pointing and clicking on letters. Frank Willett, a neuroscientist in Shenoy's group, said that they wanted to find "new ways of letting people communicate faster," adding that he could try "something different" also motivated him.

Krishna Shenoy's team now envisages a more comprehensive system that — besides handwriting for text entry — includes point-and-click navigation and even attempted speech decoding. "Having those two or three modes and switching between them is something we naturally do," he said.

For this, his team next plans to collaborate with someone who cannot speak. For example, someone with amyotrophic lateral sclerosis, a degenerative neurological disorder causing the loss of movement and speech.

Henderson says the new system, when operation, could help people suffering from paralysis caused by several conditions, including brain stem stroke.

Jean-Dominique Bauby, the author of the book The Diving Bell and the Butterfly, suffered from the same, but he was able to write a beautiful book by "selecting characters painstakingly, one at a time, using eye movement," Henderson said.

"Imagine what he could have done with Frank's handwriting interface!"

Is Mi 11 Ultra the best phone you can buy at Rs. 70,000? We discussed this on Orbital, the Gadgets 360 podcast. Orbital is available on Apple Podcasts, Google Podcasts, Spotify, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.