Mind-reading technology transforms thoughts into text

In an unprecedented scientific achievement, experts have developed a system that is capable of reading human thoughts and converting them into text. 

The technology was designed by the GrapheneX-UTS Human-centric Artificial Intelligence Centre at the University of Technology Sydney (UTS).

Human-machine interaction

The portable, non-invasive system stands as a significant milestone, offering transformative communication solutions for individuals impaired by conditions like stroke or paralysis.

However, the potential applications of this mind-reading technology extend beyond aiding speech-impaired individuals. 

The research paves the way for fluid human-machine interactions, potentially enabling seamless control of devices such as bionic arms or robots. This could ultimately lead to a new era in assistive technology.

Spotlight paper

The pioneering study has garnered international recognition, being featured as the spotlight paper at the prestigious NeurIPS conference in New Orleans on December 12, 2023. This event is renowned for showcasing the latest research in artificial intelligence and machine learning.

The project was led by Professor CT Lin, director of the GrapheneX-UTS HAI Centre, in collaboration with Yiqun Duan and Jinzhou Zhou, both PhD candidates at the UTS Faculty of Engineering and IT. 

EEG-based approach

The core of this technology lies in its ability to record and decode brain activity through an electroencephalogram (EEG) cap. Participants in the study silently read texts while the cap captured their brain’s electrical activity. 

This EEG data is then processed by an AI model named DeWave, a creation of the UTS team. DeWave distinguishes itself by translating EEG signals into coherent words and sentences, learning from extensive EEG datasets.

Pioneering effort 

“This research represents a pioneering effort in translating raw EEG waves directly into language, marking a significant breakthrough in the field,” said Professor Lin.

“It is the first to incorporate discrete encoding techniques in the brain-to-text translation process, introducing an innovative approach to neural decoding. The integration with large language models is also opening new frontiers in neuroscience and AI.”

A step beyond existing technologies

Current methods for translating brain signals to language, like Elon Musk’s Neuralink, require invasive procedures or bulky, expensive equipment like MRI machines. 

These methods often rely on additional aids such as eye-tracking, which limits their practicality. The UTS innovation, however, operates with or without eye-tracking, making it more versatile and user-friendly.

Unprecedented versatility 

Conducted with 29 participants, the research promises greater robustness and adaptability compared to previous technologies tested on fewer individuals. 

The UTS technology has a significant advantage in its versatility, as EEG waves vary among individuals.

Meaningful results 

“The model is more adept at matching verbs than nouns. However, when it comes to nouns, we saw a tendency towards synonymous pairs rather than precise translations, such as ‘the man’ instead of ‘the author,'” said Duan.

“We think this is because when the brain processes these words, semantically similar words might produce similar brain wave patterns. Despite the challenges, our model yields meaningful results, aligning keywords and forming similar sentence structures.”

Future potential

While the mind-reading technology currently excels more in matching verbs than nouns, it demonstrates an impressive ability to align keywords and form coherent sentence structures. 

The current translation accuracy is around 40 percent on the BLEU-1 scale, with aspirations to reach the efficacy of traditional language translation or speech recognition systems, which hover around 90 percent.

The research expands upon previous brain-computer interface technology developed by UTS in association with the Australian Defence Force that uses brainwaves to command a quadruped robot.

Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates.

—-

Check us out on EarthSnap, a free app brought to you by Eric Ralls and Earth.com.