Meta has upgraded its brain-computer interface research today by introducing the new Brain2Qwerty v2 tool. This is the upgraded version which translates brainwave activity into full sentences in real time from raw brain activity. There is no need for surgery or any type of implant. The only equipment needed is a scanner placed on the individual’s scalp (the user). An artificial intelligence system will perform the translation.
This is currently Meta’s best-performing tool for brain-to-text interfaces, achieving an average of 61% word accuracy across all users. But one user reached a remarkable 78% accuracy level. For reference, previous attempts at developing non-invasive brain-computer interfaces could achieve only single-digit levels of average word accuracy.
What is a brain computer interface and why does Brain2Qwerty v2 matter?
A magnetoencephalography (MEG) device can measure brain activity. How? By means of measuring the electromagnetic signals produced by the brain through a magnetic field. The MEG device sits very close to the scalp, allowing it to detect very precise electromagnetic signals produced by the brain activity of each volunteer typing on a keyboard and generating the data.
Meta trained an AI to provide a direct way to decode the signals collected to determine which signals belong to which user’s keystrokes. They trained a very large AI language model using all the 22,000 sentences in this experiment and trained the AI using the language model to predict the gaps and/or noisy signals in the demographic profile created from the signal data.
Training Sessions:
- Each of the nine volunteer participants recorded for approximately 10 hours while typing.
The combination of raw signal data and the use of AIs trained to predict language provide significantly greater levels of accuracy compared with previous attempts.
How does Brain2Qwerty v2 actually work?
The major part of this article tells us how much better the new brain-to-text devices perform. Devices that were non-invasive and worked with the brain to convert thoughts and movements into words were historically around 8% accurate. The new version of Brain2Qwerty is approximately 61% capable of generating correct words from the thoughts of all participants tested.
| Approach | Word Accuracy | Surgery Required |
|---|---|---|
| Earlier noninvasive methods | ~8% | No |
| Brain2Qwerty v2 (average) | 61% | No |
| Brain2Qwerty v2 (best participant) | 78% | No |
| Invasive BCI methods (implants) | Higher, but harder to scale | Yes |
Some invasive devices (brain-computer interfaces) have in the past produced high accuracy results using a direct electrical interface with the brain. They require physical access to the brain through surgery, thereby limiting the number of potential users. Brain2Qwerty is being offered as an alternative way to use brain-computer interfaces without needing surgical access.
What are the real accuracy numbers behind Brain2Qwerty v2?
Word accuracy is the primary metric reported by Meta. It refers to how well the tool decodes the spoken language. It was found that on average has a word decoding accuracy of 61 out of 100. As an illustrative example, the top performing volunteer had less than 1 word error for every 2 sentences that were decoded.
In addition, Meta provided information about how to improve the decoding accuracy of models as more data is put into training (or as more recordings are provided to the model). Basically this means that the more recordings you provide to the model, the more successful it will become. And that the difference in performance between an invasive method and a non-invasive method will become smaller with the more recordings you give to the model.
Who can benefit from this brain computer interface research?
People who have been unable to communicate orally due to the following are part of the primary target audience. For Meta’s non-invasive approach to brain-computer interfaces:
- Stroke
- Traumatic brain injury
- Neurological disorders such as ALS or Locked-In Syndrome
- Brain lesions that prevent normal speech patterns.
For these individuals, non-invasive options are critically important. Although surgical BCIs are available and are functional, not everyone qualifies for surgical intervention. Therefore, there is currently no way to implement surgical solutions on a large scale, which would support millions of users. This barrier is completely eliminated with a scanner-based brain/txt (BTT) system.
What’s next for brain to text technology?
Meta is showing transparency by sharing both versions of training code from its brain-to-keyboard system. Along with this, the Basque Center on Cognition, Brain, and Language, one of Meta’s research partners, is sharing its training dataset used in v1.
The idea behind these shared resources is to help accelerate the rate of advancement in neuroscience generally. The second version of brain-to-keyboard relates to other projects conducted by Meta involving brain research, such as Tribev2 (a foundational model for brain research) and an extensive system for processing brain datasets called NeuralSet. Additionally, Meta has pooled funds for making information about the brain more accessible through its Digital Brain Project. This investment also reflects the company’s broader AI ambitions, which have evolved alongside Meta’s shifting open-source strategy.
Key takeaways
- Brain2Qwerty v2 is Meta’s latest brain-computer interface (BCI) that is the most accurate non-invasive BCI created to date.
- As a non-invasive BCI, it has the ability to decode brain signals into words or complete sentences with an average of 61% accuracy and up to 78% accuracy (for the best participant).
- The way that Brain2Qwerty v2 does this is by using a magnetoencephalography (MEG) scanner that does not require any surgical procedures along with an AI model that recognizes the relationship between brain activity, words, and how they form sentences.
- The training of Brain2Qwerty v2 consisted of more than 22,000 sentences from nine volunteer participants.
- Meta has also announced that it would make the code for Brain2Qwerty v2 freely available to everyone (open-source).
- Brain2Qwerty v2 represents cutting-edge research in the development of BCIs, and could potentially result in a future where patients who can no longer speak due to injury or stroke will be able to communicate without the need for surgical alterations.
