Brain2Qwerty

Brain2Qwerty Decoding Sentences from Non-Invasive Recordings of the Brain. Brain2Qwerty v2 A non-invasive brain-computer interface for online sentence generation. Typed: the girl in front of me was very happy yesterday Decoded: the girl in front of me was very happy yesterday DOCUMENTATION RECONSTRUCTS THE EXACT SENTENCE SOMEONE TYPED Project website Meta blog PUBLICATIONS Non-invasive decoding of typed sentences from human brain activity (Nature Neuroscience, 2026). Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings (preprint, GitHub Code Issues Pull requests 6 More Watch 7 679 stars 101 forks 7 watching 3 Branches 0 Tags Activity Custom properties Public repository main Code jarod12312 4 days ago .github brain2qwerty_v1 Brain2Qwerty Paper v1 Paper v2 Code Brain2Qwerty v2 A non-invasive brain-computer interface for online sentence generation. SCROLL TO READ MORE STRAIGHT FROM THEIR BRAIN ACTIVITY NO IMPLANT REQUIRED THIS ISN'T DECODING VAGUE INTENT IT'S RECOVERING THE ACTUAL WORDS KEYSTROKE BY KEYSTROKE FROM A PARTICIPANT SITTING IN A MEG OR EEG SCANNER THE CHALLENGE Every year, thousands of people lose the ability to speak after a stroke, an accident, or a brain disorder. While communication can be restored with brain implants on the motor cortex, such neuroprostheses require open-brain surgery. Today, we introduce Brain2Qwerty v2: a model to decode natural sentences from non- invasive magnetoencephalography (MEG) recordings. ARCHITECTURE For this, we built off the Brain2Qwerty v1 architecture released last year and recently accepted at Nature Neuroscience. Brain2Qwerty v1 consisted of predicting keystrokes from MEG brain activity patterns recorded at BCBL, but could not work in real time because it needed the timing of every keypress. Brain2Qwerty v2 overcomes this limit and generates the sentences directly from a continuous recording of brain activity. This new model now combines three hierarchical modules to jointly improve the decoding of letters, words, and sentences. Conformer MEG Character detection Aligner word embeddings PERFORMANCE Brain2Qwerty v2 can decode complete and meaningful sentences solely from MEG signals of healthy volunteers, and reaches up to 78% word accuracy for the best participant. Typed sentences Word accuracy 2,200 40% 48% 2,200 61% 78% mean best participant EXPLORE RESULTS Explore examples of the sentences decoded by Brain2Qwerty v2 from the brain recordings of three subjects. THAT VIOLINIST PLAYS MUSIC THAT TOUCHES MY HEART MORE ACCURACY S0 the band inside plays music that touched my heart 54% S6 the band inside plays music that touched my heart 58% S2 the pianist played music that touched my heart 58% REMAINING CHALLENGES Two major challenges remain before this method can be hoped to transfer to the clinic. First, decoding performance is not yet good enough for everyday use; this method still makes too many word-level or character-level errors to be practical. Still, our results follow a scaling law: the more data used for training, the better the decoder, without -- at least for now -- a detectable performance plateau. We are hopeful that larger datasets will thus further improve decoding and reduce the remaining gap with invasive neuroprostheses. Second, the MEG device used in the presented study consists of a large scanner -- i.e. a setup inconvenient to most patients. However, we are optimistic about this challenge: A CHARACTER-LEVEL LANGUAGE MODEL CLEANS UP THE OUTPUT wearable, and could thus be more open science As always, we're thrilled to share our publications, code and data: Brain2Qwerty v1 Nature Neuroscience Brain2Qwerty v2 Meta preprint Full Code for v1 and v2 GitHub v1 Dataset collected by BCBL: HuggingFace v2 Dataset collected by BCBL: embargoed until journal publication.