Presentation

Search Abstracts | Symposia | Slide Sessions | Poster Sessions

Representation of verbal thought in motor cortex and implications for speech neuroprostheses

Poster Session C, Friday, October 25, 4:30 - 6:00 pm, Great Hall 3 and 4

Erin M. Kunz¹, Benyamin Meschede-Krasa¹, Foram Kamdar¹, Samuel R. Nason-Tomaszewski,², Nicholas S. Card³, Brandon Jacques², Payton Bechefsky², Nick Hahn¹, Carrina Iacobacci³, Leigh Hochberg^4,5,6,7, David M. Brandman³, Sergey Stavisky³, Nicholas AuYong², Chethan Pandarinath², Shaul Druckmann¹, Jaimie Henderson¹, Francis R. Willett^1,8; ¹Stanford University, ²Emory University, ³University of California, Davis, ⁴Brown Universty, ⁵VA Medical Center, Providence RI, ⁶Massachusetts General Hospital, Boston MA, ⁷Harvard Medical School, ⁸Howard Hughes Medical Institute

Brain-Computer Interfaces (BCIs) have recently demonstrated a viable path forward toward restoring speech to people who have lost the ability due to paralysis [1-2]. Increased electrode count and optimized decoding algorithms have yielded a level of Signal-to-Noise-Ratio (SNR) and stability necessary for long-term use of a conversational open-ended speech BCI [3]. However, as the performance of these intracortical systems has increased rapidly in decoding speech, so too has concern regarding their potential to decode private verbal thought [4]. In three research participants, each with microelectrode arrays placed along the precentral gyrus of the motor cortex, we studied different types of verbal behavior including: attempted vocalized speech, mimed speech, listening to speech, silently reading text, and three types of auditory or motor imagery of speech (“inner speech.”), We found that all behaviors, including the three inner speech conditions, showed distinct representations of individual words, with overt movement conditions generally showing the strongest modulation, as expected. The strongest modulation for perceived and inner speech was found in Area 55b and the inferior region of Area 6v (as defined by the Human Connectome Project cortical parcellations [5]). Notably, a simple classifier could decode neural signals during listening and speech to distinguis between 7 words above chance in all participants, with up to 94% accuracy for listening and 71% for inner speech. Additionally, we found these word-level representations were largely shared across perceived, inner and produced speech, differing primarily in relative strength of modulation from rest. We also demonstrate the first online speech neuroprosthesis for decoding continuous inner speech from three dysarthric individuals, achieving a word error rate as low as 14% for a 50 word vocabulary. All participants found inner speech easier, more comfortable, and aesthetically preferable as compared to actually attempting to vocalize, as is the typical paradigm for state-of-the-art speech neuroprostheses. Finally, to understand whether verbal thought might be decodable when it occurs naturally, we ran a follow-up research session that elicited the natural use of inner speech as a memory aid in a non-speech task (drawing symbolically-cued sequences). We found that the content of the inner speech in this task was discernible using a decoder trained on attempted vocalized speech. This brings to light some important questions regarding decoding intention and privacy that future design of speech neuroprostheses must take into account. 1. Willett, F.R., Kunz, E.M., Fan, C. et al. A high-performance speech neuroprosthesis. Nature 620, 1031–1036 (2023). 2. Metzger, S.L., Littlejohn, K.T., Silva, A.B. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature 620, 1037–1046 (2023). 3. Card, N.S. et al. An accurate and rapidly calibrating neuroprosthesis. Medrxiv (2023). 4. Soldado-Magraner, J. et al. Applying the IEEE BRAIN neuroethics framework to intra-cortical brain-computer interfaces. J. Neural Eng. 21 022001 (2024).

Topic Areas: Speech Motor Control, Language Production