Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Slide Session C
Saturday, October 26, 9:30 - 10:30 am, Great Hall 1 and 2
Chair: Xin Sun, University of British Columbia
Talk 1: Investigating the neurobiology of human prosody perception using (neuro)genomic and cross-species approaches
Srishti Nayak1, Alyssa Scartozzi2, Daniel Gustavson3, Cyrille Magne4, Nicole Creanza2, Jennifer Below1, Reyna Gordon1; 1Vanderbilt University Medical Center, 2Vanderbilt University, 3University of Colorado Boulder, 4Middle Tennessee State University
Exciting advances have been made in the (neuro)genetics of language in the past few years, showing that individual differences in language abilities are substantially driven by genetic factors. This includes genome-wide investigations of word reading, phonemic awareness, nonword repetition, dyslexia, and voice pitch variability amongst others. Currently there is almost a complete lack of behavioral genetics or genomics work on prosody perception, despite its known relevance to reading development, adult literacy, language disorders, and mental health disorders. In fact, prosody is one of the most underrepresented traits in language research more broadly. Prosody is defined as the pattern of stress and intonation in speech and is crucial for both speakers and listeners during human communication. For listeners, speech prosody aids in word boundary identification, the understanding of speaker emotion, distinguishing questions and statements, and parsing syntax. For speakers, speech prosody attracts listener attention to key information for efficient comprehension or conveys intentions (e.g. sarcasm). Individual differences in prosody perception skills are also associated with reading outcomes in both children and adults, and disruptions in prosody are a key feature of certain language and learning disorders (e.g. dyslexia), and mental health conditions (e.g. depression). Understanding the (neuro)genetics and evolutionary history of prosody would allow us to characterize another piece of the puzzle in the neurobiological processes that support human language abilities. Here, we report on the first major effort to investigate the (neuro)genetics of prosody perception. Participants mailed in their saliva for genotyping, and completed a previously validated internet-based prosody test: Test of Prosody via Syllable Emphasis (Nayak et al., 2022; Gustavson et al., 2023). First, we tested associations between prosody perception scores in our sample, and genetic predispositions for word reading (which is known to be behaviorally associated with prosody) in N = 1698. Results showed that individual differences in genetic predispositions for word reading explained significant variability in prosody perception scores (β = 0.109, p-value = 2.23e-5) controlling for age, sex, and commonly used genetic controls. Next, we conducted the first genome-wide association study (GWAS) of prosody in N = 1501, which revealed that one common genetic variant (SNP) is significantly associated with prosody perception (p = 8.39e-10).The SNP occurs in, or genetically upstream of, gene TMEM108, involved in brain and central nervous system development (e.g., neuronal migration; fetal brain development), function (e.g., cellular response to BDNF), and structure (e.g., cerebellum). Last, we investigated the comparative biology of prosody perception in humans and vocal learning in songbirds to explore the evolutionary history of the neurobiology of human prosody abilities. Gene-set enrichment analyses showed that genes expressed in songbird brain Area X (a key song learning brain area homologous to human basal ganglia) were overrepresented in human prosody-related genes (p < 7.14e-3). This work highlights a significant step forward in our understanding of the (neuro)genetics of prosody perception traits, which adds to complementary literature from EEG and fMRI studies which have previously revealed details about neural mechanisms of prosody perception states.
Talk 2: A speech neuroprosthesis that captures phonemic and paralinguistic elements of attempted speech
Sergey Stavisky1, Nicholas Card1, Maitreyee Wairagkar1, Xianda Hou1, Aparna Srinivasan1, Tyler Singer-Clark1, Carrina Iacobacci1, Leigh Hochberg2, David Brandman1; 1University of California, Davis, 2Brown University, VA Providence Healthcare, Massachusetts General Hospital
It has historically been rare to study the neurobiology of speech and language at the level of the population activity of individual neurons due to the invasive nature of collecting such measurements. However, ongoing early feasibility studies of speech restoration neuroprostheses represent opportunities to study the cortical basis of speech and language while restoring rapid, accurate, and naturalistic communication to people with paralysis. As part of the BrainGate2 clinical trial (ClinicalTrials.gov identifier: NCT00912041), we enrolled a participant, ‘T15’, who is a left-handed man in his 40s living with severe dysarthria due to ALS. We chronically placed 64-microelectrode Utah arrays into his left precentral gyrus, with two arrays nominally in ventral premotor cortex (area 6v), one array in primary motor cortex (area 4), and one array in middle precentral gyrus (area 55b). We found that these cortical areas all encoded attempted speech, with the strongest tuning in ventral-most premotor and middle precentral gyrus. We could train machine learning algorithms to decode what words he attempted to say with 97.5% accuracy using a 125,000-word vocabulary (“Brain-to-Text”), with high communication restoration performance persisting for over 300 days as of the time of this abstract submission. The participant has used the neuroprosthesis in his own home to converse with his family and friends, write emails and messages, and video call with colleagues for over 500 cumulative hours, up to 14 hours a day. Device connection and initiation can be performed independently by his care partners. We also were able to synthesize a digital voice instantaneously and directly from neural activity (“Brain-to-Voice”), and to detect paralinguistic features such as added stress of specific words in a sentence, changing a statement into a question, or attempting to speak at different volumes. Examining the neural ensemble dynamics during this attempted speech, we observed that the dynamics strongly reflected the phoneme being produced at each given moment, but that there was also preparatory activity that preceded voice onset which carried both phonemic and paralinguistic information. Lastly, we detected potentially higher-level language-related activity in the form of error signals that occurred after the neuroprosthesis displayed the wrong word. Not only could we decode whether or not the wrong word appeared from this error-related neural modulation, but we could also detect whether the incorrect output was a pseudoword, homophone, one-phoneme different word, or synonym. Altogether, these results indicate that speech neuroprosthesis clinical trials represent a unique and useful opportunity to study the neural basis of speaking – and perhaps even of language – at the resolution of action potentials.
Talk 3: Neural correlates of speech segmentation in typical adults
Panagiotis Boutris1, Alissa Ferry2, Perrine Brusini1; 1University of Liverpool, 2University of Manchester
Words are the building blocks of language, but are not obvious in the speech stream. When learning to segment speech, one needs to integrate both transitional probabilities (TPs; Saffran et al., 1996) and prosodic markers, such as stress (Nazzi et al., 2006). Recent research has explored how these two cues are integrated (Cunillera et al. 2006; 2008). Yet it remains unclear how they are tracked and weighted by the speech-processing system. Here, we aim to capitalize on the brain’s ability to synchronize with different speech units (Giraud and Poeppel, 2012; Buiatti et al. 2009) and investigate the neural correlates of TP, stress and their combination. Twenty young adults were presented with artificial language streams, comprised of 6 trisyllabic words, whilst their brain activity was recorded by a high-density EEG net. Six conditions were created: a TP-only stream, following the model of Saffran et al; a stress-only, where all words were concatenated in the same order (TP=1) and stress was placed on the first syllable; two mixed-cue streams where TPs were combined with stress either on the first syllable (a coherent cue for English speakers) or on the last syllable (an incongruent case in English); a random syllable, and a random stress stream, where TP is uninformative (TP=0.2-0.5) and stress is randomly assigned to one of the 3 syllables respectively. Time-frequency analysis was used to extract power and phase at the frequencies of word and syllable onset. At the end of each stream presentation, behavioural responses where collected using a forced-choice task, opposing stream-words, and part-words made from the last syllable of the stream-word and the two first syllables of another. All test stimuli were played without any stress. It is worth noting here that, in the incongruent condition, if English adults rely more on stress than TPs, participants should systematically choose the part-word, as this structure corresponds to the trochaic pattern. In the frequency domain, the power should inform us whether the brain can follow the frequency of occurrence of each unit (word and/or syllable), while the phase should allow us to see if the onset of the units detected by the brain align with the onset of the units in each condition. Behavioural results showed better segmentation and recall for the TP-with-stress-on-1st-syllabe condition, followed by stress-only. TP-only and TP-with-stress-on-3rd-syllable remained at chance. However, power showed a clear online tracking of word onset for all the cues, which did not seem to follow the pattern of our behavioural results. A subsequent look at phase, however, revealed a more nuanced difference between cues: the conditions of TP-with-stress-on-1st-syllable and Stress-only had a much closer-to-zero-degrees phase as compared to TP-only, following, thus, more reliably the behavioural data. In addition, although power at word-onset frequency was significant for the TP-with-stress-on-3rd-syllabe, participants were consistently out of phase, in line with our behavioural results. We conclude that, although power is an important indicator of cortical tracking of word-onset, phase might be a more reliable measurement of more precise and, possibly, altogether better segmentation.
Talk 4: The Neural Basis of Decline in Written Production: Evidence from Chinese Handwriting
Xufeng Duan1, Yang Yang2, Junjun Li2, Shuyi Wu1, Zhenguang Cai1,3; 1The Chinese university of Hongkong, 2Institute of Psychology of the Chinese Academy of Sciences, 3Brain and Mind Institute, The Chinese university of Hongkong
Background: People sometimes forget how to spell a word (though still being able to recognize it) as a result of attrition in the orthographic production processes. This phenomenon, known as character amnesia, is particularly common in opaque writing systems such as Chinese, where most characters do not have reliable phonology-to-orthography conversion. However, little is known about the underlying neural time-course of character amnesia and the neural substrates underlying the orthographic attribution. Our study aims to address these issues using EEG and fMRI. Methods: We conducted an EEG experiment to unravel the physiological time course of character amnesia and an fMRI experiment using the same group of participants (30 Chinese-speaking participants, 15 females, 20 – 24 years) to ensure the consistency of our findings across different neuroimaging modalities. We selected, from two handwriting databases, 360 Chinese characters (120 for fMRI; 240 for EEG), which were shown to be susceptible to character amnesia (amnesia rate mean = 20.9%). Both experiments made use of a writing-to-dictation task, where participants handwrote on a tablet a character according to a dictation prompt (e.g., 雪糕的雪, meaning the character 雪in the word 雪糕) while lying inside the scanner and being able to see their handwriting on a screen. The EEG experiment had a similar task procedure except that participants were instructed to write Chinese characters after a two-second delay following the dictation prompt (to reduce hand movement artefacts). In both experiments, we had two handwriting-outcome-dependent conditions: successful handwriting (i.e., when participants correctly wrote down a character) and character amnesia (i.e., when participants failed to handwrite a character). Results: For the EEG Experiment, time-frequency analyses revealed two distinct cognitive stages in character amnesia compared to successful handwriting. There was a significant increase in total power within the theta and alpha bands (4 -12 Hz) around 400 - 500 ms after handwriting onset, reflecting the difficulty in orthographic retrieval in character amnesia compared to successful handwriting. This was followed by an increase in theta band (4 - 8 Hz) 1000 -1650 ms after handwriting onset, which we argue reflects the continued attempt of orthographic retrieval in the amnesia state. In the fMRI experiment, comparing character amnesia with successful handwriting revealed stronger activations in the left inferior frontal gyrus (IFG; indicating difficulty in orthographic retrieval), left supplementary motor area (SMA; related to motor planning), left superior temporal gyrus (STG; associated with phonological processing), and bilateral cuneus regions (involved in visual processing). We also used the brain clusters activated during the character amnesia condition as ROIs for network connectivity analyses. There was stronger connectivity within language production networks (e.g., the left IFG and bilateral SMA, STG, precentral gyrus, postcentral gyri, and putamen) in character amnesia compared to successful handwriting. Furthermore, there was stronger connectivity between the STG and fusiform gyrus in successful handwriting compared to character amnesia, highlighting its role in retrieving orthographic information. Our study was the first to address the “when” and the “where” questions regarding the neural processes underlying orthographic attrition in handwriting.