Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Using intracranial EEG to study the effect of prosody on attention and linguistic encoding during naturalistic listening.
Poster Session A - Sandbox Series, Thursday, October 24, 10:00 - 11:30 am, Great Hall 3 and 4
This poster is part of the Sandbox Series.
Eleonora Beier1, Laura Gwilliams2; 1University of California, Davis, 2Stanford University
One of the primary goals of language is to exchange information. The amount of information varies over the course of an utterance – some moments are very informative, while others are less informative. Prosody is an important cue to informativity: words carrying novel, essential information are focused through greater prosodic prominence (e.g., higher vowel intensity and duration) and through pitch accents. Previous studies suggest that listeners track prosodic structure over time, pre-allocating attention and processing resources to the predicted timing of focused information. However, most previous literature has employed constrained psycholinguistic paradigms. It is unclear to what extent listeners track prosodic features to predict the timing of upcoming focused words during naturalistic listening, and the underlying neurocognitive mechanisms are not well understood. In this ongoing study, we overcome these limitations through the analysis of a rich intracranial EEG (iEEG) dataset from neurosurgical patients listening to naturalistic stories. Patients listen to 12 snippets of audiobooks, for a total duration of ~1 hour, while iEEG is collected via depth electrodes. We annotate the speech samples for a variety of linguistic features, including the timing of pitch accents indicating prosodic focus, and continuous measures of prosodic prominence (e.g., intensity). We train a classifier to decode the presence or absence of a pitch accent based on the continuous prosodic features of the preceding speech; based on the results of the classifier, we identify pitch accents whose presence and timing was highly predictable from preceding prosody (resulting in high decodability) from those that were not predictable (resulting in low decodability). We then compare neural activity over a time window leading up to predictable vs unpredictable pitch accents. First, we expect to find greater attention pre-allocation leading up to predictable, compared to unpredictable pitch accents, measured through modulations in the power of alpha oscillations. Second, we test whether this greater attention results in deeper encoding of linguistic features. Neural representations of linguistic units (e.g., phonemes, syntactic category) can be decoded from high-gamma (70-150 Hz) neural activity. We hypothesize that linguistic units at different levels of processing are more decodable for words focused through pitch accents, and even more so for pitch accents predictable from the preceding prosodic context. Overall, this study takes a novel approach to the study of prosodic focus, addressing the neural mechanisms by which listeners prepare to process more informative moments during naturalistic language comprehension.
Topic Areas: Prosody, Computational Approaches