Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Disfluencies reduce the effect of uh word surprisal during narrative comprehension
Poster Session C, Friday, October 25, 4:30 - 6:00 pm, Great Hall 3 and 4
Laura Giglio1, Peter Hagoort2,3, Eleanor Huizeling2; 1University of South Carolina, 2Max Planck Institute for Psycholinguistics, Netherlands, 3Donders Institute for Brain, Cognition and Behaviour, Netherlands
Disfluencies in speech most frequently occur before the production of longer and more complex speech content. There is some evidence to suggest that listeners use the distribution of disfluencies in the comprehension of speech to inform their predictions. A disfluency in speech may prepare the listener to hear a word that is more challenging to produce, such as a lower frequency word or a word with a higher surprisal. Here, we investigated whether the presence of disfluencies in speech affects word processing in naturalistic listening conditions, using a dataset openly shared on OpenNeuro (Zadbood et al., 2017). Participants (n = 36) listened to the spoken recall of the events of a TV series they had previously watched, while undergoing fMRI. The spoken recall was fully spontaneous, and as such included several disfluencies, such as silent and filled pauses, discourse markers and repairs. The speech was annotated for word length, word frequency, word index in the sentence and in text, entropy and surprisal (from GPT2). In addition, the presence of disfluencies was annotated by marking silent pauses between words longer than 0.25 seconds and fillers such as “uhm”, “uh” and “like”. Disfluencies occurred more often at sentence boundaries and after the first word in a sentence. Word surprisal was on average higher after disfluencies than for words not preceded by a disfluency. We modelled word processing effort using parametric modulations for baseline predictors, as well as surprisal and presence/absence of a disfluency. To investigate the effects of disfluencies on word processing, we tested the interaction between disfluency and frequency, and disfluency and surprisal. Word processing was associated with increased BOLD activity in the left and right superior temporal gyrus (STG) and sulcus (STS) and in the left inferior frontal gyrus (IFG). Words preceded by a disfluency were associated with increased activity in the left and right STG. Decreased word frequency was associated with an increase in activity in the left mid STG, while the effect of frequency did not interact with the presence of disfluencies. Increased word surprisal elicited a similar distribution of activity, with bilateral superior temporal activation and a left inferior frontal cluster. The interaction between disfluency and surprisal was associated with a cluster in the posterior temporal lobe, where the effect of surprisal was reduced after a disfluency. The results held when classifying disfluencies as filled pauses only. The results therefore show that the presence of disfluencies is associated with increased brain activity in response to words, while the response to surprisal is reduced following disfluencies in a posterior temporal cluster. The presence of a disfluency may thus prepare the listener for higher complexity in the upcoming speech, by potentially allocating increased attention resources or enhancing top-down modulation.
Topic Areas: Meaning: Discourse and Pragmatics, Meaning: Lexical Semantics