Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
A deep hierarchy of predictions enables on-line meaning extraction in a computational model of human speech comprehension
Poster B95 in Poster Session B, Tuesday, October 24, 3:30 - 5:15 pm CEST, Espace Vieux-Port
Yaqing Su1,2, Lucy MacGregor3, Itsaso Olasagasti1,2, Anne-Lise Giraud1,2,4; 1University of Geneva, Geneva, Switzerland, 2Swiss National Centre of Competence in Research “Evolving Language”, 3University of Cambridge, Cambridge, UK, 4Institut Pasteur, Université Paris Cité, Inserm, Institut de l’Audition, Paris, France
Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity of contextualization to facilitate this process, how internal contextual knowledge is deployed on-line by the brain remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies both linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of magnetoencephalography (MEG) responses to speech containing semantically ambiguous words, compared to using lexical predictions generated by a neural-network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input. We discuss preliminary results from a MEG study that uses this model to guide the interpretation of neural information passing during speech comprehension under different tasks.
Topic Areas: Computational Approaches, Speech Perception