Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Complex neural responses during narrative comprehension may reflect the operation of a universal meaning representation system
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster B17 in Poster Session B, Tuesday, October 24, 3:30 - 5:15 pm CEST, Espace Vieux-Port
Peter Ford Dominey1,2,3; 1INSERM U1093 Cognition, Action and Plasticity Sensorimotor, Dijon, France, 2Robot Cognition Laboratory, Marey Institute, Dijon, France, 3University of Bourgogne, Dijon, France
One method to encode all possible narratives that could be generated in a language would be to anticipate them and pre-construct the corresponding representational system. A more tenable alternative strategy, that appears to be taken by the brain, is to rely on a form of neural combinatorics that can uniquely represent all possible narratives without specifically anticipating them. This can be implemented in recurrent cortical networks that generate functionally infinite high dimensional representations of sensory inputs and derived internal states. One signature of such architectures is the tuning of neurons to mixtures of multiple task related aspects, or mixed selectivity (Rigotti et al. 2013). Initially bewildering, these complex responses are now understood to be characteristic of recurrent networks and higher cognitive function (Rigotti et al. 2013, Enel et al. 2016). Narrative comprehension represents one of the highest human cognitive functions. We hypothesize the presence of mixed selectivity in the human brain during narrative processing. Baldassano (2018) characterized naturalistic human narrative processing and published an fMRI dataset where subjects experienced 16 naturalistic narrative stimuli with modality (audio, video), schema (restaurant, airport) and event (enter, formalities, transition, realization) dimensions. This multidimensionality is suitable for evaluating mixed selectivity. We used data from 40 runs collected from 10 subjects in this dataset, segmented into ROIs using the Schaefer 7 network 100 region atlas to evaluate mixed selectivity. For each of the resulting 160 narrative instances we extracted neural activity for each of the four temporally successive events, noting the corresponding modality and schema dimensions. We performed multifactor (Modality x Schema x Event) ANOVAs on individual voxels in the 100 ROIs and identified significant main effects, and interactions. We first examined areas that had a modality effect that significantly varied from auditory cortex (LH_SomMot_1 corresponding to BA41) which was assumed not to vary. As expected, this identified areas in the LH and RH visual networks. We then tested for areas that had 3-way mixed selectivity that varied significantly from that in auditory cortex. This revealed a significant effect in the limbic (Limbic_OFC_1, Limbic_TempPole_1 and 2), and default mode (Default_Temp_1, Default_PFC_4) networks. This first demonstration of mixed selectivity in human narrative processing contributes to a mechanistic explanation of how cortex can accommodate the functionally infinite possibility of narrative content and argues for the important role of medial prefrontal cortex in narrative as observed by Baldassano. We will discuss the relation between high dimensional processing and the temporal hierarchy of narrative processing, which can be further explored by combined human fMRI and computational studies with reservoir networks. *Rigotti, M., Barak, O., Warden, M. R., Wang, X. J., Daw, N. D., Miller, E. K., & Fusi, S. (2013). The importance of mixed selectivity in complex cognitive tasks. Nature, 497(7451), 585-590. *Enel, P., Procyk, E., Quilodran, R., & Dominey, P. F. (2016). Reservoir computing properties of neural dynamics in prefrontal cortex. PLoS computational biology, 12(6), e1004967. *Baldassano, C., Hasson, U., & Norman, K. A. (2018). Representation of real-world event schemas during narrative perception. Journal of Neuroscience, 38(45)
Topic Areas: Meaning: Discourse and Pragmatics, Multisensory or Sensorimotor Integration