Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Dependence on Linguistic and Sensory Experience Predicts Individual Differences of Word Representation in Different Brain Areas
Poster Session B, Friday, October 25, 10:00 - 11:30 am, Great Hall 3 and 4
Ming Song1,2,3, Jing Wang1,2,3, Qing Cai1,2,3; 1Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), East China Normal University, 2Shanghai Changning Mental Health Center, Shanghai, China, 3Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai, China
Humans learn and represent the meanings of words through both linguistic and sensory experiences. Cognitive neuroscience research on semantics has shown that cross-individual variations in representation word meanings can be modeled by the strength of their association with either linguistic or sensory experiences. These variations tend to increase with the abstractness of the words. However, the contributions of linguistic and sensory information have not been dissociated for either abstract or concrete words, partly due to ineffective selection of experiment stimulus. In this study, we separately quantified words' reliance on linguistic and sensory experiences, identified words with varying dependencies on these experiences, and investigated their activation in different brain regions. For each word in the vocabulary, we calculated the “alignment score” among its vector representations in models trained on different corpora, to represent the word’s dependence on linguistic experience. We calculated the alignment score among sensory-motor ratings from different participant samples to represent a word’s dependence on sensory experience. We investigated the relationship between word abstractness and these two types of alignment scores. Three groups of words were selected for the functional MRI experiment: words with high dependence on linguistic experience and low dependence on sensory experience, words with low dependence on linguistic experience and high dependence on sensory experience, and those with intermediate dependence on both. Neural activations of these 90 words were estimated using single-trial beta coefficients in general linear models. We then calculated the inter-subject correlation (ISC) scores in the regions of interest (ROIs) selected from the language network and sensory-motor areas. Variables representing the dependence on linguistic and sensory experiences for each word were used to predict its ISC score in each ROI. The results indicated that words' dependence on linguistic experience was positively correlated with their abstractness, while dependence on sensory experience was negatively correlated with abstractness. The discriminative words selected for the neural experiment exhibited dissociated effects in language-related and sensory-related areas. Specifically, words with higher dependence on linguistic experience showed greater individual differences in sensory-related areas, whereas words with higher dependence on sensory experience showed greater individual differences in language-related areas. To summarize, this study aimed to disentangle the contributions of linguistic and sensory experiences to word representation. We quantified words' alignment in terms of linguistic and sensory experiences, examined their relationship with word abstractness, and identified words that were discriminative in their dependency on these experiences. These discriminative words exhibited dissociated effects in language-related and sensory-related brain areas. Our study provides insights into the complex interplay among language, sensory information, and abstractness and how they affect neural semantic representations.
Topic Areas: Meaning: Lexical Semantics, Computational Approaches