Presentation

Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Poster Slams

Using fMRI to study language regions in a conversational context

Poster B41 in Poster Session B and Reception, Thursday, October 6, 6:30 - 8:30 pm EDT, Millennium Hall
Also presenting in Poster Slam B, Thursday, October 6, 6:15 - 6:30 pm EDT, Regency Ballroom

Halie Olson¹, Emily Chen¹, Kirsten Lydic¹, Somaia Saba¹, Rebecca Saxe¹; ¹MIT

Language is encountered and produced in complex, dynamic, and social contexts – such as conversations. Although language comprehension builds on information acquired over time, previous neuroimaging evidence found that language regions are no more sensitive to narratives than isolated sentences (Blank & Fedorenko, NeuroImage, 2020). In this study, we asked whether language brain regions respond distinctively to sentences that occur in dialogue, reflecting the embedding of those sentences in the conversational context, or respond to them as independent sentences. We scanned 20 adults using functional MRI on two novel language tasks, as well as an independent language localizer (Scott et al, Cog Neuro, 2017) and theory of mind localizer (Dodell-Feder et al, NeuroImage, 2011). In Experiment 1, participants watched 20-second edited audiovisual clips of Sesame Street during which either two puppets speak to each other (dialogue), or a single puppet addresses the viewer (monologue), while the auditory speech is played either forwards or backwards. Individually-defined language regions showed robust responses to forward versus backwards speech, but did not discriminate dialogue versus monologue. Specific responses to sentences embedded within a dialogic context (interaction contrast: [dialogue forwards – monologue forwards] – [dialogue backwards – monologue backwards]) were observed outside language regions, in right superior temporal sulcus and bilateral temporal poles. In Experiment 2, participants watched 1-3 minutes of continuous back-and-forth dialogue in which one character speaks in backwards speech, while the other speaks forward. Which character speaks in forward versus backwards speech, in each video, was flipped for half of the participants. We compared the time courses in participants who heard the same version of the stimuli (e.g., the same characters forward vs. backward) to participants who heard the flipped version of the stimuli (e.g., opposite characters forward vs. backward). In language regions, only the linguistic input mattered: these regions’ timecourses of response to a given video were positively correlated when the auditory stream was aligned, and not correlated when the auditory stream was flipped. By contrast in regions defined by the theory of mind localizer, like right temporoparietal junction, the timecourses of response to a given video were positively correlated both for the aligned, and for the flipped, auditory stream. Together, both experiments show that during exposure to naturalistic multi-modal dialogue, left-hemisphere language regions respond specifically to temporally local linguistic features, whereas right-hemisphere social regions respond to the social and conversational context of language that emerge over longer timescales.

Topic Areas: Meaning: Discourse and Pragmatics, Perception: Speech Perception and Audiovisual Integration