Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Do language and vision employ the same mechanism for tracking individual representations? An ERP study
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster E14 in Poster Session E, Thursday, October 26, 10:15 am - 12:00 pm CEST, Espace Vieux-Port
This poster is part of the Sandbox Series.
Xiaoyu Yang1, Ellen Lau1; 1University of Maryland, College Park
Language comprehension oftentimes requires tracking representation of entities in working memory. That is also the case with visual perception, but it’s yet unclear whether the same neural mechanism is used for the two. As a preliminary attempt to explore this question, we are collecting data for an ERP study that investigates whether tracking discourse entities in language comprehension induces sustained neural activity, which previous vision studies found to reflect the tracking of visual objects. Vogel and Machizawa (2004) found that increasing the number of visual objects represented in working memory induces increased sustained ERP response. If language comprehension yields representations supported by the same neural mechanism as visual perception does, then presumably we would expect to see the same sustained response as people track entities conveyed by language. Interestingly, sustained ERP responses with a similar distribution have been observed in King and Kutas (1995) for object relative clauses compared to subject relative clauses (e.g. ‘The reporter who the senator harshly attacked’ vs. ‘The reporter who harshly attacked the senator’), and in Cruz Heredia et al. (2021) for wh-questions compared to polar questions (e.g. ‘What did the commentary from the spokesman…’ vs. ‘Did the commentary from the spokesman…’). Although these kinds of responses have often been interpreted as reflecting syntactic working memory, an alternative interpretation is that these responses are induced by tracking the extra entity conveyed by the object and the wh-word. In order to test this hypothesis, we use a novel ERP paradigm designed to emphasize natural referential processing of language input, through the use of spoken materials and connected narratives. We present audio recordings of short stories that introduce varying numbers of discourse referents. To encourage participants to build a rich discourse representation, we used dramatic storylines and a continuation task in which participants are asked to ‘tell how the story ends’. For each story item we created two versions that diverge at a point in which a second character is introduced in one version but not the other. This brief divergence in content is immediately followed by a subsequent region in which material across versions is again matched. Our critical ERP time-window is the first 2000ms of this matched region, immediately after the introduction of a new discourse-relevant entity in one condition but not the other. If our hypothesis holds, we expect to see a sustained neural response increase in the two-character stories compared to their one-character counterparts in this time-window, due to the continued tracking of this extra referent.
Topic Areas: Meaning: Discourse and Pragmatics,