Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Modeling memory retrieval during naturalistic comprehension
Poster A46 in Poster Session A, Tuesday, October 24, 10:15 am - 12:00 pm CEST, Espace Vieux-Port
Tzu-Yun Tung1, Jonathan R. Brennan1; 1University of Michigan
INTRODUCTION: Retrieval interference effects during agreement processing have been explained by cue-based retrieval theories under Adaptive Control of Thought-Rational (ACT-R) [3]. However, previous studies rely primarily on artificially-constructed sentences, which differ from every-day language. How similarity-based interference and decay may modulate naturalistic comprehension [1] remains unknown. We test for interference and decay effects during the resolution of subject-verb agreement during Chinese audiobook naturalistic listening. Extending [6]’s characterisation of pronoun reference in natural stories with ACT-R, we fit electroencephalography (EEG) signals against two ACT-R variants [4] and word-predictability measure from Chinese GPT2 [2, 9]. Variants differ in cue-combinatorics [8, 10]: structural cues may be weighted (i) equally [3] or (ii) preferably [8] over nonstructural cues in ACT-R. We find that interference effects, indexed by sustained negativity Event-Related Potential (ERP) component, surface in naturalistic comprehension, and are captured by both ACT-R models. METHODS: We extracted all subject-verb dependencies and intervening distractor nouns from audiobook text and annotated animacy features of the target and distractor nouns to determine interference effects (formalized as weighted associative strength in ACT-R); the time-interval between subject and verb was noted to model activation decay. Interference was higher where the animacy feature matched between target and distractor nouns. We then derived the ACT-R metrics reflecting activation of subject noun-phrase and GPT2 surprisal at the critical verb. 19 Chinese native speakers (12 female; aged 20–38) listened to Chinese ”The Little Prince” audiobook [5] during EEG recordings (sampling 500 Hz, online filter 0.1–200 Hz). Data was segmented around critical verb onset (-300–1000 ms); artifacts removed (0%-4.9%) using ICA and visual inspection. Single-trial mean amplitude was computed per trial for central channels (Fz, FC1, FC2, Cz, CP1, CP2, Pz) during 100–300, 300–500 and 500–800 ms [7, 11]. EEG amplitude was the dependent variable in separate Bayesian statistical models with different ACT-R and GPT2 metrics as fixed effect and random slope of each metric by participant. 223 trials were analyzed per participant. RESULTS & CONCLUSION: Results show interference effects during naturalistic comprehension: Higher subject activation and weighted associative strength, estimated via both ACT-R models, leads to less negativity in all three time-windows. Regression coefficients b (posterior mean and 95% CI) are: ACT-R-1: 0.06 ([0.01, 0.10]), 0.04 ([0.007, 0.08]), 0.04 ([0.006, 0.07]). ACT-R-2: 0.06 ([0.01, 0.10]), 0.04 ([0.007, 0.08]), 0.04 ([0.008, 0.07]). Compared to GPT2 surprisal, ACT-R metrics receive stronger evidence for successfully predicting single-trial EEG amplitude of the sustained negativity: ACT-R-1: ΔELPD = −2.7 SE = 2.6, ΔELPD = −3.7 SE = 3.6, ΔELPD = −2.7 SE = 3.0. ACT-R-2: ΔELPD = −2.7 SE = 2.6, ΔELPD = −3.8 SE = 3.7, ΔELPD = −2.6 SE = 3.0. We thus report one of the first cortical electrophysiological evidence of the memory interference effects during naturalistic language processing. Supplement: https://shorturl.at/MPSWZ
Topic Areas: Syntax and Combinatorial Semantics, Computational Approaches