Presentation

David Abugaber¹, Jonathan Brennan¹; ¹University of Michigan

A critical part of improving second language (L2) teaching involves understanding the brain mechanisms involved when learners comprehend grammatical structures. However, previous research on L2 processing has typically involved artificial tasks/stimuli in isolated laboratory settings different from what a real-world language user typically encounters. To address this, our project harnesses advances in wireless portable brain-scanning and computerized speech recognition to investigate the grammar processing mechanisms underlying naturalistic conversation in native and L2 English. EEG will be recorded during unscripted conversation between native/L2 interlocutors and synchronized with transcriptions. Data will be compared against predictions from computer models based on different possible grammar mechanisms as in prior work from a native audiobook listening study (Brennan & Hale, 2019). Specifically, we test hierarchical models involving nesting of abstract grammatical structures vs. sequential models based on word co-occurrence statistics. First, we ask whether previous findings of hierarchical processing in native speaker audiobook listening also hold for social interaction. Second, we ask whether native and L2 speakers differ in the hierarchical/sequential nature of their grammar processing. Thirdly, turning to the direct effects of social context, we ask whether neural signatures of grammar processing are affected by brain-to-brain synchrony. Piloting was conducted to determine whether neural markers of language processing could be adequately collected using our wireless consumer-grade EEG headset. First, reproducing Brennan and Hale (2019), we recorded EEGs from a single participant during listening to chapter 1 of Alice in Wonderland. Epochs were extracted from each word and baseline-corrected at -0.3s prior to word onset. Average amplitudes in anterior electrodes AF4, AF3, F3, F4, FC5, and FC6 (corresponding to effects in Brennan & Hale, 2019) in an early (300-500ms) and late (600-1000ms) time window (corresponding to timing of N400 and P600 ERPs; Tanner, 2019) were correlated against model predictions from Brennan and Hale (2019). We found significant correlations for the recurrent neural network model in the late time window, r(2127)=.06, p=.005, and for the n-gram model in both the early, r(2127)=.04, p=.040, and late time windows, r(2127)=.07, p=.002. No effects were found for the context-free grammar. This partly replicates Brennan and Hale (2019). Next, we recorded EEGs from a single participant during 30 minutes of unscripted conversation, transcribed using Amazon Transcribe yielding 3,503 words after omitting fillers and proper nouns with no hits in the British National Corpus (BNC, 2007). After preprocessing as above, we found a significant correlation, r(3501)=.04, p=.039 between early time window amplitudes and log-transformed word frequency, replicating well-attested frequency effects (Kutas & Federmeier, 2011). These findings indicate the viability of consumer-grade wireless EEG to test our grammar model predictions and to detect language processing effects in unscripted speech. This project informs language teaching praxis by revealing how the statistics of L2 input affect grammar learning. It also broadens participation in neuroscience by using a “crowdsource-able” experiment design with relatively affordable portable brain-scanning devices. Finally, our open-access corpus of natural conversation (comprising speech audio, neural signals, and text transcriptions) will be made available to future researchers for address other language-related research questions.

Topic Areas: Methods, Syntax