Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Classifying brain activity during semantic integration of sentences: An ERP study
Poster Session A - Sandbox Series, Thursday, October 24, 10:00 - 11:30 am, Great Hall 3 and 4
This poster is part of the Sandbox Series.
Toru Egashira1, Shinri Ohta1; 1Kyushu university
Keywords: ERPs, RNNs, semantic integration, logistic regression analysis This study aims to create a model that uses ERPs to determine how a given text is being interpreted by the listener/reader. For example, a sentence such as “The man saw the boy with the binoculars.” is open to multiple interpretations, and the listener/reader can integrate either interpretation when reading it. We made three types of sentences as stimuli, two of which are unambiguous, and we measured brain waves while reading each interpretation of the sentence. We will analyze those data in two ways to create a model that discriminates which interpretation of a sentence is being read by performing a two-class classification. Previous studies have attempted to determine the part of speech of words pronounced in the head, but few have classified sentences in terms of syntax and meaning (Sahil & Nikolaos, 2021). The following three types of sentences written in Japanese were used as stimuli: structurally ambiguous sentences (e.g., apaato-ni sumu kareshi-no oneesan-wa oshitoyakada, a sister of my boyfriend who lives in an apartment is modest; in Japanese, it cannot be distinguished syntactically who lives in an apartment), high attachment sentences (e.g., joshiryo-ni sumu kareshi-no oneesan-wa oshitoyakada, a sister of my boyfriend who lives in a girls’ dormitory is modest), and low attachment sentences (e.g., danshiryo-ni sumu kareshi-no oneesan-wa oshitoyakada, a sister of my boyfriend who lives in a boys’ dormitory is modest) (40 sentences/term). For ambiguous sentences, the relative clause can modify both nouns (kareshi or oneesan), while in low-attachment and high-attachment sentences, the first and second nouns are modified, respectively. After the sentence presentation, participants answered a sentence comprehension question by pressing buttons to test the participants’ interpretation. For the data analysis, we will employ the widely used logistic regression analysis and recurrent neural networks (RNNs) to examine the classification accuracy of sentence comprehension of structurally ambiguous sentences based on the event-related potentials (ERPs). We recruited 13 healthy, right-handed native Japanese speakers (3 females). ERPs were acquired using 32 Ag/AgCl electrodes (actiCAP, Brain Products; Neurofax EEG-1200, Nihon Kohden), and EEG data were analyzed using EEGLAB. P3 and P4 were selected as electrodes of interest. We have already recorded the ERP data, and we will analyze that data as follows. We plan to use the following basic structure, but the details of the hyperparameters will be optimized through future adjustments. The LSTM layer, a key component of our model, will consist of 1 to 3 layers. The number of units in each LSTM layer will range from 50 to 128, and a dropout rate will be implemented to prevent over-learning (e.g., 0.2, 0.5). The dense layer will be added to all coupling layers after LSTM (e.g., 64, 128 units), and we will adopt ReLU (LSTM and Dense layer). We will use grid search or random search to find the optimal hyperparameters, then train the final model based on the optimal hyperparameter settings obtained and analyze the accuracy and other evaluation metrics.
Topic Areas: Computational Approaches,