Search Abstracts | Symposia | Slide Sessions | Poster Sessions
The impact of conversational eyebrow movements on intention recognition: An fMRI Study
Poster Session B, Friday, October 25, 10:00 - 11:30 am, Great Hall 3 and 4
Yuyong Liu1, Heyonjeong Jeong2,3, Ayumi Takemoto3,4, Motoaki Sugiura3,5; 1Graduate School of Medicine, Tohoku Univ., 2Graduate School of International Cultural Studies, Tohoku Univ., 3Institute of Development, Aging and Cancer, Tohoku Univ., 4Riga Stradins University, 5International Research Institute for Disaster Science, Tohoku Univ.
Rapidly recognizing the speaker’s intentions, such as asking a question, is important for successful communication. This recognition is often facilitated by conversational facial signals (e.g., eyebrow frowns and raises). Based on the observation that the facial cues appear earlier than the accompanying speech, it is hypothesized that the precedence of facial cues aids the following predictive language processing. To the best of our knowledge, however, this hypothesis has not been empirically validated. Additionally, we examined whether certain eyebrow movements specifically signal questioning. While eyebrow frowns and raises have long been associated with questioning in face-to-face communication, several studies challenge this with the evidence that asking questions did not have more eyebrow raises than providing responses. This ongoing debate forms the second research question of the current study. Regarding the two research questions, our hypotheses are: 1) the precedence of conversational facial signals enhance the recognition of questions, and 2) the eyebrow frowns and raises specifically signal questions. To examine these hypotheses, we conducted an fMRI experiment to investigate the brain activity related to perceiving the early and late appearance of eyebrow movements accompanying questions and statements. We predicted that the brain regions involved in understanding intentions, such as the temporal-parietal junction (TPJ) and middle prefrontal cortex (mPFC) would be more efficient in processing the early appearance than the late appearance of eyebrow movements when identifying questions. We created video clips of Japanese uttering one sentence while making facial expressions that involve eyebrow frowns or raises as experiment stimuli. Crucially, we manipulated the timing of the facial signals, which appear either 750ms earlier(early-condition) or later (late-condition) than the onset of the speech, and we used two social actions: asking questions and stating facts. Forty-nine native Japanese participated in the experiment, watching each video stimulus, and judging whether the speaker was asking a question or stating a fact. The two-way repeated measures ANOVA on response time (RT) revealed a main effect of timing, with faster RT in the early-conditions compared to the late-conditions. Furthermore, the whole brain analysis revealed a main effect of the timing in the middle temporal gyrus (MTG), with decreased activation in early-conditions in comparison with late-conditions. The region-of-interest (ROI) analyses on mPFC and TPJ showed no significant difference among conditions. The MTG’s involvement in inferring intentions from perceiving actions (Schurz et al., 2014) supports our hypothesis that the precedence of facial signals helps to efficiently predict speakers’ intentions based on facial motion perception. However, the lack of interaction effects indicates that eyebrow frowns and raises do not specifically signal questions. In addition, the negative findings of ROI analyses imply that understanding intentions may not necessarily rely on the precedence of facial signals.
Topic Areas: Meaning: Discourse and Pragmatics, Multisensory or Sensorimotor Integration