Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Neuro-behavioural correlates of audio-visual speech perception at different speech intelligibility levels in infants and toddlers
Poster D92 in Poster Session D, Wednesday, October 25, 4:45 - 6:30 pm CEST, Espace Vieux-Port
This poster is part of the Sandbox Series.
Irene Arrieta1,2, Cesar Caballero-Gaudes2, Zuriñe Martinez3, Xabier Altuna3, Manuel Carreiras1,2,4, Marina Kalashnikova2,4; 1Universidad del País Vasco, Spain, 2Basque center on Cognition, Brain and Language. San-sebastián, Spain, 3Hospital de Donostia - San Sebastián, Spain, 4Ikerbasque, Basque Foundation for Science, Bilbao, Spain
When auditory speech is degraded during face-to-face communication, more resources are needed for comprehension. As intelligibility decreases, a wider neural network involving premotor and prefrontal cortex activates in adults, children and infants. When intelligibility decreases, adults increase their attention to the speaker’s mouth to reconstruct the degraded auditory speech signal. In infants, attention to the mouth depends on their language acquisition stage. During their first half year of life, infants look more at the speaker’s eyes than at the mouth. From 6 to 12 months, during the perceptual attunement stage, when native speech perception abilities undergo a significant increase, infants look more at the mouth than at the eyes, regardless of speech intelligibility. After 12 months, infants gradually start to selectively look to the mouth when intelligibility decreases. After 2.5 years, attention to the mouth becomes more stable and is deployed only if needed. Recently, 6-month-olds’ tendency to look at a speakers’ mouth has been related to their neural activity in the prefrontal and inferior-frontal gyrus, in response to audio-visually presented vowels (Altvater-Mackensen & Grossmann, 2016, 2018). These results reflect a behavioural and neural correspondence on the use of resources to perceive speech. However, it is unclear whether this relation holds across different points in development, and whether it is modulated by speech intelligibility. To address this gap, we are conducting a study that includes two groups of children acquiring Basque. One group includes 8-10-month-old infants, the age corresponding to perceptual attunement, and the other includes 27-30-month-old toddlers, the age corresponding to significant lexical development. Infants are presented with 18 videos of audio-visual speech in Basque (15-20 seconds each; separated by a 3-6 second attention getter). The visual modality remains non-degraded across all videos while the auditory signal is presented as clear audio, degraded (vocoded) audio and silent audio (6 trials each). Infants’ gaze to the speaker’s mouth is measured using an Eye-Tracker (Eyelink 1000). Concurrently, neural activity is measured in prefrontal, premotor, temporal and occipital brain areas using fNIRS (NIRX systems), a neuro-imaging technique based on optical signals which capture the oxygen changes in the blood driven by neuro-vascular coupling. Data collection is in progress and will continue until 20 participants in each age group contribute analysable eye tracking and fNIRS data. Behaviourally, toddlers are expected to increase their attention to the speaker’s mouth when speech intelligibility decreases, whereas infants are expected to look at the mouth regardless of the speech intelligibility level. Neurally, toddlers are expected to show increasing neural activity (indexed as amplitude of the hemodynamic response) in prefrontal, premotor and visual brain areas as intelligibility decreases, while infants are expected to maintain their neural activity constant across conditions. Neuro-behaviourally, the looks to the mouth are expected to positively correlate with neural activity in the prefrontal cortex, premotor and visual areas (Altvater-Mackensen & Grossmann, 2016, 2018). These results will inform the current behavioural evidence and expand the knowledge on neural mechanisms behind attentional patterns during speech processing at different stages of language acquisition.
Topic Areas: Language Development/Acquisition, Multisensory or Sensorimotor Integration