Presentation

Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Poster Slams

Computational pressures behind the development of parallel dorsal and ventral stream lexica

Poster E4 in Poster Session E, Saturday, October 8, 3:15 - 5:00 pm EDT, Millennium Hall

Enes Avcu¹, Michael Hwang², Kevin Brown³, David Gow^1,4; ¹Massachusetts General Hospital / Harvard Medical School, ²Harvard College, ³Oregon State University, ⁴Salem State University

Words play a pivotal role in almost every aspect of language processing. The dual-stream model of spoken language processing (Hickok & Poeppel, 2007) suggests that processing is organized broadly into parallel dorsal and ventral processing streams concerned with dissociable aspects of motor and conceptual-semantic processing. Drawing on converging evidence from pathology, neuroimaging, behavioral research, and histology, Gow (2012) proposes that each pathway has its own lexicon or lexical interface area, which mediates mappings between acoustic-phonetic representation and stream-specific processing. In the dorsal processing stream, the supramarginal gyrus and inferior parietal lobe mediate the mapping between sound and word-level articulatory representation. In the ventral processing stream, the posterior middle temporal gyrus mediates the mapping between sound and semantic/syntactic representation. We hypothesize that this separation arose in part because of fundamental differences in the computational requirements of these mappings. The mapping between sound and articulation, though complex, is largely systematic and temporally continuous. In contrast, the mapping between sound and syntactic/semantic information, though partially systematic at the level of productive morphology, is largely arbitrary and dependent on identifying larger temporal units. To test this hypothesis, we created two LSTM networks and trained them independently on the same set of auditory word tokens. A dorsal model was trained to identify individual spoken words, while a ventral model was trained to map them onto overlapping sets of word context frames drawn from a corpus of meaningful text as a surrogate representation of semantic content. After training both models to asymptote, we extracted patterns of network activation from the hidden layer of each network and tested how well the features extracted from the dorsal network supported the classification of input based on articulatory versus semantic or syntactic properties. We predicted that: (i) Features from dorsal LSTM models trained on wordform identification should have an advantage for categorization related to articulation but not semantic/syntactic categorization, and (ii) Features from ventral LSTMs trained on sentential context frames should have an advantage for semantic/syntactic categorization but not categorization related to articulation. Our results demonstrate that training the same set of networks on differently structured lexical representations produced different featural representations at the hidden layer of each model and that these emergent representations supported different patterns of performance on secondary tasks. Despite being trained on output vectors that were not structured to reflect the phonological structure, the dorsal model discovered a feature space that supported the classification of word-initial phonemes by articulatory classes. In the same vein, the ventral model discovered a feature set that supported grammatical categorization without explicitly having been trained on grammatical categories. The finding that the ventral model outperformed the dorsal model on grammatical category classification is not surprising, but it again demonstrates that task demands shape feature spaces that are better suited for different types of generalization. These results suggest that the development of parallel lexica in the dorsal and ventral pathways arose from computational pressures for optimizing the primary mapping functions that support lexically organized processes in the dorsal and ventral processing streams.

Topic Areas: Computational Approaches, Speech Perception