Uncovering the effects of semantic context on the cortical processing of continuous speech using computational models of language

Abstract

The semantic context in which spoken words appear will greatly shape how they are understood by the human brain. This understanding is underpinned by a hierarchical system that processes increasingly abstract features of words at successive stages. Discerning how semantic context influences processing at each of these hierarchical stages has been major objective for the cognitive neurosciences. Modern neuroimaging has provided researchers with much insight in this regard, with electrophysiology measuring speech-related neural processes with precise temporal resolution. However, the cortical mechanisms underlying contextual language processing remain unclear. Furthermore, the impact of context on the neural processing of more continuous stretches of naturalistic speech materials is poorly understood. In this thesis, we introduce methodological frameworks for addressing these issues. We utilize modern computational language models, popular in the field natural language processing, in conjunction with recently developed system identification approaches to investigate the role of context in everyday language processing. The first half of this thesis introduces a novel approach to derive electrophysiological correlates for the semantic processing of continuous speech. In the first study (chapter 3), subjects were presented with narrative speech excerpts while their non-invasive EEG signal was recorded. Semantic features of the stimulus were estimated using word embedding models. We found that the function mapping semantic features to neural responses shared traits with classic N400 component. This index of semantic processing was sensitive to attention and speech intelligibility when tested using different EEG datasets. Our results indicate that the derived neural measures were robustly sensitive to speech comprehension. Chapter 4 further tests the sensitivity of the measure as a marker of the semantic processing of words in context and attempts to dissociate it from lower levels of processing such as word identification or recognition. This was done by systematically scrambling the order of words in a speech stimulus that was presented to subjects while their EEG was recorded. We found that, when higher levels of word scrambling were introduced, subjects? ability to comprehend the speech decreased. This coincided decrease in the neural marker of semantic processing. These results further support the notion that the newly derived measure reflects the semantic processing of speech. The second half of this thesis investigates the precise mechanisms underlying contextual language processing and whether the brain uses context to predictively preactivate features of upcoming words. In chapter 5, the method is applied to datasets of younger and older subjects listening to natural speech. Difference in these populations use context-based predictions, particularly at the level of semantic representation, have been observed in previous studies. This chapter builds on the previous 2 studies by using additional language models that index the relationship between words and context at different linguistic levels. We observe a dissociation between the neural correlates of different language models for older adults. Our results suggest that, while younger and older subjects both employ context-based lexical predictions, older subjects are significantly less likely to pre-activate the semantic features relating to upcoming words. Chapter 6 investigates how bottom-up sensory inputs combine with top-down contextual prior knowledge to subserve perception. As with previous chapters, we used computational language models to quantify how strongly words relate to their preceding context. We then measure whether this information impacts the acoustic and phonetic encoding of words using a 2-stage regression approach. Our results support the top-down influence of semantic context on the early auditory encoding of words. In addition, we find that the different language model measures independently affect the encoding of auditory information that their effects are dissociable in time. We interpret these results through the lens of predictive coding, where it is believed that distinct neuronal population exist at each cortical level, encoding prediction and error.