| dc.description.abstract | Speech comprehension is a remarkable human ability. Most normal-hearing people are adept at attending to a speech stream even amidst a noisy multi-talker background and parsing the layers of information it contains in real-time to uncover its meaning. The cortex is thought to be hierarchically organised to perform the latter feat, with higher levels processing increasingly abstract features. Nonetheless, the precise computations it undertakes and how these processes are modulated by attention remain incompletely understood.
The development of novel encoding/decoding approaches for relating high temporal resolution neuroimaging modalities to representations of multivariate stimuli like continuous natural speech have enabled researchers to shed some light on this topic. An important recent discovery is that low-frequency, non-invasive EEG/MEG tracks the amplitude envelope of speech (a low-level acoustic measure conveying many cues important for speech comprehension) and that a robust reconstruction of the measure can be acquired from neural data. Moreover, during cocktail party listening tasks, this tracking and the accuracy of reconstruction have been shown to be modulated by attention. These findings have given rise to further research along several lines, including studies employing the framework to investigate how cortex encodes other speech processing stages along the hierarchy and how this encoding is affected by attention, as well as studies developing the idea of exploiting envelope reconstruction as a means of decoding attentional selection for the implementation of smart devices like steerable hearing aids.
In this thesis, we employ a particular encoding/decoding approach ? the temporal response function (TRF) ? in conjunction with EEG to address several questions within these subareas. We first tested the efficacy of a state-of-the-art framework for utilising envelope reconstruction to decode auditory attention (O Sullivan et al., 2015) within the context of a cocktail party paradigm with moving talkers and showed that it is robust. This was motivated by the non-stationarity of real-world environments. We then considered if (1) the decoder weights themselves (i.e., the model weights mapping from EEG data to the acoustic envelope) and (2) alpha power might contain unique information that can be leveraged for the decoding of attention. We showed that incorporating a metric based on the consistency of model weights across subjects into the decoding framework yielded an improvement in performance above and beyond using envelope reconstruction alone.
We then investigated the neural processing of prosody ? an aspect of spoken language that conveys another layer of meaning on top of linguistic units. In particular, we were interested to explore how prosodic pitch is encoded in low-frequency EEG during listening to continuous natural speech. We mapped two measures of prosodic pitch ? relative pitch and harmonic resolvability ? to concurrently-recorded EEG. These measures were inspired by an ECoG study showing neural tracking of relative pitch during listening to sentences (Tang, Hamilton, & Chang, 2017), and an fMRI study demonstrating that there are cortical regions that respond primarily to resolved harmonics (Norman-Haignere, Kanwisher, & McDermott, 2013). We found that delta-band EEG tracks relative pitch during listening to continuous natural speech, and that this tracking is dissociable from the tracking of other acoustic and phonetic features.
Finally, we tested how attention modulates EEG signatures hypothesized to represent processing at the pre-lexical level, as well as the signature of relative pitch found in the previous study. The former inquiry was motivated by the longstanding debates as to whether attention modulation operates at an early or late stage in the speech processing hierarchy, and whether an acoustic-phonetic transformation stage occurs as an intermediate step in mapping the acoustic speech signal to the mental store of words. We found that a phonetic feature representation could uniquely predict neural activity above and beyond other measures. Additionally, this unique predictive power was significantly modulated by attention, unlike that of a representation derived from acoustics that had been suggested to explain away the contribution of phonetic features (Daube et al., 2019). This lends support to the notion of the phonetic feature representation being a distinct and higher-level stage in the hierarchy. We also found attentional modulation of our signature of relative pitch and showed that incorporating the reconstruction accuracy of this representation into the decoding attentional selection framework led to a small improvement in decoding performance for some subjects. | en |
