Publications

Abstract

Words are not processed in isolation; instead, they are commonly embedded in phrases and sentences. The sentential context influences the perception and processing of a word. However, how this is achieved by brain processes and whether predictive mechanisms underlie this process remain a debated topic. Here, we employed an experimental paradigm in which we orthogonalized sentence context constraints and predictive validity, which was defined as the ratio of congruent to incongruent sentence endings within the experiment. While recording electroencephalography, participants read sentences with three levels of sentential context constraints (high, medium, and low). Participants were also separated into two groups that differed in their ratio of valid congruent to incongruent target words that could be predicted from the sentential context. For both groups, we investigated modulations of alpha power before, and N400 amplitude modulations after target word onset. The results reveal that the N400 amplitude gradually decreased with higher context constraints and cloze probability. In contrast, alpha power was not significantly affected by context constraint. Neither the N400 nor alpha power were significantly affected by changes in predictive validity.

Abstract

EEG and eye-tracking provide complementary information when investigating language comprehension. Evidence that speech processing may be facilitated by speech prediction comes from the observation that a listener's eye gaze moves towards a referent before it is mentioned if the remainder of the spoken sentence is predictable. However, changes to the trajectory of anticipatory fixations could result from a change in prediction or an attention shift. Conversely, N400 amplitudes and concurrent spectral power provide information about the ease of word processing the moment the word is perceived. In a proof-of-principle investigation, we combined EEG and eye-tracking to study linguistic prediction in naturalistic, virtual environments. We observed increased processing, reflected in theta band power, either during verb processing - when the verb was predictive of the noun - or during noun processing - when the verb was not predictive of the noun. Alpha power was higher in response to the predictive verb and unpredictable nouns. We replicated typical effects of noun congruence but not predictability on the N400 in response to the noun. Thus, the rich visual context that accompanied speech in virtual reality influenced language processing compared to previous reports, where the visual context may have facilitated processing of unpredictable nouns. Finally, anticipatory fixations were predictive of spectral power during noun processing and the length of time fixating the target could be predicted by spectral power at verb onset, conditional on the object having been fixated. Overall, we show that combining EEG and eye-tracking provides a promising new method to answer novel research questions about the prediction of upcoming linguistic input, for example, regarding the role of extralinguistic cues in prediction during language comprehension.

Abstract

While some aspects of social processing are shared between humans and other species, some aspects are not. The former seems to apply to merely tracking another's visual perspective in the world (i.e., what a conspecific can or cannot perceive), while the latter applies to perspective taking in form of mentally “embodying” another's viewpoint. Our previous behavioural research had indicated that only perspective taking, but not tracking, relies on simulating a body schema rotation into another's viewpoint. In the current study we employed Magnetoencephalography (MEG) and revealed that this mechanism of mental body schema rotation is primarily linked to theta oscillations in a wider brain network of body-schema, somatosensory and motor-related areas, with the right posterior temporo-parietal junction (pTPJ) at its core. The latter was reflected by a convergence of theta oscillatory power in right pTPJ obtained by overlapping the separately localised effects of rotation demands (angular disparity effect), cognitive embodiment (posture congruence effect), and basic body schema involvement (posture relevance effect) during perspective taking in contrast to perspective tracking. In a subsequent experiment we interfered with right pTPJ processing using dual pulse Transcranial Magnetic Stimulation (dpTMS) and observed a significant reduction of embodied processing. We conclude that right TPJ is the crucial network hub for transforming the embodied self into another's viewpoint, body and/or mind, thus, substantiating how conflicting representations between self and other may be resolved and potentially highlighting the embodied origins of high-level social cognition in general.