Publications

Abstract

EEG and eye-tracking provide complementary information when investigating language comprehension. Evidence that speech processing may be facilitated by speech prediction comes from the observation that a listener's eye gaze moves towards a referent before it is mentioned if the remainder of the spoken sentence is predictable. However, changes to the trajectory of anticipatory fixations could result from a change in prediction or an attention shift. Conversely, N400 amplitudes and concurrent spectral power provide information about the ease of word processing the moment the word is perceived. In a proof-of-principle investigation, we combined EEG and eye-tracking to study linguistic prediction in naturalistic, virtual environments. We observed increased processing, reflected in theta band power, either during verb processing - when the verb was predictive of the noun - or during noun processing - when the verb was not predictive of the noun. Alpha power was higher in response to the predictive verb and unpredictable nouns. We replicated typical effects of noun congruence but not predictability on the N400 in response to the noun. Thus, the rich visual context that accompanied speech in virtual reality influenced language processing compared to previous reports, where the visual context may have facilitated processing of unpredictable nouns. Finally, anticipatory fixations were predictive of spectral power during noun processing and the length of time fixating the target could be predicted by spectral power at verb onset, conditional on the object having been fixated. Overall, we show that combining EEG and eye-tracking provides a promising new method to answer novel research questions about the prediction of upcoming linguistic input, for example, regarding the role of extralinguistic cues in prediction during language comprehension.

Abstract

We recently reported that refocusing attention between temporal and spatial tasks becomes more difficult with increasing age, which could impair daily activities such as driving (Callaghan et al., 2017). Here, we investigated the extent to which difficulties in refocusing attention extend to naturalistic settings such as simulated driving. A total of 118 participants in five age groups (18–30; 40–49; 50–59; 60–69; 70–91 years) were compared during continuous simulated driving, where they repeatedly switched from braking due to traffic ahead (a spatially focal yet temporally complex task) to reading a motorway road sign (a spatially more distributed task). Sequential-Task (switching) performance was compared to Single-Task performance (road sign only) to calculate age-related switch-costs. Electroencephalography was recorded in 34 participants (17 in the 18–30 and 17 in the 60+ years groups) to explore age-related changes in the neural oscillatory signatures of refocusing attention while driving. We indeed observed age-related impairments in attentional refocusing, evidenced by increased switch-costs in response times and by deficient modulation of theta and alpha frequencies. Our findings highlight virtual reality (VR) and Neuro-VR as important methodologies for future psychological and gerontological research.

Abstract

While some aspects of social processing are shared between humans and other species, some aspects are not. The former seems to apply to merely tracking another's visual perspective in the world (i.e., what a conspecific can or cannot perceive), while the latter applies to perspective taking in form of mentally “embodying” another's viewpoint. Our previous behavioural research had indicated that only perspective taking, but not tracking, relies on simulating a body schema rotation into another's viewpoint. In the current study we employed Magnetoencephalography (MEG) and revealed that this mechanism of mental body schema rotation is primarily linked to theta oscillations in a wider brain network of body-schema, somatosensory and motor-related areas, with the right posterior temporo-parietal junction (pTPJ) at its core. The latter was reflected by a convergence of theta oscillatory power in right pTPJ obtained by overlapping the separately localised effects of rotation demands (angular disparity effect), cognitive embodiment (posture congruence effect), and basic body schema involvement (posture relevance effect) during perspective taking in contrast to perspective tracking. In a subsequent experiment we interfered with right pTPJ processing using dual pulse Transcranial Magnetic Stimulation (dpTMS) and observed a significant reduction of embodied processing. We conclude that right TPJ is the crucial network hub for transforming the embodied self into another's viewpoint, body and/or mind, thus, substantiating how conflicting representations between self and other may be resolved and potentially highlighting the embodied origins of high-level social cognition in general.