Presentations

Abstract

Transitions between speakers in conversation are usually smooth, lasting around 200 milliseconds. Such rapid response latencies suggest that, at least sometimes, responders must begin planning their response before the ongoing turn is finished. Indeed, evidence from EEG suggests that listeners start planning their responses to questions as soon as they can, often midway through the incoming turn [1]. But given substantial overlap in the neural hardware for language production and comprehension, early response planning might incur a cost on participants’ concurrent comprehension of the ongoing turn. Do early responses come at the expense of less careful listening? We performed an EEG study in which participants played an interactive game with a confederate partner. Participants saw two pictures on their screen (e.g., a banana and a pineapple), then heard a (prerecorded) question from their partner, and then responded verbally by naming the correct picture. Participants were made to believe that their partner spoke to them live. Examples of the conditions in the experiment: 1. Early planning: 'Which object is curved and is considered to be fruit/healthy?'; 2. Late planning: 'Which object is considered to be fruit/healthy and is curved?' (response: 'the banana'). The questions were designed such that participants could start planning their response early (Example 1) or late (Example 2) in the turn. Crucially, in another part of the turn, we included either an expected word (e.g., 'fruit') or an unexpected one (e.g., 'healthy') to elicit a differential N400 effect. Our aims were two-fold: replicating the prior planning effect [1] and testing the effect of planning on comprehension. First, our results largely replicated the earlier study [1], showing a large positivity in the ERPs and an alpha/beta reduction in the time-frequency domain, both immediately following the onset of the critical information when participants could have first started planning their verbal response (i.e., 'curved'). As before [1], we interpret these effects as indicating the start of response planning. Second, and more importantly, we hypothesized that the N400 effect (the ERP difference between 'fruit' and 'healthy') would be attenuated when participants were already planning a response (i.e., in early vs. late planning). In contrast, we found an N400 effect of similar size in both the early and late planning conditions, although a small late positivity was only found in the late planning condition. Interestingly, we found a positive correlation between participants' overall response time and the size of the N400 effect after planning had started (i.e., in early planning), illustrating a trade-off between comprehension and production during turn taking. That is, quick responders showed a smaller N400 effect. We argue that their focus on production planning reduced their attention to the incoming audio signal and probably also their predictive processing, leading to a smaller N400 effect. Slow responders focused instead on the audio signal, preserving their N400 effect but delaying their response. Reference [1]: Bögels, S., Magyari, L., & Levinson, S. C. (2015). Neural signatures of response planning occur midway through an incoming question in conversation. Scientific Reports, 5: 12881. Topic Area: Meaning: Discourse and Pragmatics

Abstract

In conversation, negative responses to invitations,
requests, offers and the like more often occur with
a delay – conversation analysts talk of them as
dispreferred. Here we examine the contrastive
cognitive load ‘yes’ and ‘no’ responses make,
either when given relatively fast (300 ms) or
delayed (1000 ms). Participants heard minidialogues,
with turns extracted from a spoken
corpus, while having their EEG recorded. We find
that a fast ‘no’ evokes an N400-effect relative to a
fast ‘yes’, however this contrast is not present for
delayed responses. This shows that an immediate
response is expected to be positive – but this
expectation disappears as the response time
lengthens because now in ordinary conversation
the probability of a ‘no’ has increased.
Additionally, however, 'No' responses elicit a late
frontal positivity both when they are fast and when
they are delayed. Thus, regardless of the latency of
response, a ‘no’ response is associated with a late
positivity, since a negative response is always
dispreferred and may require an account. Together
these results show that negative responses to social
actions exact a higher cognitive load, but especially
when least expected, as an immediate response.

Abstract

In psycholinguistic experiments on language processing, researchers have traditionally focused on either comprehension or production. However, real-life, communicative language use happens most often in an interactive setting, involving rapid turn-taking between interlocutors. In such a setting, listening to a turn probably overlaps with preparing an answer to this turn. In the current EEG experiment, participants answered quiz questions, asked by the experimenter. Unknowingly to participants, these questions were pre-recorded, while the experimenter gave live feedback on participants’ answers. Questions appeared in two different conditions. Participants could confidently guess the answer to the question either halfway through the question (e.g., "Which character, also known as James Bond, appears in the famous movies?"), or only when they heard the last word(s) (e.g., "Which character, who appears in the famous movies, is also known as James Bond?"). Participants took longer to respond to the latter than the former question type, indicating that they start response preparation already during the question if they can, but leaving open when exactly production planning starts. ERP results showed a small N400 effect (Kutas & Hillyard, 1984), followed by a large positivity time-locked to the moment within the question that the answer started to become apparent (the critical point), relative to an equivalent position in the other condition. The N400 effect likely reflects the comprehension of the question, caused by a difference in the predictability of the words. In contrast, the positivity is more likely to be triggered by production processes, which was supported by source localisations of this effect in language production areas (e.g., Broca’s area and the temporal lobe, Indefrey & Levelt, 2004). In the frequency domain, less power in the alpha/mu band was found, starting within 500 milliseconds after the critical point. A follow-up control-experiment in which participants only listened to the questions and tried to remember them, was necessary to determine to what extent the positivity in the ERPs and the alpha/mu decrease indeed reflected production processes. Such a control-experiment showed a qualitatively similar pattern in the ERPs. However, the N400 was larger and the positivity was smaller and not localized in production areas, in contrast to the positivity in the main experiment. The effect in the alpha/mu band was absent or at least very much reduced. In combination with the localisations from the main experiment, we tentatively interpret the relative decrease in alpha/mu power as a signal of a shift of attention from comprehension to production-related processing. In all, both this effect and the positivity in the ERPs suggest that response preparation in interactive turn-taking situations starts quickly (within half a second) after an appropriate response can be retrieved. References Indefrey, P., & Levelt, W. J. (2004). The spatial and temporal signatures of word production components. Cognition, 92, 101-144. Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307, 161–163.

Abstract

Combining data from two longitudinal studies of young children, we track the development of turn-timing in spontaneous infant-caregiver interactions. We focus on three aspects of timing: overlap, gap, and delay marking. We find evidence for early development of turn-timing skills, in-line with the Interaction Engine Hypothesis. (see attached .pdf for our 2-page abstract)

Abstract

Introduction:
Underlying spatial memory and talking about spatial layouts are common cognitive processes (Haun et al. 2005). For example, to locate an object in space it is obligatory to choose a coordinate system called frame of reference in cognition as well as in its verbal expression. Coding space within different frames of reference requires different cognitive processes (e.g. Neggers et al. 2005). In relative frames of reference the origin of the coordinate system is the viewpoint of a person. In intrinsic frames of reference an object is located in relation to another object (Levinson 2003). FMRI data have suggested that different frames of reference show different patterns of neural activation (Burgess et al. 2002; Committeri et al. 2004). However, the number of existing frames of reference and their neural correlates remain controversial. In an event-related fMRI study we investigated whether differential neural networks for relative and intrinsic frames of reference can be isolated.
Methods:
In the present study an implicit sentence picture matching task was used to investigate differential neural correlates for relative and intrinsic frames of reference. Twenty-eight healthy human adults (16 women, 12 men) read a sentence describing a spatial scene followed by a picture, and decided whether the sentence matches the picture or not. Feedback was given either supporting a relative or an intrinsic frame of reference. After half of the trails the feedback switched from one reference frame to the respective other reference frame (Fig.1). Participants were instructed to respond as accurately and as quickly as possible. They responded with their right hand by pressing a key with the index finger for a correct decision and a second key with the middle finger for an incorrect judgment. Two baseline tasks were included (Fig.1): a high level baseline (c5) and a low level baseline (c6).
A 3 Tesla MRI system (Siemens TRIO, Erlangen, Germany) was used to acquire functional images of the whole brain. Using a gradient-echo echo planar scanning sequence 36 axial slices were obtained for each participant (voxel-size 3 x 3 x 3 mm, TR = 2310 ms, field of view = 192, TE = 30 ms, flip angle = 75). All functional images were acquired in one run that lasted for 50 minutes. Following the acquisition of functional images a high-resolution anatomical scan (T1-weighted MP-RAGE, 176 slices) was acquired. FMRI data were analyzed using BrainVoyager QX (Brain Innovation, Maastricht, The Netherlands). Random-effects whole brain group analyses were performed. The statistical threshold at the voxel level was set at p < 0.001, uncorrected for multiple comparisons.
Results:
Intrinsic trials as compared to baseline trials revealed increased activity in the parietal lobe and in the parahippocampal gyrus. Relative as compared to baseline trails revealed a widespread network of activity. Increased activity was observed in occipitotemporal cortices, in the parietal lobe, and in frontal areas.
We focused on the direct comparison between relative and intrinsic trials. Results showed increased activity in the left parahippocampal gyrus only for intrinsic trials as compared to relative trails. An ANOVA of the averaged beta-weights with the within factors Reference frame and Condition and the between factor Block order (relative-intrinsic and intrinsic-relative), obtained for all voxels in the parahippocampal gyrus, showed no main effect of Reference frames and Condition. A significant interaction between the factors Reference frame and Condition was observed (p < 0.05). T-contrasts showed a significant effect for intrinsic (c4) as compared to relative trials (c3; p < 0.001).
Conversely, relative as compared to intrinsic trials showed strong increased activity in the left medial frontal gyrus. An ANOVA of the beta-weights in the brain area showed no main effects. A significant interaction between the factors Reference frame and Condition was observed (p < 0.05). T-contrasts showed a significant effect for intrinsic (c4) as compared to relative trials (c3, p < 0.01).
When comparing all intrinsic and relative conditions together to the baseline we observed increased activity in the right and left frontal eye fields (Fig. 2). An ANOVA of the averaged beta-weights with the within factors Reference frame and the between factor Block order obtained for all voxels in the left frontal eye fields showed a main effect of Block order (p < 0.001) and an trend effect of Reference frame (p = 0.08). An ANOVA of the averaged beta-weights for all voxels in the right frontal eye fields showed a main effect of Block order (p < 0.05) only.
Conclusions:
Using a sentence-picture matching task, we investigated whether differential neural correlates for intrinsic and relative frames of reference can be isolated. Intrinsic trials compared to relative trials showed increased activity in the parahippocampal gyrus whereas relative trails compared to intrinsic trials revealed increased neural activity in the frontal and parietal lobe. Both frames of reference together compared to a baseline show increased activity in the frontal eye fields which was stronger for the second block. This could be related to switching of reference frames (Wallentin et al. 2008). The present results confirm studies which report the parietal lobe to be involved in relative coding (Cohen & Andersen 2002). The neural correlates of intrinsic frames of reference were previously less well investigated. The present results show differential neural networks for both frames of reference that are crucial to spatial language.
References:
Burgess, N. (2002), 'The human hippocampus and spatial and episodic memory', Neuron, vol. 36, pp. 625-641.
Cohen, Y. (2002), 'A common reference frame for movement plans in the posterior parietal cortex', Nature Reviews Neuroscience, vol. 3, pp. 553-562.
Committeri, G. (2004), 'Reference frames for spatial cognition: Different brain areas are involved in viewer-, object-, and landmark centered judgments about object location', Cognitive Neuroscience, vol. 16, pp. 1517-1535.
Haun, D. (2005), 'Bias in spatial memory: a categorical endorsement', Acta Psychologia, vol. 118, pp. 149-170.
Levinson, S. (2003), 'Space in language and cognition: Explorations in cognitive diversity', Cambridge: CUP.
Neggers, S. (2005), 'Quantifing the interactions between allo- and egocentric representation of space', Acta Psychologia, vol. 118, pp. 25-45.
Wallentin, M. (2008), 'Frontal eye fields involved in shifting frames of reference within working memory for scenes', Neuropsychologia, vol. 46, pp. 399-408.