Presentations

Abstract

In experimental psycholinguistics, research has focused overwhelmingly on either language comprehension or language production. However, the ultimate goal should be to identify the cognitive mechanisms that occur during the actual use of comprehension and production in everyday life, for example in conversation. Turn-taking (Sacks, Schegloff, & Jefferson, 1974) and action-sequencing (Schegloff, 2007) are essential mechanisms in everyday communication, but have not been studied much yet in the domain of experimental psycholinguistics (but see De Ruiter, Mitterer, & Enfield, 2006; Magyari & De Ruiter, 2012; Roberts, Francis, & Morgan, 2006). CA, on the other hand, has focused extensively on these mechanisms, via observing the behavior of conversationalists. To bring these two research domains closer together, in the present study, we aim to look at the cognitive processes that form the foundations for observable turn-taking and action-sequencing behavior in social interaction. We used electroencephalography (EEG), a method that measures electrical brain activity with a very good time resolution, to address this question. In EEG, experimental control is necessary to identify patterns of brain activity related to specific cognitive processes. Using an interactive quiz paradigm in Dutch, we investigated the processes underlying turn-taking in question-answer sequences, while still exerting enough experimental control over both the presented questions and expected answers. Participants were asked quiz questions by a research assistant while their EEG was measured. These questions were in fact pre-recorded, but participants believed they were asked live and they did receive live feedback from the assistant. We compared 'early questions' such as "Which character, also called 007, appears in the famous movies?" in which answer planning could start early in the question (at "007") with matched 'late questions' like "Which character from the famous movies, is also called 007?" in which answer planning could only start at the last word. Reaction times were longer for answers ("James Bond") to late than to early questions. Averaging the EEG signal to the critical word ("007" in the example) in both types of questions, we first see a small N400 effect (Kutas & Hillyard, 1980), a negative effect after about 400 milliseconds, which has been related to processing of unexpected words in a sentence. After this negativity, a large positive effect emerges which appears to be sustained until the response. We tentatively interpret this positivity to reflect a cognitive process associated with the response, such as the retrieval of the answer or the planning of production. These results thus show that in a question-answer sequence, speakers appear to start planning their response immediately when they have enough information to do so. Further results in the domain of brain oscillations and localizations of the effects in the brain will also be discussed. These results imply that cognitive processes might have a different timing than interactional processes. In some cases (as in our 'early questions'), the cognitive processes might be ready while interactionally the answer has to be postponed until the speaker finished speaking, possibly freeing up cognitive resources for a better timing of the response, for example. In other cases, cognitive processes might still be ongoing while the speaker has already finished her turn, for example when the question is difficult, calling for other interactional mechanisms such as the use of fillers. Future studies might employ the patterns of brain activation found to determine the presence and timing of cognitive processes within different types of action-sequences.

Abstract

In psycholinguistic experiments on language processing, researchers have traditionally focused on either comprehension or production. However, real-life, communicative language use happens most often in an interactive setting, involving rapid turn-taking between interlocutors. In such a setting, listening to a turn probably overlaps with preparing an answer to this turn. In the current EEG experiment, participants answered quiz questions, asked by the experimenter. Unknowingly to participants, these questions were pre-recorded, while the experimenter gave live feedback on participants’ answers. Questions appeared in two different conditions. Participants could confidently guess the answer to the question either halfway through the question (e.g., "Which character, also known as James Bond, appears in the famous movies?"), or only when they heard the last word(s) (e.g., "Which character, who appears in the famous movies, is also known as James Bond?"). Participants took longer to respond to the latter than the former question type, indicating that they start response preparation already during the question if they can, but leaving open when exactly production planning starts. ERP results showed a small N400 effect (Kutas & Hillyard, 1984), followed by a large positivity time-locked to the moment within the question that the answer started to become apparent (the critical point), relative to an equivalent position in the other condition. The N400 effect likely reflects the comprehension of the question, caused by a difference in the predictability of the words. In contrast, the positivity is more likely to be triggered by production processes, which was supported by source localisations of this effect in language production areas (e.g., Broca’s area and the temporal lobe, Indefrey & Levelt, 2004). In the frequency domain, less power in the alpha/mu band was found, starting within 500 milliseconds after the critical point. A follow-up control-experiment in which participants only listened to the questions and tried to remember them, was necessary to determine to what extent the positivity in the ERPs and the alpha/mu decrease indeed reflected production processes. Such a control-experiment showed a qualitatively similar pattern in the ERPs. However, the N400 was larger and the positivity was smaller and not localized in production areas, in contrast to the positivity in the main experiment. The effect in the alpha/mu band was absent or at least very much reduced. In combination with the localisations from the main experiment, we tentatively interpret the relative decrease in alpha/mu power as a signal of a shift of attention from comprehension to production-related processing. In all, both this effect and the positivity in the ERPs suggest that response preparation in interactive turn-taking situations starts quickly (within half a second) after an appropriate response can be retrieved. References Indefrey, P., & Levelt, W. J. (2004). The spatial and temporal signatures of word production components. Cognition, 92, 101-144. Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307, 161–163.

Abstract

In order to produce relevant responses in conversation, participants monitor turns at talk for the actions they perform, actions such as requests, offers, complaints, etc. (Schegloff, 2007). However, the link between turn construction and action is not straightforward. Turns at talk do not contain discrete ‘illocutionary force indicators’, and what subtle action cues are available, such as interrogative syntax, can be overridden by top-down factors like epistemic status (Heritage, 2012). Given that utterances are often underspecified for action, how is it that participants recognize actions so efficiently, as evidenced by the extraordinarily fast transitions between turns (Sacks, Schegloff, & Jefferson, 1974; Stivers et al., 2009; Levinson, 2013)? As the first step in investigating the cognitive underpinnings of action recognition in conversation, we conducted Event-Related Potential (ERP) experiments using short scripted dialogues in Dutch. The excellent time resolution of ERPs allows us to track listeners’ brain responses as utterances unfold. The critical utterances were assertions (e.g., “I have a credit card”) produced in three sequential environments, affording different ascriptions of the action; as an answer to a question, as an indirect rejection, or as a pre-offer. In each case the assertion is used as a vehicle for some other action, and it is “part of competent membership in the society/culture and being a competent interactant to analyze assertions of this sort for what (else) they may be doing at this moment, at this juncture of the interaction, in this specific sequential context” (Schegloff, 2007, p. 35). We tapped into this competence by exploring the time-course of action recognition, using the following rationale: If comprehension at the action level takes place early in the incoming utterance, enabling quick turn transitions, we should find ERP differences between the actions at the first word or the verb. On the other hand, if action comprehension requires analysis of the complete utterance, ERP effects are expected to predominantly occur at the final word. The results indicate that recipients tune in to the action of an utterance as early as 400 ms after first word onset. However, the time-course of speech act comprehension depends on the specific action. Rejections elicit an ERP effect at the first word and the verb, but not at the final word. We take this to show that when the utterance is a second pair part in an adjacency pair sequence – as was the case in the rejections – recipients seem to recognize the action before the final word, even though the final word is a critical part of the propositional content. The pre-offers, on the other hand, do elicit an ERP component at the end of the utterance, suggesting that analysis of the entire turn is needed to understand the action. These findings indicate that utterance interpretation is sensitive to specific actions and how they are organized in sequences. By bridging conversation analysis and neuropragmatics we have come one step closer to understanding language comprehension in its natural habitat, where action is omnirelevant (Schegloff, 1995).

Abstract

The insight that language and other social behavior should be analyzed sequentially – unit-by-unit, turn-by-turn, action-by-action – is arguably the central methodological innovation of conversation analysis. The force of this insight motivated early investigations into the sequential organization of phenomena such as laughter (Jefferson et al. 1977), jokes (Sacks 1974a, 1978), and story-telling (Sacks 1974b; Jefferson 1978). Although sequentiality is a general concern in all conversation-analytic research, it has been the primary object of study in a line of work on one specific form of sequence organization, the adjacency pair (Schegloff 1968, 1972, 1980, 1988, 2007; Schegloff and Sacks 1973). An adjacency pair is a sequential structure of two actions, produced by two participants, where the second action is contingent upon and normatively obliged by the production of the first (e.g., greeting-greeting, question-answer, request-acceptance/rejection). Though not all courses of action are organized through adjacency pairs, adjacency pairs are used to manage many basic social and communicative contingencies, including the transfer of goods, services, and information (offers, requests, statements, questions), and the initiation and termination of social encounters (openings, closings), among others (Schegloff and Sacks 1973).
The rich tradition of research on the adjacency pair and its organization has been based almost exclusively on audio and video recordings of social interaction made in the U.S. and U.K. Psychologists warn us that research on WEIRD people, that is, people form Western, Educated, Industrialized, Rich, and Democratic societies, may not generalize beyond this niche of outliers to the species as a whole (Henrich et al. 2010). While previous comparative research gives us reason to suspect that, unlike some psychological experiments, the core findings of conversation analysis successfully generalize beyond Anglo-American culture (see, e.g., Sidnell 2007, 2009; Stivers et al. 2009; Dingemanse and Floyd, in press), the linguistic and cultural universality of sequence organization remains an open question.
In this talk, we report on a collaborative investigation of sequence organization in 12 languages from distinct linguistic stocks and different geographical areas. We begin with a basic empirical question: Is sequence organization, as described by Schegloff (2007), universal? To answer this, we draw on video recordings of everyday social interaction made in fieldsites across the globe, with speakers of the following languages: ǂAkhoe Haiǀǀom (Khoisan; Namibia), Cha’palaa (Barbacoan; Ecuador), English (Germanic; U.S. and U.K.), Italian (Romance; Italy), Japanese (Japonic; Japan), LSA (sign language; Argentina), Mandarin Chinese (Sinitic; Taiwan), Siwu (Kwa; Ghana), Turkmen (Turkic; Turkmenistan), Tzeltal (Mayan; Mexico), Yélî Dnye (isolate; Papua New Guinea), and Yurakaré (isolate; Bolivia).
With the model of sequence organization in English as our point of departure (Schegloff 2007), we examine the structures that the speakers of these languages use to construct courses of action – unit-by-unit, turn-by-turn, action-by-action. While the primary object of study is the adjacency pair and its systematic expansion (Schegloff 2007; Levinson 2013), we also explore culture-specific forms of action-sequencing, such as the proliferation of repetitional post-expansions in Tzeltal, which can span six turns or more, and “broadcasting” in ǂAkhoe Haiǀǀom, in which speakers produce multi-unit tellings that neither occasion displays of recipiency nor solicit responses from those around them.
The results of our preliminary investigation reveal that all languages in the sample make use of the basic machinery of the adjacency pair and its expansion. In each language, we observe not only base adjacency pair sequences, but also pre-expansions, insert expansions, and post-expansions, as well as subtypes of these (see Schegloff 2007). The occurrence of these structures across a diverse sample of unrelated languages and cultures leads us to conclude that the structures do not belong to “language” or “culture” per se, but rather to a universal infrastructure for social interaction, an interaction engine (Levinson 2006) that all humans and human societies have in common and for which precursors may even be found among our nearest cousins, the apes (Rossano 2013). In agreement with Schegloff (2006), we propose that these structures emerge as solutions to recurrent socio-interactional problems, which are themselves basic to human sociality.

Abstract

This talk will focus on a central property of interactional uses of language, namely turn-taking, and show how this has major implications for language processing. I’ll sketch how the PI Group ‘Interactional Foundations of Language’ is exploring these issues, using insights arising from many different domains (prosody, breathing, gaze, cross-linguistic and cross-modal studies, human development) and many different methods (corpus studies, perception experiments, eyetracking, brain imaging). I’ll outline an interim model of how we think it all works, and (time-permitting) broach some implications for language typology, and ask where such a system comes from.

Abstract

The path-breaking paper by Sacks, Schegloff & Jefferson (1974) on turn-taking was a crucial foundation stone in the establishment of CA: turn-taking gives conversation its essential character and its analysis can be exploited to shed light on 101 further topics. Despite continuing work by a handful of scholars, we have thus learnt to take the phenomenon largely for granted. But recent work now shows just how extraordinary the turn - taking phenomenon is from a cognitive perspective, and makes clear that there is still a great deal to be explained about how the system actually works in detail (e.g. how turn-ends are actually projected). In this paper,drawing on the joint work of our MPI Nijmegen research project, I will bring many different avenues of investigation – from human development, the phonetics of breathing and intonation, psycholog ical experimentation, brain -imaging and cross -cultural and cross -linguistic perspectives –to bear on the underlying issues about how the system works in real time. One central psycholinguistic puzzle is that speech encoding is very slow, but turn -transition very fast, implying much more extensive projection than had been commonly imagined. These different lines of investigation also hint at a phylogenetically ancient interactive system that may have played a central role in the evolution of language.

Abstract

Dediu and Levinson (2013) argue that Neandertals had essentially modern language and speech, and that they were in genetic contact with the ancestors of modern humans during our dispersal out of Africa. This raises the possibility of cultural and linguistic contact between the two human lineages. If such contact did occur, then it might have influenced the cultural evolution of the languages. Since the genetic traces of contact with Neandertals are limited to the populations outside of Africa, Dediu & Levinson predict that there may be structural differences between the present-day languages derived from languages in contact with Neanderthals, and those derived from languages that were not influenced by such contact. Since the signature of such deep contact might reside in patterns of features, they suggested that machine learning methods may be able to detect these differences. This paper attempts to test this hypothesis and to estimate particular linguistic features that are potential candidates for carrying a signature of Neandertal languages.

Abstract

In this talk we suggest that the constraints on language processing and planning imposed by the structuring of interaction into turns at talk can affect the evolution of structural features of languages, such as basic word order. Speakers strive to reduce gaps and overlap between turns at talk (Sacks, Schegloff & Jefferson, 1974). Indeed, most gaps between turns are shorter than the minimum reaction time from planning to speak to actually speaking, suggesting that speakers project turn endings, and begin planning responses before the previous turn has ended. Recent research has demonstrated variation in the timing of turn-taking between speakers of different languages (Stivers et al., 2009). There is also variation between languages in the structure of individual turns, such as the basic order of subject, object and verb. There may be links between these two phenomena (Schegloff, 1996), mediated by the speaker’s planning of an utterance or the listener’s projection of the utterance. For instance, in Japanese, the verb appears at the end of an utterance, making projection difficult for the listener (Tanaka, 2000). Interestingly, the shortest average gaps occur for a verb-initial language (Tzeltal), which should make planning for the speaker difficult. We take a cultural evolutionary approach to this puzzle. If the basic timing of turn-taking is a fundamental principle of human languages (Levinson, 2006), then linguistic structures may have adapted to the constraints of turn-taking. For example, if the verb provides the syntactic frame for a sentence, then its position in the sentence might adapt to several pressures: verbs in final position give speakers more time to plan the most complex component of the turn; verbs in initial position allow recipients to project the shape of the turn relatively early. If speakers use the same configuration in a sequence of turns, then the amount of time for planning between each turn is optimised – what we call the principle of symmetry. Another constraint is that verbs in medial position are less vulnerable to overlap on the margins of the turn. We present a cultural evolutionary model as a proof of this concept. While many explanations of the origins and structure of language focus on constraints on individual cognition, this hypothesis suggests that constraints of interaction between individuals can also shape the distribution of structural features that we observe in the world’s languages.