Publications

Abstract

This chapter reviews the computational processes that are responsible for recognizing word forms in the speech stream. We outline the different stages in a processing hierarchy from the extraction of general acoustic features, through speech‐specific prelexical processes, to the retrieval and selection of lexical representations. We argue that two recurring properties of the system as a whole are abstraction and adaptability. We also present evidence for parallel processing of information on different timescales, more specifically that segmental material in the speech stream (its consonants and vowels) is processed in parallel with suprasegmental material (the prosodic structures of spoken words). We consider evidence from both psycholinguistics and neurobiology wherever possible, and discuss how the two fields are beginning to address common computational problems. The challenge for future research in speech perception will be to build an account that links these computational problems, through functional mechanisms that address them, to neurobiological implementation.

Abstract

The most basic question for the study of morphology and the mental lexicon is whether or not words are _decomposed_: informally, this is the question of whether words are represented (and processed) in terms of some kind of smaller units; that is, broken down into constituent parts. Formally, what it means to represent or process a word as decomposed or not turns out to be quite complex. One of the basic lines of division in the field classifies approaches according to whether they decompose all “complex” words (“Full Decomposition”), or none (“Full Listing”), or some but not all, according to some criterion (typical of “Dual-Route” models). However, if we are correct, there are at least three senses in which an approach might be said to be decompositional or not, with the result that ongoing discussions of what appears to be a single large issue might not always be addressing the same distinction. Put slightly differently, there is no single question of decomposition. Instead, there are independent but related questions that define current research. Our goal here is to identify this finer-grained set of questions, as they are the ones that should assume a central place in the study of morphological and lexical representation.

Abstract

Research on video- and audio-recordings of spontaneous naturally-occurring conversation in English has shown that conversation is a rule-guided, practice-oriented domain that can be investigated for its underlying mechanics or structure. Systematic study could yield something like a grammar for conversation. The goal of this task is to acquire a corpus of video-data, for investigating the underlying structure(s) of interaction cross-linguistically and cross-culturally.

Abstract

This task is designed to elicit virtual 3D ‘models’ created in gesture space using iconic and other representational gestures. This task has been piloted with Lao speakers, where two speakers were asked to explain the meaning of terms referring to different kinds of fish trap mechanisms. The task elicited complex performances involving a range of iconic gestures, and with especially interesting use of (a) the ‘model/diagram’ in gesture space as a virtual object, (b) the non-dominant hand as a prosodic/semiotic anchor, (c) a range of different techniques (indexical and iconic) for evoking meaning with the hand, and (d) the use of nearby objects and parts of the body as semiotic ‘props’.

Abstract

Research on video- and audio-recordings of spontaneous naturally-occurring conversation in English has shown that conversation is a rule-guided, practice-oriented domain that can be investigated for its underlying mechanics or structure. Systematic study could yield something like a grammar for conversation. The goal of this task is to acquire a corpus of video-data, for investigating the underlying structure(s) of interaction cross-linguistically and cross-culturally

Abstract

We want to know how people think about their field of kin, on the supposition that it is quasi-spatial. To get some insights here, we need to video a discussion about kinship reckoning, the kinship system, marriage rules and so on, with a view to looking at both the linguistic expressions involved, and the gestures people use to indicate kinship groups and relations. Unlike the task in the 2001 manual, this task is a direct interview method.

Abstract

This sub-project is concerned with analysis and cross-linguistic comparison of the mechanisms of signaling and redressing ‘trouble’ during conversation. Speakers and listeners constantly face difficulties with many different aspects of speech production and comprehension during conversation. A speaker may mispronounce a word, or may be unable to find a word, or be unable to formulate in words an idea he or she has in mind. A listener may have troubling hearing (part of) what was said, may not know who a speaker is referring to, may not be sure of the current relevance of what is being said. There may be problems in the organisation of turns at talk, for instance, two speakers’ speech may be in overlap. The goal of this task is to investigate the range of practices that a language uses to address problems of speaking, hearing and understanding in conversation.

Abstract

This task is a complement to the questionnaire ‘Multimodal interaction in your field site: a preliminary investigation’. The objective of the task is to obtain high quality video data on structured and symmetrical dyadic multimodal interaction. The features of interaction we are interested in include turn organization in speech and nonverbal behavior, eye-gaze behavior, use of composite signals (i.e. communicative units of speech-combined-with-gesture), and linguistic and other resources for ‘navigating’ interaction (e.g. words like okay, now, well, and um).

Abstract

Over the decades, high-intensity language learners scattered over the globe referred to as “hyperpolyglots” have undertaken a natural experiment into the limits of learning and acquiring proficiencies in multiple languages. This chapter details several ways in which hyperpolyglots are relevant to research on aptitude. First, historical hyperpolyglots Cardinal Giuseppe Mezzofanti, Emil Krebs, Elihu Burritt, and Lomb Kató are described in terms of how they viewed their own exceptional outcomes. Next, I draw on results from an online survey with 390 individuals to explore how contemporary hyperpolyglots consider the explanatory value of aptitude. Third, the challenges involved in studying the genetic basis of hyperpolyglottism (and by extension of language aptitude) are discussed. This mosaic of data is meant to inform the direction of future aptitude research that takes hyperpolyglots, one type of exceptional language learner and user, into account.

Abstract

In Dutch, all morpheme-final obstruents are voiceless in word-final position. As a consequence, the distinction between obstruents that are voiced before vowel-initial suffixes and those that are always voiceless is neutralized. This study adds to the existing evidence that the neutralization is incomplete: neutralized, alternating plosives tend to have shorter bursts than non-alternating plosives. Furthermore, in a rating study, listeners scored the alternating plosives as more voiced than the nonalternating plosives, showing sensitivity to the subtle subphonemic cues in the acoustic signal. Importantly, the participants who were presented with the complete words, instead of just the final rhymes, scored the alternating plosives as even more voiced. This shows that listeners’ perception of voice is affected by their knowledge of the obstruent’s realization in the word’s morphological paradigm. Apparently, subphonemic paradigmatic levelling is a characteristic of both production and perception. We explain the effects within an analogy-based approach.

Abstract

The Cha’palaa language of Ecuador (Barbacoan) features verbal morphology for marking knowledge-based categories that, in usage, show a variant of the cross-linguistically recurrent pattern of ‘egophoric distribution': specific forms associate with speakers in contrast to others in statements and with addressees in contrast to others in questions. These are not person markers, but rather are used by speakers to portray their involvement in states of affairs as active, agentive participants (ego) versus other types of involvement (non-ego). They interact with person and argument structure, but through pragmatic ‘person sensitivities’ rather than through grammatical agreement. Not only does this pattern appear in verbal morphology, it also can be observed in alternations of predicate construction types and case alignment, helping to show how egophoric marking is a pervasive element of Cha'palaa's linguistic system. This chapter gives a first account of egophoricity in Cha’palaa, beginning with a discussion of person sensitivity, egophoric distribution, and issues of flexibility of marking with respect to degree of volition or control. It then focuses on a set of intransitive experiencer (or ‘endopathic') predicates that refer to internal states which mark egophoric values for the undergoer role, not the actor role, showing ‘quirky’ accusative marking instead of nominative case. It concludes with a summary of how egophoricity in Cha'palaa interacts with issues of argument structure in comparison to a language with person agreement, here represented by examples from Cha’palaa’s neighbor Ecuadorian Highland Quechua.

Abstract

Lesion symptom mapping has revolutionized our understanding of the functioning of the human brain. Associating damaged voxels in the brain with loss of function has created a map of the brain that identifies critical areas. While these methods have significantly advanced our understanding, recent improvements have identified the need for multivariate and multimodal methods to map hidden lesions and damage to white matter networks beyond the lesion voxels. This article reviews the evolution of lesion-symptom mapping from single case studies to the human disconnectome.

Abstract

Structural imaging based on computerized tomography (CT) and magnetic resonance imaging (MRI) has progressively replaced traditional post‐mortem studies in the process of identifying the neuroanatomical basis of language. In the clinical setting, the information provided by structural imaging has been used to confirm the exact diagnosis and formulate an individualized treatment plan. In the research arena, neuroimaging has permitted to understand neuroanatomy at the individual and group level. The possibility to obtain quantitative measures of lesions has improved correlation analyses between severity of symptoms, lesion load, and lesion location. More recently, the development of structural imaging based on diffusion MRI has provided valid solutions to two major limitations of more conventional imaging. In stroke patients, diffusion can visualize early changes due to a stroke that are otherwise not detectable with more conventional structural imaging, with important implications for the clinical management of acute stroke patients. Beyond the sensitivity to early changes, diffusion imaging tractography presents the possibility of visualizing the trajectories of individual white matter pathways connecting distant regions. A pathway analysis based on tractography is offering a new perspective in neurolinguistics. First, it permits to formulate new anatomical models of language function in the healthy brain and allows to directly test these models in the human population without any reliance on animal models. Second, by defining the exact location of the damage to specific white matter connections we can understand the contribution of different mechanisms to the emergence of language deficits (e.g., cortical versus disconnection mechanisms). Finally, a better understanding of the anatomical variability of different language networks is helping to identify new anatomical predictors of language recovery. In this chapter we will focus on the principles of structural MRI and, in particular, diffusion imaging and tractography and present examples of how these methods have informed our understanding of variance in language performances in the healthy brain and language deficits in patient populations.

Abstract

Several sources of information are used in choosing the intended referent of an ambiguous pronoun. The two sources considered in this paper are foregrounding and context. The first refers to the accessibility of discourse entities. An entity that is foregrounded is more likely to become the pronoun’s referent than an entity that is not. Context information affects pronoun resolution when world knowledge is needed to find the referent. The model presented here simulates how world knowledge invoked by context, together with foregrounding, influences pronoun resolution. It was developed as an extension to the Distributed Situation Space (DSS) model of knowledge-based inferencing in story comprehension (Frank, Koppen, Noordman, & Vonk, 2003), which shall be introduced first.

Abstract

Speaking and listening are complex tasks that we perform on a daily basis, almost without conscious effort. Interestingly, speaking almost never occurs without listening: whenever we speak, we at least hear our own speech. The research in this thesis is concerned with how the perception of our own speech influences our speaking behavior. We show that unconsciously, we actively monitor this auditory feedback of our own speech. This way, we can efficiently take action and adapt articulation when an error occurs and auditory feedback does not correspond to our expectation. Processing the auditory feedback of our speech does not, however, automatically affect speech production. It is subject to a number of constraints. For example, we do not just track auditory feedback, but also its consistency. If auditory feedback is more consistent over time, it has a stronger influence on speech production. In addition, we investigated how auditory feedback during speech is processed in the brain, using magnetoencephalography (MEG). The results suggest the involvement of a broad cortical network including both auditory and motor-related regions. This is consistent with the view that the auditory center of the brain is involved in comparing auditory feedback to our expectation of auditory feedback. If this comparison yields a mismatch, motor-related regions of the brain can be recruited to alter the ongoing articulations.

Abstract

This project is part of a collaborative project with the research group “Reciprocals across languages” led by Nick Evans. One goal of this project is to develop a typology of reciprocals. This questionnaire is designed to help field workers get an overview over the type of markers used in the expression of reciprocity in the language studied.

Abstract

This doctoral dissertation investigated the learning of auditory categories by applying insights from child language learning, visual category learning and phonetic categorization by adults. The experiments in chapter 2 concern supervised learning of multidimensional non-speech categories varying in two dimensions: duration and a non speech analogue of formant frequency. In experiment 1, one dimension of variation determined category membership, variation in the other dimension was irrelevant. Listeners quickly learned to categorize according to this distinction. In experiment 2, both dimensions needed to be combined to categorize correctly. Performance was much lower, but most listeners succeeded in this task. However, in a maintenance phase without feedback or distributional information, listeners reverted to a unidimensional solution. In a maintenance phase with distributional information, listeners did use both dimensions correctly, arguing for the importance of distributional information in (auditory) category acquisition. In chapter 3, the listeners had to classify the same categories, but without feedback. In experiment 1, listeners succeeded to discover the relevant dimension of variation (and ignore the irrelevant one) without feedback. Much of this learning was lost in the maintenance phase, especially for the dimension formant frequency. With two relevant dimensions (Experiment 2), listeners were not able to learn to use both dimensions and reverted to a unidimensional solution. Chapter 4 applied the paradigms of chapter 2 and 3 to the learning of speech categories. Spanish native listeners learned Dutch vowel contrast with one relevant dimension of variation. With feedback, learning was swift, although was not well maintained without feedback or distributional information. Without feedback, Spanish listeners reverted to the dimensions best known in their native phonology, formant frequency, even when distributional information pointed to duration. The results are discussed in chapter 5. The implications for models of speech acquisition are discussed.

Abstract

Gestures are visuospatial events, meaning carriers, and social interactional phenomena. As such they constitute a particularly favourable area for investigating visual attention in a complex everyday situation under conditions of competitive processing. This chapter discusses visual attention to spontaneous gestures in human face-to-face interaction as explored with eye-tracking. Some basic fixation patterns are described, live and video-based settings are compared, and preliminary results on the relationship between fixations and information processing are outlined.

Abstract

This paper discusses how the gestural modality can contribute to our understanding of anaphoric linkage in learner varieties, focusing on gestural anaphoric linkage marking the introduction, maintenance, and shift of reference in story retellings by learners of French and Swedish. The comparison of gestural anaphoric linkage in native and non-native varieties reveals what appears to be a particular learner variety of gestural cohesion, which closely reflects the characteristics of anaphoric linkage in learners' speech. Specifically, particular forms co-occur with anaphoric gestures depending on the information organisation in discourse. The typical nominal over-marking of maintained referents or topic elements in speech is mirrored by gestural (over-)marking of the same items. The paper discusses two ways in which this finding may further the understanding of anaphoric over-explicitness of learner varieties. An addressee-based communicative perspective on anaphoric linkage highlights how over-marking in gesture and speech may be related to issues of hyper-clarity and ambiguity. An alternative speaker-based perspective is also explored in which anaphoric over-marking is seen as related to L2 speech planning.

Abstract

This chapter aims to provide an updated list of all Bantu languages known at present and to provide individual pointers to further information on the inventory. The area division has some correlation with what are perceived genealogical relations between Bantu languages, but they are not defined as such and do not change whenever there is an update in our understanding of genealogical relations. Given the popularity of Guthrie codes in Bantu linguistics, our listing also features a complete mapping to Guthrie codes. The language inventory listed excludes sign languages used in the Bantu area, speech registers, pidgins, drummed/whistled languages and urban youth languages. Pointers to such languages in the Bantu area are included in the continent-wide overview in Hammarstrom. The most important alternative names, subvarieties and spelling variants are given for each language, though such lists are necessarily incomplete and reflect some degree of arbitrary selection.

Abstract

This thesis elaborates on a methodological approach to reliably infer cognitive preferences in an extinct evolutionary ancestor of modern humans. In attempts to understand cognitive evolution, humans have been compared to capuchin monkeys, tamarins, and chimpanzees to name but a few. But comparisons between humans and one other, maybe even distantly related primate, as interesting as they might be, will not tell us anything about an evolutionary ancestor to humans. To put it bluntly: None of the living primates, not even chimpanzees, are a human ancestor. With that in mind, we can still use a comparative approach to gain information about our evolutionary ancestors, as long as we are careful about whom we compare with whom. If a certain trait exists in all genera of a phylogenetic clade, it was most likely present in their common ancestor. The great apes are such a clade (Pongo, Gorilla, Pan and Homo). It follows that, if members of all great ape genera shared a particular cognitive preference or ability, it is most likely part of the evolutionary inheritance of the clade at least ever since their last common ancestor, and therefore also an evolutionarily old, inherited cognitive default in humans. This thesis contains studies comparing all 4 extant Hominid genera, including humans of 4 different age-groups and 2 different cultures. Results show that all great apes do indeed share some cognitive preferences, which they most likely inherited from an evolutionary ancestor. Additionally, human cognitive preferences can change away from such an inherited predisposition given ontogenetic factors, and are at least in part variably adaptable to cultural circumstance.

Abstract

Conversation Analysis (CA) is an inductive, micro-analytic, and predominantly qualitative
method for studying human social interactions. This chapter describes and illustrates the basic
methods of CA. We first situate the method by describing its sociological foundations, key areas
of analysis, and particular approach in using naturally occurring data. The bulk of the chapter is
devoted to practical explanations of the typical conversation analytic process for collecting data
and producing an analysis. We analyze a candidate interactional practice – the assessmentimplicative
interrogative – using real data extracts as a demonstration of the method, explicitly
laying out the relevant questions and considerations for every stage of an analysis. The chapter
concludes with some discussion of quantitative approaches to conversational interaction, and
links between CA and psycholinguistic concerns

Abstract

This chapter investigates the fit of genetic, phenotypic, and linguistic data to two well-known models of population history. The first of these models, termed the population fissions model, emphasizes population splitting, isolation, and independent evolution. It predicts that genetic and linguistic data will be perfectly tree-like. The second model, termed isolation by distance, emphasizes genetic exchange among geographically proximate populations. It predicts a monotonic decline in genetic similarity with increasing geographic distance. While these models are overly simplistic, deviations from them were expected to provide important insights into the population history of northern Island Melanesia. The chapter finds scant support for either model because the prehistory of the region has been so complex. Nonetheless, the genetic and linguistic data are consistent with an early radiation of proto-Papuan speakers into the region followed by a much later migration of Austronesian speaking peoples. While these groups subsequently experienced substantial genetic and cultural exchange, this exchange has been insufficient to erase this history of separate migrations.

Abstract

Neurocognitive studies of language production have provided sufficient evidence on both the spatial and the temporal patterns of brain activation to allow tentative and in some cases not so tentative conclusions about function-structure relationships. This chapter reports meta-analysis results that identify reliable activation areas for a range of word, sentence, and narrative production tasks both in the native language and a second language. Based on a theoretically motivated analysis of language production tasks it is possible to specify relationships between brain areas and functional processing components of language production that could not have been derived from the data provided by any single task.

Abstract

This chapter deals with the question of whether there is one syntactic system that is shared by language production and comprehension or whether there are two separate systems. It first discusses arguments in favor of one or the other option and then presents the current evidence on the brain structures involved in sentence processing. The results of meta-analyses of numerous neuroimaging studies suggest that there is one system consisting of functionally distinct cortical regions: the dorsal part of Broca’s area subserving compositional syntactic processing; the ventral part of Broca’s area subserving compositional semantic processing; and the left posterior temporal cortex (Wernicke’s area) subserving the retrieval of lexical syntactic and semantic information. Sentence production, the comprehension of simple and complex sentences, and the parsing of sentences containing grammatical violations differ with respect to the recruitment of these functional components.

Abstract

This chapter reviews the findings of 58 word production experiments using different tasks and neuroimaging techniques. The reported cerebral activation sites are coded in a common anatomic reference system. Based on a functional model of language production, the different word production tasks are analyzed in terms of their processing components. This approach allows a distinction between the core process of word production and preceding task-specific processes (lead-in processes) such as visual or auditory stimulus recognition. The core process of word production is subserved by a left-lateralized perisylvian/thalamic language production network. Within this network there seems to be functional specialization for the processing stages of word production. In addition, this chapter includes a discussion of the available evidence on syntactic production, self-monitoring, and the time course of word production.

Abstract

Computer models of cultural evolution have shown language properties emerging on interacting agents with a brain that lacks dedicated, nativist language modules. Notably, models using Bayesian agents provide a precise specification of (extra-)liguististic factors (e.g., genetic) that shape language through iterated learning (biases on language), and demonstrate that weak biases get expressed more strongly over time (bias amplification). Other models attempt to lessen assumption on agents’ innate predispositions even more, and emphasize self-organization within agents, highlighting glossogenesis (the development of language from a nonlinguistic state). Ultimately however, one also has to recognize that biology and culture are strongly interacting, forming a coevolving system. As such, computer models show that agents might (biologically) evolve to a state predisposed to language adaptability, where (culturally) stable language features might get assimilated into the genome via Baldwinian niche construction. In summary, while many questions about language evolution remain unanswered, it is clear that it is not to be completely understood from a purely biological, cognitivist perspective. Language should be regarded as (partially) emerging on the social interactions between large populations of speakers. In this context, agent models provide a sound approach to investigate the complex dynamics of genetic biasing on language and speech

Abstract

The configuration of a spatial layout has a substantial effect on the acquisition and the representation of the environment. In four experiments, we investigated navigation difficulties arising at oblique angled intersections. In the first three studies we investigated specific arrow-fork configurations. In dependence on the branch subjects use to enter the intersection different decision latencies and numbers of errors arise. If subjects see the intersection as a fork, it is more difficult to find the correct way as if it is seen as an arrow. In a fourth study we investigated different heuristics people use while making a detour around a barrier. Detour behaviour varies with the perspective. If subjects learn and navigate through the maze in a field perspective they use a heuristic of preferring right angled paths. If they have a view from above and acquire their knowledge in an observer perspective they use oblique angled paths more often.

Abstract

It is an important feature of the human sentence production system that semantic and syntactic processes may overlap in time and do not proceed strictly serially. That is, the process of building the syntactic form of an utterance does not always wait until the complete semantic content for that utterance has been decided upon. On the contrary, speakers will often start pronouncing the first words of a sentence while still working on further details of its semantic content. An important advantage is memory economy. Semantic and syntactic fragments do not have to occupy working memory until complete semantic and syntactic structures for an utterance have been computed. Instead, each semantic and syntactic fragment is processed as soon as possible and is kept in working memory for a minimum period of time. This raises the question of how the sentence production system can maintain syntactic coherence across syntactic fragments. Presumably there are processes of "syntactic bookkeeping" which (1) store in working memory those syntactic properties of a fragmentary sentence which are needed to eliminate ungrammatical continuations, and (2) check whether a prospective continuation is indeed compatible with the sentence constructed so far. In reaction time experiments where subjects described, under time pressure, simple static pictures of an action performed by an actor, the second aspect of syntactic bookkeeping could be demonstrated. This evidence is used for modelling bookkeeping processes as part of a computational sentence generator which aims at simulating the syntactic operations people carry out during spontaneous speech.

Abstract

This paper is an abridged version (in Dutch) of an in-press article by the same author (Kempen, G. (2008/9). Clausal coordination and coordinate ellipsis in a model of the speaker. To be published in: Linguistics). The two papers present a psycholinguistically inspired approach to the syntax of clause-level coordination and coordinate ellipsis. It departs from the assumption that coordinations are structurally similar to so-called appropriateness repairs Ñ an important type of self-repairs in spontaneous speech. Coordinate structures and appropriateness repairs can both be viewed as ÒupdateÓ con-structions. Updating is defined as a special sentence production mode that efficiently revises or augments existing sentential structure in response to modifications in the speakerÕs communicative intention. This perspective is shown to offer an empirically satisfactory and theoretically parsimonious account of two prominent types of coordinate ellipsis, in particular Forward Conjunction Reduction (FCR) and Gapping (including Long-Distance Gapping and Subgapping). They are analyzed as different manifestations of Òincremental updatingÓ Ñ efficient updating of only part of the existing sentential structure. Based on empirical data from Dutch and German, novel treatments are proposed for both types of clausal coordinate ellipsis. Two other forms of coordinate ellipsis Ñ SGF (ÒSubject Gap in Finite clauses with fronted verbÓ), and Backward Conjunction Reduction (BCR; also known as Right Node Raising or RNR) Ñ are shown to be incompatible with the notion of incremental updating. Alternative theoretical interpretations of these phenomena are proposed. The four types of clausal coordinate ellipsis Ñ SGF, Gapping, FCR and BCR Ñ are argued to originate in four different stages of sentence production: Intending (i.e. preparing the communicative intention), Conceptualization, Grammatical Encoding, and Phonological Encoding, respectively.