Publications

Abstract

Words evolve not as blobs of ink on paper but in face to face interaction. The nature of language as fundamentally interactive and multimodal is shown by the study of ideophones, vivid sensory words that thrive in conversations around the world. The ways in which these Lautbilder enable precise communication about sensory knowledge has for the first time been studied in detail. It turns out that we can paint with language, and that the onomatopoeia we sometimes classify as childish might be a subset of a much richer toolkit for depiction in speech, available to us all.

Abstract

The Mawu people of eastern Ghana make common use of ideophones: marked words that depict sensory imagery. Ideophones have been described as “poetry in ordinary language,” yet the shadow of Lévy-Bruhl, who assigned such words to the realm of primitivity, has loomed large over linguistics and literary theory alike. The poet Ezra Pound is a case in point: while his fascination with Chinese characters spawned the ideogrammic method, the mimicry and gestures of the “primitive languages in Africa” were never more than a mere curiosity to him. This paper imagines Pound transposed into the linguaculture of the Mawu. What would have struck him about their ways of ‘charging language’ with imagery? I juxtapose Pound’s views of the poetic image with an analysis of how different layers of iconicity in ideophones combine to depict sensory imagery. This exercise illuminates aspects of what one might call ‘the ideophonic

Abstract

Informal paraphrases by native speaker consultants are crucial tools in linguistic fieldwork. When recorded, archived, and analysed, they offer rich data that can be mined for many purposes, from lexicography to semantic typology and from ethnography to the investigation of gesture and speech. This paper describes a procedure for the collection and analysis of folk definitions that are native (in the language under study rather than the language of analysis), informal (spoken rather than written), and multi-modal (preserving the integrity of gesture-speech composite utterances). The value of folk definitions is demonstrated using the case of ideophones, words that are notoriously hard to study using traditional elicitation methods. Three explanatory strategies used in a set of folk definitions of ideophones are examined: the offering of everyday contexts of use, the use of depictive gestures, and the use of sense relations as semantic anchoring points. Folk definitions help elucidate word meanings that are hard to capture, bring to light cultural background knowledge that often remains implicit, and take seriously the crucial involvement of native speaker consultants in linguistic fieldwork. They provide useful data for language documentation and are an essential element of any toolkit for linguistic and ethnographic field research.

Abstract

Doing things for and with others is one of the foundations of human social life. This chapter studies a systematic collection of 207 requests for assistance and collaboration from a video corpus of everyday conversations in Siwu, a Kwa language of Ghana. A range of social action formats and semiotic resources reveals how language is adapted to the interactional challenges posed by recruiting assistance. While many of the formats bear a language-specific signature, their sequential and interactional properties show important commonalities across languages. Two tentative findings are put forward for further cross-linguistic examination: a “rule of three” that may play a role in the organisation of successive response pursuits, and a striking commonality in animal-oriented recruitments across languages that may be explained by convergent cultural evolution. The Siwu recruitment system emerges as one instance of a sophisticated machinery for organising collaborative action that transcends language and culture.

Abstract

Spatial congruity effects have often been interpreted as evidence for metaphorical thinking, but an alternative markedness-based account challenges this view. In two experiments, we directly compared metaphor and markedness explanations for spatial congruity effects, using musical pitch as a testbed. English speakers who talk about pitch in terms of spatial height were tested in speeded space-pitch compatibility tasks. To determine whether space-pitch congruency effects could be elicited by any marked spatial continuum, participants were asked to classify high- and low-frequency pitches as 'high' and 'low' or as 'front' and 'back' (both pairs of terms constitute cases of marked continuums). We found congruency effects in high/low conditions but not in front/back conditions, indicating that markedness is not sufficient to account for congruity effects (Experiment 1). A second experiment showed that congruency effects were specific to spatial words that cued a vertical schema (tall/short), and that congruity effects were not an artifact of polysemy (e.g., 'high' referring both to space and pitch). Together, these results suggest that congruency effects reveal metaphorical uses of spatial schemas, not markedness effects.

Abstract

Some languages describe musical pitch in terms of spatial height; others in terms of thickness. Differences in pitch metaphors also shape adults’ nonlinguistic space-pitch representations. At the same time, 4-month-old infants have both types of space-pitch mappings available. This tension between prelinguistic space-pitch associations and their subsequent linguistic mediation raises questions about the acquisition of space-pitch metaphors. To address this issue, 5-year-old Dutch children were tested on their linguistic knowledge of pitch metaphors, and nonlinguistic space-pitch associations. Our results suggest 5-year-olds understand height-pitch metaphors in a reversed fashion (high pitch = low). Children displayed good comprehension of a thickness-pitch metaphor, despite its absence in Dutch. In nonlinguistic tasks, however, children did not show consistent space-pitch associations. Overall, pitch representations do not seem to be influenced by linguistic metaphors in 5-year-olds, suggesting that effects of language on musical pitch arise rather late during development.

Abstract

Humans readily generalize, applying prior knowledge to novel situations and stimuli. Advances in machine learning have begun to approximate and even surpass human performance, but these systems struggle to generalize what they have learned to untrained situations. We present a model based on wellestablished neurocomputational principles that demonstrates human-level generalisation. This model is trained to play one video game (Breakout) and performs one-shot generalisation to a new game (Pong) with different characteristics. The model
generalizes because it learns structured representations that are functionally symbolic (viz., a role-filler binding calculus) from unstructured training data. It does so without feedback, and without requiring that structured representations are specified a priori. Specifically, the model uses neural co-activation to discover which characteristics of the input are invariant and to learn relational predicates, and oscillatory regularities in network firing to bind predicates to arguments. To our knowledge,
this is the first demonstration of human-like generalisation in a machine system that does not assume structured representa-
tions to begin with.

Abstract

Research into sound-symbolism has shown that people can
consistently associate certain pseudo-words with certain referents;
for instance, pseudo-words with rounded vowels and
sonorant consonants are linked to round shapes, while pseudowords
with unrounded vowels and obstruents (with a noncontinuous
airflow), are associated with sharp shapes. Such
sound-symbolic associations have been proposed to arise from
cross-modal abstraction processes. Here we assess the link between
sound-symbolism and cross-modal abstraction by testing
dyslexic individuals’ ability to make sound-symbolic associations.
Dyslexic individuals are known to have deficiencies
in cross-modal processing. We find that dyslexic individuals
are impaired in their ability to make sound-symbolic associations
relative to the controls. Our results shed light on the cognitive
underpinnings of sound-symbolism by providing novel
evidence for the role —and disruptability— of cross-modal abstraction
processes in sound-symbolic eects.

Abstract

This paper summarizes the results of a descriptive syntactic category analysis of child English no which reveals that young children use and represent no as a determiner and negatives like no pen as NPs, contra standard analyses.

Abstract

Previous research revealed remarkable flexibility of native and non-native listeners’ perceptual system, i.e., native and non-native phonetic category boundaries can be quickly recalibrated in the face of ambiguous input.
The present study investigates the limitations of the flexibility of the non-native perceptual system. In two lexically-guided perceptual learning experiments, Dutch listeners were exposed to a short story in English, where either all /l/ or all /ɹ/ sounds were replaced by an ambiguous [l/ɹ] sound. In the first experiment, the story was presented in clean, while in the second experiment, intermittent noise was added to the story, although never on the critical words. Lexically-guided perceptual learning was only observed in the clean condition. It is argued that the introduction of intermittent noise reduced the reliability of the evidence of hearing a particular word, which in turn blocked retuning of the phonetic categories.

Abstract

The article analyzes the relation between Aweti and Kamayurá on different levels. Both languages belong to different branches of the subfamily “Maweti-Guarani” within the large Tupi ‘stock’. Both peoples have arrived rather late to the complex Upper Xinguan society, but probably independently and from different directions. Both resulted from mergers of different groups and suffered a dramatic demographic decline in the first half of last century. There is no concrete evidence that these groups spoke varieties of more than 2 different languages (Pre-Aweti and Pre-Kamayurá). Today, many Aweti are at least passive bilinguals with Kamayurá, their most important allies, but the opposite does not hold. The article also discusses the relations between the languages on the main structural levels. In phonology, the phoneme inventories are compared and the sound changes are listed that occurred from the hypothetical proto-language “Proto-Maweti-Guarani” to Aweti, on the one hand, and to Proto-Tupi-Guarani and further to Kamayurá, on the other. In morpho-syntax, the article offers a comparison of the person systems and of affixes in general, treating in particular the so-called ‘relational prefixes’, which do not exist in Aweti. The most important syntactic shared properties are also listed. There seem to be very little mutual lexical borrowing. In the appendix, a list of more than 60 cognates with reconstructed proto-forms is given. Key-words: Aweti; Kamayurá; Sociolinguistics; History; Phonology.

Abstract

A key for understanding the Upper Xingu system is the comparison of the different languages which are part of that multilingual society. This article discusses the notion ‘comparing languages’ and delineates a research program in accordance to which a fruitful comparison can be done on four levels: 1) structural (phonological and morphosyntactic), 2) lexical (semantic structure of the lexica and individual lexical items), 3) discourse (figures of speech and thought), 4) content (in particular, narratives). The language data of the project gathered so far (focusing on level 2 and 4) is described in detail: 10 comparative word lists from different semantic domains, and a core of 5 analogous texts of different genera. Finally, some general considerations are offered about how to analyze both similarities and divergence found among the compared material.

Abstract

In recent years, large scale initiatives like CLARIN set out to overcome the notorious heterogeneity of metadata formats in the domain of language resource. The CLARIN Component Metadata Infrastructure established means for flexible resouce descriptions for the domain of language resources. The Data Category Registry ISOcat and the accompanying Relation Registry foster semantic interoperability within the growing heterogeneous collection of metadata records. This paper describes the CMD Infrastructure focusing on the facilities for semantic mapping, and gives also an overview of the current status in the joint component metadata domain.

Abstract

This revised version supersedes all previous versions (e.g., Field Manual 2010).

Abstract

This article describes the grammatical resources available to speakers of Lao for describing situations that can be described broadly as ‘reciprocal’. The analysis is based on complementary methods: elicitation by means of non-linguistic stimuli, exploratory consultation with native speakers, and investigation of corpora of spontaneous language use. Typically, reciprocal situations are described using a semantically general ‘collaborative’ marker on an action verb. The resultant meaning is that some set of people participate in a situation ‘together’, broadly construed. The collaborative marker is found in two distinct syntactic constructions, which differ in terms of their information structural contexts of use. The paper first explores in detail the semantic range of the collaborative marker as it occurs in the more common ‘Type 1’ construction, and then discusses a special pragmatic context for the ‘Type 2’ construction. There is some methodological discussion concerning the results of elicitation via video stimuli. The chapter also discusses two specialised constructions dedicated to the expression of strict reciprocity.

Abstract

(from the chapter) Recognizing others' goals in the flow of interaction is complex, not only for analysts but for participants too. This chapter explores a semiotic approach, with the utterance-in-context as a basic-level unit, and where the interpreter, not the producer, is the driving force in how utterances come to have meaning. We first want to know how people extract meaning from others' communicative behavior. We then ask what are the elements of producers' formulation of communicative actions in anticipation of how others will interpret that behavior.

Abstract

Introduction

A comparison of conversation in twenty-one languages from around the world reveals commonalities and differences in the way that people do open-class other-initiation of repair (Schegloff, Jefferson, and Sacks, 1977; Drew, 1997). We find that speakers of all of the spoken languages in the sample make use of a primary interjection strategy (in English it is Huh?), where the phonetic form of the interjection is strikingly similar across the languages: a monosyllable featuring an open non-back vowel [a, æ, ə, ʌ], often nasalized, usually with rising intonation and sometimes an [h-] onset. We also find that most of the languages have another strategy for open-class other-initiation of repair, namely the use of a question word (usually “what”). Here we find significantly more variation across the languages. The phonetic form of the question word involved is completely different from language to language: e.g., English [wɑt] versus Cha'palaa [ti] versus Duna [aki]. Furthermore, the grammatical structure in which the repair-initiating question word can or must be expressed varies within and across languages. In this chapter we present data on these two strategies – primary interjections like Huh? and question words like What? – with discussion of possible reasons for the similarities and differences across the languages. We explore some implications for the notion of repair as a system, in the context of research on the typology of language use.

The general outline of this chapter is as follows. We first discuss repair as a system across languages and then introduce the focus of the chapter: open-class other-initiation of repair. A discussion of the main findings follows, where we identify two alternative strategies in the data: an interjection strategy (Huh?) and a question word strategy (What?). Formal features and possible motivations are discussed for the interjection strategy and the question word strategy in order. A final section discusses bodily behavior including posture, eyebrow movements and eye gaze, both in spoken languages and in a sign language.

Abstract

This version is reprinted from the 2010 Field Manual

Abstract

This chapter contains sections titled: Introduction Lexical Selection in Reference: Introductory Examples of Reference to Times Multiple “Preferences” Future Directions Conclusion

Abstract

The larynx is the essential articulatory mechanism that primes the vocal tract. Far from being only a glottal source of voicing, the complex laryngeal mechanism entrains the ontogenetic acquisition of speech and, through coarticulatory coupling, guides the production of oral sounds in the infant vocal tract. As such, it is not possible to speculate as to the origins of the speaking modality in humans without considering the fundamental role played by the laryngeal articulatory mechanism. The Laryngeal Articulator Model, which divides the vocal tract into a laryngeal component and an oral component, serves as a basis for describing early infant speech and for positing how speech sounds evolving in various hominids may be related phonetically. To this end, we offer some suggestions for how the evolution and development of vocal tract anatomy fit with our infant speech acquisition data and discuss the implications this has for explaining phonetic learning and for interpreting the biological evolution of the human vocal tract in relation to speech and speech acquisition.

Abstract

Reciprocity lies at the heart of social cognition, and with it so does the encoding of reciprocity in language via reciprocal constructions. Despite the prominence of strong universal claims about the semantics of reciprocal constructions, there is considerable descriptive literature on the semantics of reciprocals that seems to indicate variable coding and subtle cross-linguistic differences in meaning of reciprocals, both of which would make it impossible to formulate a single, essentialising definition of reciprocal semantics. These problems make it vital for studies in the semantic typology of reciprocals to employ methodologies that allow the relevant categories to emerge objectively from cross-linguistic comparison of standardised stimulus materials. We situate the rationale for the 20-language study that forms the basis for this book within this empirical approach to semantic typology, and summarise some of the findings.

Abstract

The aim of the present work is to identify the evolutionary origins of the ability to speak and understand a natural language. I will adopt Botha’s “Windows Approach” (Language and Communication, 2006, 26, pp. 129–143) in order to justify the following two assumptions, which concern the evolutionary continuity between human language and animals’ communication systems: (a) despite the uniqueness of human language in sharing and conveying utterances with an open-ended structure, some isolated components of our linguistic competence are shared with non- human primates, grounding a line of evolutionary continuity; (b) the very first “linguistic” utterances were holistic, that is, whole bunches of sounds able to convey information despite their lack of modern syntax. I will address such suppositions through the comparative analysis of three constitutive features of human language: syntax, the semantic value of utterances, and the ability to attribute mental states to conspecifics, i.e. the theory of mind.

Abstract

Relative clause processing depends on the grammatical role of the head noun in the subordinate clause. This has traditionally been explained in terms of cognitive limitations. We suggest that structure-related processing differences arise from differences in experience with these structures. We present a connectionist model which learns to produce utterances with relative clauses from exposure to message-sentence pairs. The model shows how various factors such as frequent subsequences, structural variations, and meaning conspire to create differences in the processing of these structures. The predictions of this learning-based account have been confirmed in behavioral studies with adults. This work shows that structural regularities that govern relative clause processing can be explained within a usage-based approach to recursion.

Abstract

Language acquisition involves learning nonadjacent dependencies that can obtain between words in a sentence. Several artificial grammar learning studies have shown that the ability of adults and children to detect dependencies between A and B in frames AXB is influenced by the amount of variation in the X element. This paper presents a model of statistical learning which displays similar behavior on this task and generalizes in a human-like way. The model was also used to predict human behavior for increased distance and more variation in dependencies. We compare our model-based approach with the standard invariance account of the variability effect.

Abstract

The ethnographic literature has sometimes described parts of the northwest Amazon as areas of shared culture across linguistic groups. This paper illustrates how a principle of semantic transparency across languages is a key means of establishing elements of a common regional culture through practices like the calquing of ethnonyms and toponyms so that they are semantically, but not phonologically, equivalent across languages. It places the upper Rio Negro area of the northwest Amazon in a general discussion of cross-linguistic naming practices in South America and considers the extent to which a preference for semantic transparency can be linked to cases of widespread cultural ‘calquing’, in which culturally-important meanings are kept similar across different linguistic systems. It also addresses the principle of semantic transparency beyond specific referential phrases and into larger discourse structures. It concludes that an attention to semiotic practices in multilingual settings can provide new and more complex ways of thinking about the idea of shared culture.

Abstract

This study aims to test a prediction of recent
theoretical frameworks in speech motor control: if speech production targets are specified in auditory
terms, people with better auditory acuity should have more precise speech targets.
To investigate this, we had participants perform speech perception and production tasks in a counterbalanced order. To assess speech perception acuity, we used an adaptive speech discrimination
task. To assess variability in speech production, participants performed a pseudo-word reading task; formant values were measured for each recording.
We predicted that speech production variability to correlate inversely with discrimination performance.
The results suggest that people do vary in their production and perceptual abilities, and that better discriminators have more distinctive vowel production targets, confirming our prediction. This
study highlights the importance of individual
differences in the study of speech motor control, and sheds light on speech production-perception interaction.

Abstract

Acquiring language is notoriously complex, yet for the majority of children this feat is accomplished with remarkable ease. Usage-based accounts of language acquisition suggest that this success can be largely attributed to the wealth of experience with language that children accumulate over the course of language acquisition. One field of research that is heavily underpinned by this principle of experience is statistical learning, which posits that learners can perform powerful computations over the distribution of information in a given input, which can help them to discern precisely how that input is structured, and how it operates. A growing body of work brings this notion to bear in the field of language acquisition, due to a developing understanding of the richness of the statistical information contained in speech. In this chapter we discuss the role that statistical learning plays in language acquisition, emphasising the importance of both the distribution of information within language, and the situation in which language is being learnt. First, we address the types of statistical learning that apply to a range of language learning tasks, asking whether the statistical processes purported to support language learning are the same or distinct across different tasks in language acquisition. Second, we expand the perspective on what counts as environmental input, by determining how statistical learning operates over the situated learning environment, and not just sequences of sounds in utterances. Finally, we address the role of variability in children’s input, and examine how statistical learning can accommodate (and perhaps even exploit) this during language acquisition.

Abstract

We propose a Random-Forest based sign language identification system. The system uses low-level visual features and is based on the hypothesis that sign languages have varying distributions of phonemes (hand-shapes, locations and movements). We evaluated the system on two sign languages -- British SL and Greek SL, both taken from a publicly available corpus, called Dicta Sign Corpus. Achieved average F1 scores are about 95% - indicating that sign languages can be identified with high accuracy using only low-level visual features.

Abstract

We present a vision-based method for signer diarization -- the task of automatically determining "who signed when?" in a video. This task has similar motivations and applications as speaker diarization but has received little attention in the literature. In this paper, we motivate the problem and propose a method for solving it. The method is based on the hypothesis that signers make more movements than their interlocutors. Experiments on four videos (a total of 1.4 hours and each consisting of two signers) show the applicability of the method. The best diarization error rate (DER) obtained is 0.16.

Abstract

This paper presents a Native Language Identification (NLI) system based on TF-IDF weighting schemes and using linear classifiers - support vector machines, logistic regressions and perceptrons. The system was one of the participants of the 2013 NLI Shared Task in the closed-training track, achieving 0.814 overall accuracy for a set of 11 native languages. This accuracy was only 2.2 percentage points lower than the winner's performance. Furthermore, with subsequent evaluations using 10-fold cross-validation (as given by the organizers) on the combined training and development data, the best average accuracy obtained is 0.8455 and the features that contributed to this accuracy are the TF-IDF of the combined unigrams and bigrams of words.

Abstract

We present and solve the speaker diarization problem in a novel way. We hypothesize that the gesturer is the speaker and that identifying the gesturer can be taken as identifying the active speaker. We provide evidence in support of the hypothesis from gesture literature and audio-visual synchrony studies. We also present a vision-only diarization algorithm that relies on gestures (i.e. upper body movements). Experiments carried out on 8.9 hours of a publicly available dataset (the AMI meeting data) show that diarization error rates as low as 15% can be achieved.

Abstract

How are space and time related in the brain? This study contrasts two proposals that make different predictions about the interaction between spatial and temporal magnitudes. Whereas ATOM implies that space and time are symmetrically related, Metaphor Theory claims they are asymmetrically related. Here we investigated whether space and time activate the same neural structures in the inferior parietal cortex (IPC) and whether the activation is symmetric or asymmetric across domains. We measured participants’ neural activity while they made temporal and spatial judgments on the same visual stimuli. The behavioral results replicated earlier observations of a space-time asymmetry: Temporal judgments were more strongly influenced by irrelevant spatial information than vice versa. The BOLD fMRI data indicated that space and time activated overlapping clusters in the IPC and that, consistent with Metaphor Theory, this activation was asymmetric: The shared region of IPC was activated more strongly during temporal judgments than during spatial judgments. We consider three possible interpretations of this neural asymmetry, based on 3 possible functions of IPC.

Abstract

Using a poorly mastered second language (L2) in interaction with a native speaker is a challenging task. This paper explores how L2 speakers and their native interlocutors together deploy gestures and speech to sustain problematic interaction. Drawing on native and non-native interactions in Swedish, French, and Dutch, I examine lexical, grammatical and interaction-related problems in turn. The analyses reveal that (a) different problems yield behaviours with different formal and interactive properties that are common across the language pairs and the participant roles; (b) native and non-native behaviour differs in degree, not in kind; and (c) that individual communicative style determines behaviour more than the gravity of the linguistic problem. I discuss the implications for theories opposing 'efficient' L2 communication to learning. Also, contra the traditional view of compensatory gestures, I will argue for a multi-functional 'hydraulic' view grounded in gesture theory where speech and gesture are equal partners, but where the weight carried by the modalities shifts depending on expressive pressures.

Abstract

This study focuses on the effect of the semantics of placement verbs on placement event representations. Specifically, it explores to what extent the semantic properties of habitually used verbs guide attention to certain types of spatial information. French, which typically uses a general placement verb (mettre, 'put'), is contrasted with Dutch, which uses a set of fine-grained (semi-)obligatory posture verbs (zetten, leggen, 'set/stand', 'lay'). Analysis of the concomitant gesture production in the two languages reveals a patterning toward two distinct, language-specific event representations. The object being placed is an essential part of the Dutch representation, while French speakers instead focus only on the (path of the) placement movement. These perspectives permeate the entire placement domain regardless of the actual verb used.

Abstract

A key problem in studies of bilingual linguistic cognition is how to probe the details of underlying representations in order to gauge whether bilinguals' conceptualizations differ from those of monolinguals, and if so how. This chapter provides an overview of a line of studies that rely on speech-associated gestures to explore these issues. The gestures of adult monolingual native speakers differ systematically across languages, reflecting consistent differences in what information is selected for expression and how it is mapped onto morphosyntactic devices. Given such differences, gestures can provide more detailed information on how multilingual speakers conceptualize events treated differently in their respective languages, and therefore, ultimately, on the nature of their representations. This chapter reviews a series of studies in the domain of (voluntary and caused) motion event construal. I first discuss speech and gesture evidence for different construals in monolingual native speakers, then review studies on second language speakers showing gestural evidence of persistent L1 construals, shifts to L2 construals, and of bidirectional influences. I consider the implications for theories of ultimate attainment in SLA, transfer and convergence. I will also discuss the methodological implications, namely what gesture data do and do not reveal about linguistic conceptualisation and linguistic relativity proper.

Abstract

The shape of pitch contours has been shown to have an effect on the perceived duration of vowels. For instance, vowels with high level pitch and vowels with falling contours sound longer than vowels with low level pitch. Depending on whether the
comparison is between level pitches or between level and dynamic contours, these findings have been interpreted in two ways. For inter-level comparisons, where the duration results are the reverse of production results, a hypercorrection strategy in production has been proposed [1]. By contrast, for comparisons between level pitches and dynamic contours, the
longer production data for dynamic contours have been held responsible. We report an experiment with Dutch and Chinese listeners which aimed to show that production data and perception data are each other’s opposites for high, low, falling and rising contours. We explain the results, which are consistent with earlier findings, in terms of the compensatory listening strategy of [2], arguing that the perception effects are due to a perceptual compensation of articulatory strategies and
constraints, rather than that differences in production compensate for psycho-acoustic perception effects.

Abstract

This chapter sketches in very general terms the cognitive architecture of both language comprehension and production, as well as the neurobiological infrastructure that makes the human brain ready for language. Focus is on spoken language, since that compares most directly to processing music. It is worth bearing in mind that humans can also interface with language as a cognitive system using sign and text (visual) as well as Braille (tactile); that is to say, the system can connect with input/output processes in any sensory modality. Language processing consists of a complex and nested set of subroutines to get from sound to meaning (in comprehension) or meaning to sound (in production), with remarkable speed and accuracy. The fi rst section outlines a selection of the major constituent operations, from fractionating the input into manageable units to combining and unifying information in the construction of meaning. The next section addresses the neurobiological infrastructure hypothesized to form the basis for language processing. Principal insights are summarized by building on the notion of “brain networks” for speech–sound processing, syntactic processing, and the construction of meaning, bearing in mind that such a neat three-way subdivision overlooks important overlap and shared mechanisms in the neural architecture subserving language processing. Finally, in keeping with the spirit of the volume, some possible relations are highlighted between language and music that arise from the infrastructure developed here. Our characterization of language and its neurobiological foundations is necessarily selective and brief. Our aim is to identify for the reader critical questions that require an answer to have a plausible cognitive neuroscience of language processing.

Abstract

This paper joins studies of linguistic variation (e.g. Labov 1972; Dubois & Horvath 2000) and discourse (e.g. Jaworski & Lawson 2005; Jaworski & Pritchard 2005; Thurlow & Jaworski 2010) that consider the intersection between language and tourism. By examining the language attitudes that tourists hold toward linguistic variability in their host community, we find that attitudes differ by context and with respect to tourists’ travel motivations. We suggest that these results are particularly likely in a context like Edinburgh, Scotland, where linguistic variation has an iconic link to place authenticity. We propose that the joint commodification of ‘intelligibility’ and ‘authenticity’ explains this variability. The results raise questions about how the commodity value of travel motivation and the associated context of language use influence language attitudes.

Abstract

The present paper considers the problem of annotating bibliographical references with labels/classes, given training data of references already annotated with labels. The problem is an instance of document categorization where the documents are short and written in a wide variety of languages. The skewed distributions of title words and labels calls for special carefulness when choosing a Machine Learning approach. The present paper describes how to induce Disjunctive Normal Form formulae (DNFs), which have several advantages over Decision Trees. The approach is evaluated on a large real-world collection of bibliographical references.

Abstract

This corpus study demonstrates that the realization of wordfinal /t/ in Dutch past-participles in various speech styles is affected by a word’s predictability and paradigmatic relative frequency. In particular, /t/s are shorter and more often absent if the two preceding words are more predictable. In addition, /t/s, especially in irregular verbs, are more reduced, the lower the verb’s lemma frequency relative to the past-participle’s frequency. Both effects are more pronounced in more spontaneous speech. These findings are expected if speech planning plays an important role in speech reduction. Index Terms: pronunciation variation, acoustic reduction, corpus research, word predictability, morphological decomposability

Abstract

This paper studies the robustness of exemplar effects in word comprehension by means of four long-term priming experiments with lexical decision tasks in Dutch. A prime and target represented the same word type and were presented with the same or different degree of reduction. In Experiment 1, participants heard only a small number of trials, a large proportion of repeated words, and stimuli produced by only one speaker. They recognized targets more quickly if these represented the same degree of reduction as their primes, which forms additional evidence for the exemplar effects reported in the literature. Similar effects were found for two speakers who differ in their pronunciations. In Experiment 2, with a smaller proportion of repeated words and more trials between prime and target, participants recognized targets preceded by primes with the same or a different degree of reduction equally quickly. Also, in Experiments 3 and 4, in which listeners were not exposed to one but two types of pronunciation variation (reduction degree and speaker voice), no exemplar effects arose. We conclude that the role of exemplars in speech comprehension during natural conversations, which typically involve several speakers and few repeated content words, may be smaller than previously assumed.

Abstract

Students who are learning to write in a foreign language, often want feedback on the grammatical quality of the sentences they produce. The usual NLP approach to this problem is based on parsing student-generated text. Here, we propose a generation-based ap- proach aiming at preventing errors ("scaffolding"). In our ICALL system, the student constructs sentences by composing syntactic trees out of lexically anchored "treelets" via a graphical drag & drop user interface. A natural-language generator computes all possible grammatically well-formed sentences entailed by the student-composed tree. It provides positive feedback if the student-composed tree belongs to the well-formed set, and negative feedback otherwise. If so requested by the student, it can substantiate the positive or negative feedback based on a comparison between the student-composed tree and its own trees (informative feedback on demand). In case of negative feedback, the system refuses to build the structure attempted by the student. Frequently occurring errors are handled in terms of "malrules." The system we describe is a prototype (implemented in JAVA and C++) which can be parameterized with respect to L1 and L2, the size of the lexicon, and the level of detail of the visually presented grammatical structures.

Abstract

Faced with planning problems related to lexical access, speakers take advantage of a major function of disfluencies: buying time. It is reasonable, then, to expect that the structure of disfluencies sheds light on the mechanisms underlying lexical access. Using data from the Switchboard Corpus, we investigated the effect of semantic competition during lexical access on repetition disfluencies. We hypothesized that the more time the speaker needs to access the following unit, the longer the repetition. We examined the repetitions preceding verbs and nouns and tested predictors influencing the accessibility of these items. Results suggest that speed of lexical access negatively correlates with the length of repetition and that the main determinants of lexical access speed differ for verbs and nouns. Longer disfluencies before verbs appear to be due to significant paradigmatic competition from semantically similar verbs. For nouns, they occur when the noun is relatively unpredictable given the preceding context.

Abstract

There is evidence that the process of executing a planned utterance involves the use of both preceding-context and top-down cues. Utterance-initial words are cued only by the top-down plan. In contrast, non-initial words are cued both by top-down cues and preceding-context cues. Co-existence of both cue types raises the question of how they interact during learning. We argue that this interaction is competitive: items that tend to be preceded by predictive preceding-context cues are harder to activate from the plan without this predictive context. A novel computational model of this competition is developed. The model is tested on a corpus of repetition disfluencies and shown to account for the influences on patterns of restarts during production. In particular, this model predicts a novel Initiation Effect: following an interruption, speakers re-initiate production from words that tend to occur in utterance-initial position, even when they are not initial in the interrupted utterance.

Abstract

The representations generated by many mod-
els of language (word embeddings, recurrent
neural networks and transformers) correlate
to brain activity recorded while people read.
However, these decoding results are usually
based on the brain’s reaction to syntactically
and semantically sound language stimuli. In
this study, we asked: how does an LSTM (long
short term memory) language model, trained
(by and large) on semantically and syntac-
tically intact language, represent a language
sample with degraded semantic or syntactic
information? Does the LSTM representation
still resemble the brain’s reaction? We found
that, even for some kinds of nonsensical lan-
guage, there is a statistically significant rela-
tionship between the brain’s activity and the
representations of an LSTM. This indicates
that, at least in some instances, LSTMs and the
human brain handle nonsensical data similarly.

Abstract

Cross-culturally, the human mind varies more than we generally assume

Abstract

Contemporary comparative cognition has a large repertoire of animal models and methods, with concurrent theoretical advances that are providing initial answers to crucial questions about human cognition. What cognitive traits are uniquely human? What are the species-typical inherited predispositions of the human mind? What is the human mind capable of without certain types of specific experiences with the surrounding environment? Here, we review recent findings from the domains of space, time and number cognition. These findings are produced using different comparative methodologies relying on different animal species, namely birds and non-human great apes. The study of these species not only reveals the range of cognitive abilities across vertebrates, but also increases our understanding of human cognition in crucial ways.

Abstract

This chapter contains sections titled: Introduction Questions Questioning and the Epistemic Gradient Presuppositions, Agenda Setting and Preferences Social Actions Implemented by Questions Questions as Building Blocks of Institutional Activities Future Directions

Abstract

Three eye-tracking experiments investigated the impact of the complexity of the visual environment on the likelihood of word-object mapping taking place at phonological, semantic and visual levels of representation during language-mediated visual search. Dutch participants heard spoken target words while looking at four objects embedded in displays of different complexity and indicated the presence or absence of the target object. During filler trials the target objects were present, but during experimental trials they were absent and the display contained various competitor objects. For example, given the target word “beaker”, the display contained a phonological (a beaver, bever), a shape (a bobbin, klos), a semantic (a fork, vork) competitor, and an unrelated distractor (an umbrella, paraplu). When objects were presented in simple four-object displays (Experiment 2), there were clear attentional biases to all three types of competitors replicating earlier research (Huettig and McQueen, 2007). When the objects were embedded in complex scenes including four human-like characters or four meaningless visual shapes (Experiments 1, 3), there were biases in looks to visual and semantic but not to phonological competitors. In both experiments, however, we observed evidence for inhibition in looks to phonological competitors, which suggests that the phonological forms of the objects nevertheless had been retrieved. These findings suggest that phonological word-object mapping is contingent upon the nature of the visual environment and add to a growing body of evidence that the nature of our visual surroundings induces particular modes of processing during language-mediated visual search.

Abstract

This study uses a classical referential communication task to
investigate the role of co-speech gestures in the process of
coordination. The study manipulates both the common ground between the interlocutors, as well as the visibility of the gestures they use. The findings show that co-speech gestures are an integral part of the referential utterances speakers
produced with regard to both initial references as well as repeated references, and that the availability of gestures appears to impact on interlocutors’ referential oordination. The results are discussed with regard to past research on
common ground as well as theories of gesture production.

Abstract

In human face-to-face communication, language comprehension is a multi-modal, situated activity. However, little is known about how we combine information from these different modalities, and how perceived communicative intentions, often signaled through visual signals, such as eye
gaze, may influence this processing. We address this question by simulating a triadic communication context in which a
speaker alternated her gaze between two different recipients. Participants thus viewed speech-only or speech+gesture
object-related utterances when being addressed (direct gaze) or unaddressed (averted gaze). Two object images followed
each message and participants’ task was to choose the object that matched the message. Unaddressed recipients responded significantly slower than addressees for speech-only
utterances. However, perceiving the same speech accompanied by gestures sped them up to a level identical to
that of addressees. That is, when speech processing suffers due to not being addressed, gesture processing remains intact and enhances the comprehension of a speaker’s message

Abstract

Recent experimental evidence from cognitive psychology and cognitive neuroscience shows that reading acquisition has non-trivial consequences for cognitive processes other than reading per se. In the present chapter I present evidence from three areas of cognition: phonological processing, prediction in language processing, and visual search. These findings suggest that literacy on cognition influences are far-reaching. This implies that a good understanding of the dramatic impact of literacy acquisition on the human mind is an important prerequisite for successful education policy development and guidance of educational support.

Abstract

The current chapter discusses a structural priming experiment that investigated the on-line processing of English subject- and object- relative clauses. Sixty-one monolingual English-speaking adults participated in a self-paced reading experiment where they read prime-target pairs that fully crossed the relativised element within the relative clause (subject- versus object) across prime and target sentences. Following probabilistic theories of sentence processing, which predict that low frequency structures like object relatives are subject to greater priming effects due to their marked status, it was hypothesised that the normally-observed subject RC processing advantage would be eliminated following priming. The hypothesis was supported, identifying an important role for structural frequency in the processing of relative clause structures.

Abstract

Models of cognitive processes often include simplifications, idealisations, and fictionalisations, so how should we learn about cognitive processes from such models? Particularly in cognitive science, when many features of the target system are unknown, it is not always clear which simplifications, idealisations, and so on, are appropriate for a research question, and which are highly misleading. Here we use a case-study from studies of language evolution, and ideas from philosophy of science, to illustrate a robustness approach to learning from models. Robust properties are those that arise across a range of models, simulations and experiments, and can be used to identify key causal structures in the models, and the phenomenon, under investigation. For example, in studies of language evolution, the emergence of compositional structure is a robust property across models, simulations and experiments of cultural transmission, but only under pressures for learnability and expressivity. This arguably illustrates the principles underlying real cases of language evolution. We provide an outline of the robustness approach, including its limitations, and suggest that this methodology can be productively used throughout cognitive science. Perhaps of most importance, it suggests that different modelling frameworks should be used as tools to identify the abstract properties of a system, rather than being definitive expressions of theories.

Abstract

The human hard palate shows between-subject variation
that is known to influence articulatory strategies.
In order to link such variation to human speech, we
are conducting a cross-sectional MRI study on multiple
populations. A model based on Bezier curves
using only three parameters was fitted to hard palate
MRI tracings using evolutionary computation. The
fits produced consistently yield high accuracies. For
future research, this new method may be used to classify
our MRI data on ethnic origins using e.g., cluster
analyses. Furthermore, we may integrate our model
into three-dimensional representations of the vocal
tract in order to investigate its effect on acoustics and
cultural transmission.