Publications

Abstract

This chapter sketches in very general terms the cognitive architecture of both language comprehension and production, as well as the neurobiological infrastructure that makes the human brain ready for language. Focus is on spoken language, since that compares most directly to processing music. It is worth bearing in mind that humans can also interface with language as a cognitive system using sign and text (visual) as well as Braille (tactile); that is to say, the system can connect with input/output processes in any sensory modality. Language processing consists of a complex and nested set of subroutines to get from sound to meaning (in comprehension) or meaning to sound (in production), with remarkable speed and accuracy. The fi rst section outlines a selection of the major constituent operations, from fractionating the input into manageable units to combining and unifying information in the construction of meaning. The next section addresses the neurobiological infrastructure hypothesized to form the basis for language processing. Principal insights are summarized by building on the notion of “brain networks” for speech–sound processing, syntactic processing, and the construction of meaning, bearing in mind that such a neat three-way subdivision overlooks important overlap and shared mechanisms in the neural architecture subserving language processing. Finally, in keeping with the spirit of the volume, some possible relations are highlighted between language and music that arise from the infrastructure developed here. Our characterization of language and its neurobiological foundations is necessarily selective and brief. Our aim is to identify for the reader critical questions that require an answer to have a plausible cognitive neuroscience of language processing.

Abstract

This thesis focuses on one aspect of interactional competence: competence to manage knowledge distribution in conversation. In order to be considered competent in everyday interaction, participants need not only to index one another's knowledge states but also to engage in dynamic negotiation of knowledge distribution. Adopting the methodology of conversation analysis, the thesis investigates how participants' orientations to knowledge distribution, 'epistemicity', are manifested. The thesis examines three interactional environments: assessment sequences, informing sequences and polar question-answer sequences. A systematic analysis reveals that interactants orient to different aspects of knowledge in different environments, employing different grammatical resources. When they assess an object, they are concerned about who possesses 'epistemic primacy'. Japanese final particles and the practices of intensification serve together to claim epistemic primacy and provide support for the claim. It is also reported that interactants are oriented to achieve 'epistemic congruence' − consensus regarding how knowledge is distributed among them. When one provides the other with new information, the exchange commonly develops into a four-turn sequence, instead of a minimal adjacency pair. It is shown that this sequence organization allows interactants to achieve a balance between territories of experience, affiliation and empathy. In polar question-answer sequences, how (un)expected or novel a given piece of information is becomes an issue. Answers are found to be formulated such that they adopt epistemic stances that are assertive enough to match the level of (un)certainty expressed by questioners. The thesis contributes to our understanding of how social interaction is organized. It becomes clear from the findings that a wide range of aspects of language use and interactional organization are dominated by interactants' orientations to epistemicity. Participants manage knowledge distribution in everyday interaction, which may be the most fundamental means of managing their social statuses and relations.

Abstract

This chapter contains sections titled: Introduction Questions Questioning and the Epistemic Gradient Presuppositions, Agenda Setting and Preferences Social Actions Implemented by Questions Questions as Building Blocks of Institutional Activities Future Directions

Abstract

What kinds of resources to languages have for describing location and position? For some languages, verbs have an important role to play in describing different kinds of situations (e.g., whether a bottle is standing or lying on the table). This task is designed to examine the use of positional verbs in locative constructions, with respect to the presence or absence of a human “positioner”. Participants are asked to describe video clips showing locative states that occur spontaneously, or because of active interference from a person. The task follows on from two earlier tools for the elicitation of static locative descriptions (BowPed and the Ameka picture book task). A number of additional variables (e.g. canonical v. non-canonical orientation of the figure) are also targeted in the stimuli set.

Abstract

In face-to-face communication, information from the face as well as from the voice contributes to the identification of spoken words. This dissertation investigates the time-course of the evaluation and integration of visual and auditory speech in audiovisual word identification. A large-scale audiovisual gating study extends previous research on this topic by (1) using a set of words that includes all possible initial consonants in English in three vowel contexts, (2) tracking the information processing for individual words not only across modalities, but also over time, and (3) testing quantitative models of the time-course of multimodal word recognition. There was an advantage in accuracy for audiovisual speech over auditory-only and visual-only speech. Auditory performance was, however, close to ceiling while performance on visual-only trials was near the floor of the scale, but well above chance. Visual information was used at all gates to identify the presented words. Information theoretic feature analyses of the confusion matrices revealed that the auditory signal is highly informative about voicing, manner, frication, duration, and place of articulation. Visual speech is mostly informative about place of articulation, but also about frication and duration. The auditory signal provides more information about the place of articulation for back consonants, whereas the visual signal provides more information for the labial consonants. The data were sufficient to discriminate between models of audiovisual word recognition. The Fuzzy Logical Model of Perception (FLMP; Massaro, 1998) gave a better account of the confusion matrix data than additive models of perception. A dynamic version of the FLMP was expanded to account for the evaluation and integration of information over time. This dynamic FLMP provided a better description of the data than dynamic additive competitor models. The present study builds a good foundation to investigate the role of the complex interplay between stimulus information and the structure of the lexicon. It provides an important step in building a formal representation of a lexical dynamic FLMP that can account not only for the time-course of speech information and its perceptual processing, but also for lexical influences.

Abstract

Words can occur as constituents of other words. Some words have a high morphological productivity, in that they occur in many complex words, whereas others are morphological islands. Previous studies have found that the size of a word's morphological family can co-determine response latencies in lexical decision tasks. This thesis shows, using lexical decision as well as otherexperimental tasks, that the effect of family size is a semantic effect,reflecting the spreading of activation in the mental lexicon along the lines of morphological and semantic relatedness between words.

Abstract

Anthropology began by comparing aspects of kinship across cultures, while linguists interested in semantic domains such as kinship necessarily compare across languages. In this chapter I show how phylogenetic comparative methods from evolutionary biology can be used to study evolutionary processes relating to kinship and kinship terminologies across language and culture.

Abstract

In this paper we present a definition of Performance Grammar (PG), a psycholinguistically motivated syntax formalism, in declarative terms. PG aims not only at describing and explaining intuitive judgments and other data concerning the well–formedness of sentences of a language, but also at contributing to accounts of syntactic processing phenomena observable in language comprehension and language production. We highlight two general properties of human sentence generation, incrementality and late linearization,which make special demands on the design of grammar formalisms claiming psychological plausibility. In order to meet these demands, PG generates syntactic structures in a two-stage process. In the first and most important ‘hierarchical’ stage, unordered hierarchical structures (‘mobiles’) are assembled out of lexical building blocks. The key operation at work here is typed feature unification, which also delimits the positional options of the syntactic constituents in terms of so-called topological features. The second, much simpler stage takes care of arranging the branches of the mobile from left to right by ‘reading–out’ one positional option of every constituent. In this paper we concentrate on the structure assembly formalism in PG’s hierarchical component. We provide a declarative definition couched in an HPSG–style notation based on typed feature unification. Our emphasis throughout is on linear order constraints.

Abstract

Gesture is an integral part of face-to-face communication, and provides a rich area for cross-cultural comparison. “Locally-anchored spatial gestures” are gestures that are roughly oriented to the actual geographical direction of referents. For example, such gestures may point to a location or a thing, trace the shape of a path, or indicate the direction of a particular area. The goal of this task is to elicit locally-anchored spatial gestures across different cultures. The task follows an interview format, where one participant prompts another to talk in detail about a specific area that the main speaker knows well. The data can be used for additional purposes such as the investigation of demonstratives.

Abstract

It is generally assumed that verbs have an ‘argument structure’, which imposes various constraints on the noun phrases that can or must go with the verb, and an ‘event structure’, which characterises the particular temporal characteristics of the ‘event’ which the verb relates to: this event may be a state, a process, an activity, an ‘event in the narrow sense’, and others. In this paper, it is argued that that argument structure and event structure should be brought together. The lexical content of a verb assigns descriptive properties to one or more arguments at one or more times, hence verbs have an ‘argument time-structure’ (AT-structure). Numerous morphological and syntactical operations, such as participle formation or complex verb constructions, modify this AT-structure. This is illustrated with German recipient constructions such as ein Buch geschenkt bekommen or das Fenster geöffnet kriegen.

Abstract

It is usually assumed that the language spoken by a human community is independent of the community's genetic makeup, an assumption supported by an overwhelming amount of evidence. However, the possibility that language is influenced by its speakers' genes cannot be ruled out a priori, and a recently discovered correlation between the geographic distribution of tone languages and two human genes seems to point to a genetically influenced bias affecting language. This entry describes this specific correlation and highlights its major implications. Voice pitch has a variety of communicative functions. Some of these are probably universal, such as conveying information about the speaker's sex, age, and emotional state. In many languages, including the European languages, voice pitch also conveys certain sentence-level meanings such as signaling that an utterance is a question or an exclamation; these uses of pitch are known as intonation. Some languages, however, known as tone languages, nian ...

Abstract

How do languages express ideas of movement, and how do they package different components of this domain, such as manner and path of motion? This task uses one large set of stimuli to gain knowledge of certain key aspects of motion verb meanings in the target language, and expands the investigation beyond simple verbs (e.g., go) to include the semantics of motion predications complete with adjuncts (e.g., go across something). Consultants are asked to view and briefly describe 96 animations of a few seconds each. The task is designed to get linguistic elicitations of motion predications under contrastive comparison with other animations in the same set. Unlike earlier tasks, the stimuli focus on inanimate moving items or “figures” (in this case, a ball).

Abstract

Since the core matrix for language use is interaction, the main job of language
is not to express propositions or abstract meanings, but to deliver actions.
For in order to respond in interaction we have to ascribe to the prior turn
a primary ‘action’ – variously thought of as an ‘illocution’, ‘speech act’, ‘move’,
etc. – to which we then respond. The analysis of interaction also relies heavily
on attributing actions to turns, so that, e.g., sequences can be characterized in
terms of actions and responses. Yet the process of action ascription remains way
understudied. We don’t know much about how it is done, when it is done, nor even
what kind of inventory of possible actions might exist, or the degree to which they
are culturally variable.
The study of action ascription remains perhaps the primary unfulfilled task in
the study of language use, and it needs to be tackled from conversationanalytic,
psycholinguistic, cross-linguistic and anthropological perspectives.
In this talk I try to take stock of what we know, and derive a set of goals for and
constraints on an adequate theory. Such a theory is likely to employ, I will suggest,
a top-down plus bottom-up account of action perception, and a multi-level notion
of action which may resolve some of the puzzles that have repeatedly arisen.

Abstract

Given the diversity of languages, it is unlikely that the human capacity for language resides in rich universal syntactic machinery. More likely, it resides centrally in the capacity for vocal learning combined with a distinctive ethology for communicative interaction, which together (no doubt with other capacities) make diverse languages learnable. This chapter focuses on face-to-face communication, which is characterized by the mapping of sounds and multimodal signals onto speech acts and which can be deeply recursively embedded in interaction structure, suggesting an interactive origin for complex syntax. These actions are recognized through Gricean intention recognition, which is a kind of “ mirroring” or simulation distinct from the classic mirror neuron system. The multimodality of conversational interaction makes evident the involvement of body, hand, and mouth, where the burden on these can be shifted, as in the use of speech and gesture, or hands and face in sign languages. Such shifts having taken place during the course of human evolution. All this suggests a slightly different approach to the mystery of music, whose origins should also be sought in joint action, albeit with a shift from turn-taking to simultaneous expression, and with an affective quality that may tap ancient sources residual in primate vocalization. The deep connection of language to music can best be seen in the only universal form of music, namely song.

Abstract

Demonstratives (e.g., words such as this and that in English) pivot on relationships between the item being talked about, and features of the speech act situation (e.g., where the speaker and addressee are standing or looking). However, they are only rarely investigated multi-modally, in natural language contexts. This task is designed to build a video corpus of cross-linguistically comparable discourse data for the study of “deixis in action”, while simultaneously supporting the investigation of joint attention as a factor in speaker selection of demonstratives. In the task, two or more speakers are asked to discuss and evaluate a group of similar items (e.g., examples of local handicrafts, tools, produce) that are placed within a relatively defined space (e.g., on a table). The task can additionally provide material for comparison of pointing gesture practices.

Abstract

This entry contains: 1. An invitation to think about to what extent the grammar of space and time share lexical and morphosyntactic resources − the suggestions here are only prompts, since it would take a long questionnaire to fully explore this; 2. A suggestion about how to collect gestural data that might show us to what extent the spatial and temporal domains, have a psychological continuity. This is really the goal − but you need to do the linguistic work first or in addition. The goal of this task is to explore the extent to which time is conceptualised on a spatial basis.

Abstract

The ECHO proposal aims to bring to life the cultural heritage of Europe, through internet technology that encourages collaboration across the Humanities disciplines which interpret it – at the same time making all this scholarship accessible to the citizens of Europe. An essential part of the cultural heritage of Europe is the diverse set of languages used on the continent, in their historical, literary and spoken forms. Amongst these are the ‘hidden languages’ used by minorities but of wide interest to the general public. We take the 18 Sign Languages of the EEC – the natural languages of the deaf - as an example. Little comparative information about these is available, despite their special scientific importance, the widespread public interest and the policy implications. We propose a research project on these languages based on placing fully annotated digitized moving images of each of these languages on the internet. This requires significant development of multi-media technology which would allow distributed annotation of a central corpus, together with the development of special search techniques. The technology would have widespread application to all cultural performances recorded as sound plus moving images. Such a project captures in microcosm the essence of the ECHO proposal: cultural heritage is nothing without the humanities research which contextualizes and gives it comparative assessment; by marrying information technology to humanities research, we can bring these materials to a wider public while simultaneously boosting Europe as a research area.

Abstract

As for 'Locally-anchored spatial gestures task, version 2', a major goal of this task is to elicit locally-anchored spatial gestures across different cultures. “Locally-anchored spatial gestures” are gestures that are roughly oriented to the actual geographical direction of referents. Rather than set up an interview situation, this task involves recording informal, animated narrative delivered to a native-speaker interlocutor. Locally-anchored gestures produced in such narrative are roughly comparable to those collected in the interview task. The data collected can also be used to investigate a wide range of other topics.

Abstract

Words are fundamental to reading and yet over a century of research has not masked the controversies around how words are recognized. We review some old and new research that disproves simple ideas such as words are read as wholes or are simply mapped directly to spoken language. We also review theory and research relevant to the question of sublexical influences in word recognition. We describe orthography and phonology, how they are related to each other and describe a series of new experiments on how these sources of information are processed. Tasks include lexical decision, perceptual identification, and naming. Dependent measures are reaction time, accuracy of performance, and a new measure, initial phoneme duration, that refers to the duration of the first phoneme when the target word is pronounced. Important factors in resolving the controversies include the realization that reading has multiple determinants, as well as evaluating the type of task, proper controls such as familiarity of the test items and accuracy of measurement of the response. We also address potential limitations with measures related to the mapping between orthography and phonology, and show that the existence of a sound-to-spelling consistency effect does not require interactive activation, but can be explained and predicted by a feedforward model, the Fuzzy logical model of perception.