Publications

Abstract

The previous literature has suggested that the hand movement in co-speech gestures and signs consists of a series of phases with qualitatively different dynamic characteristics. In this paper, we propose a syntagmatic rule system for movement phases that applies to both co-speech gestures and signs. Descriptive criteria for the rule system were developed for the analysis video-recorded continuous production of signs and gesture. It involves segmenting a stream of body movement into phases and identifying different phase types. Two human coders used the criteria to analyze signs and cospeech gestures that are produced in natural discourse. It was found that the criteria yielded good inter-coder reliability. These criteria can be used for the technology of automatic recognition of signs and co-speech gestures in order to segment continuous production and identify the potentially meaningbearing phase.

Abstract

Gesture is an integral part of face-to-face communication, and provides a rich area for cross-cultural comparison. “Locally-anchored spatial gestures” are gestures that are roughly oriented to the actual geographical direction of referents. For example, such gestures may point to a location or a thing, trace the shape of a path, or indicate the direction of a particular area. The goal of this task is to elicit locally-anchored spatial gestures across different cultures. The task follows an interview format, where one participant prompts another to talk in detail about a specific area that the main speaker knows well. The data can be used for additional purposes such as the investigation of demonstratives.

Abstract

The VALID Data Archive is an open multimedia data archive (under construction) with data from speakers suffering from language impairments. We report on a pilot project in the CLARIN-NL framework in which five data resources were curated. For all data sets concerned, written informed consent from the participants or their caretakers has been obtained. All materials were anonymized. The audio files were converted into wav (linear PCM) files and the transcriptions into CHAT or ELAN format. Research data that consisted of test, SPSS and Excel files were documented and converted into CSV files. All data sets obtained appropriate CMDI metadata files. A new CMDI metadata profile for this type of data resources was established and care was taken that ISOcat metadata categories were used to optimize interoperability. After curation all data are deposited at the Max Planck Institute for Psycholinguistics Nijmegen where persistent identifiers are linked to all resources. The content of the transcriptions in CHAT and plain text format can be searched with the TROVA search engine

Abstract

This chapter explores the expression of placement (or Goal-oriented) and removal (or Source-oriented) events by speakers of Polish (a West Slavic language). Its aim is to investigate the hypothesis known as ‘Source/Goal asymmetry’ according to which languages tend to favor the expression of Goals (e.g., into, onto) and to encode them more systematically and in a more fine-grained way than Sources (e.g., from, out of). The study provides both evidence and counter-evidence for Source/Goal asymmetry. On the one hand, it shows that Polish speakers use a greater variety of verbs to convey Manner and/or mode of manipulation in the expression of placement, encoding such events in a more fine-grained manner than removal events. The expression of placement is also characterized by a greater variety of verb prefixes conveying Path and prepositional phrases (including prepositions and case markers) conveying Ground. On the other hand, the study reveals that Polish speakers attend to Sources as often as to Goals, revealing no evidence for an attentional bias toward the endpoints of events.

Abstract

In different fields of the humanities annotations of multimodal resources are a necessary component of the research workflow. Examples include linguistics, psychology, anthropology, etc. However, creation of those annotations is a very laborious task, which can take 50 to 100 times the length of the annotated media, or more. This can be significantly improved by applying innovative audio and video processing algorithms, which analyze the recordings and provide automated annotations. This is the aim of the AVATecH project, which is a collaboration of the Max Planck Institute for Psycholinguistics (MPI) and the Fraunhofer institutes HHI and IAIS. In this paper we present a set of results of automated annotation together with an evaluation of their quality.

Abstract

Evolution and changes of all modern languages is a wellknown fact. However, recently it is reaching dynamics never seen before, which results in loss of the vast amount of information encoded in every language. In order to preserve such rich heritage, and to carry out linguistic research, properly annotated recordings of world languages are necessary. Since creating those annotations is a very laborious task, reaching times 100 longer than the length of the annotated media, innovative video processing algorithms are needed, in order to improve the efficiency and quality of annotation process. This is the scope of the AVATecH project presented in this article

Abstract

Introduction Human spoken communication is multimodal, i.e. it encompasses both speech and gesture. Acoustic properties of voice, body movements, facial expression, etc. are an inherent and meaningful part of spoken interaction; they can provide attitudinal, grammatical and semantic information. In the recent years interest in audio-visual corpora has been rising rapidly as they enable investigation of different communicative modalities and provide more holistic view on communication (Kipp et al. 2009). Moreover, for some languages such corpora are the only available resource, as is the case for endangered languages for which no written resources exist.

Abstract

In this paper we describe audio and video processing algorithms that are developed in the scope of AVATecH project. The purpose of these algorithms is to shorten the time taken by manual annotation of audio and video recordings by extracting features from media files and creating semi-automated annotations. We show that the use of such supporting algorithms can shorten the annotation time to 30-50% of the time necessary to perform a fully manual annotation of the same kind.

Abstract

People's language is shaped by the input from the environment. The environment, however, offers a range of linguistic inputs that differ in their reliability. We test whether listeners accordingly weigh input from sources that differ in reliability differently. Using a perceptual learning paradigm, we show that listeners adjust their representations according to linguistic input provided by native but not by non-native speakers. This is despite the fact that listeners are able to learn the characteristics of the speech of both speakers. These results provide evidence for a disassociation between adaptation to the characteristic of specific speakers and adjustment of linguistic representations in general based on these learned characteristics. This study also has implications for theories of language change. In particular, it cast doubts on the hypothesis that a large proportion of non-native speakers in a community can bring about linguistic changes

Abstract

How do languages express ideas of movement, and how do they package different components of this domain, such as manner and path of motion? This task uses one large set of stimuli to gain knowledge of certain key aspects of motion verb meanings in the target language, and expands the investigation beyond simple verbs (e.g., go) to include the semantics of motion predications complete with adjuncts (e.g., go across something). Consultants are asked to view and briefly describe 96 animations of a few seconds each. The task is designed to get linguistic elicitations of motion predications under contrastive comparison with other animations in the same set. Unlike earlier tasks, the stimuli focus on inanimate moving items or “figures” (in this case, a ball).

Abstract

Demonstratives (e.g., words such as this and that in English) pivot on relationships between the item being talked about, and features of the speech act situation (e.g., where the speaker and addressee are standing or looking). However, they are only rarely investigated multi-modally, in natural language contexts. This task is designed to build a video corpus of cross-linguistically comparable discourse data for the study of “deixis in action”, while simultaneously supporting the investigation of joint attention as a factor in speaker selection of demonstratives. In the task, two or more speakers are asked to discuss and evaluate a group of similar items (e.g., examples of local handicrafts, tools, produce) that are placed within a relatively defined space (e.g., on a table). The task can additionally provide material for comparison of pointing gesture practices.

Abstract

This entry contains: 1. An invitation to think about to what extent the grammar of space and time share lexical and morphosyntactic resources − the suggestions here are only prompts, since it would take a long questionnaire to fully explore this; 2. A suggestion about how to collect gestural data that might show us to what extent the spatial and temporal domains, have a psychological continuity. This is really the goal − but you need to do the linguistic work first or in addition. The goal of this task is to explore the extent to which time is conceptualised on a spatial basis.

Abstract

The ECHO proposal aims to bring to life the cultural heritage of Europe, through internet technology that encourages collaboration across the Humanities disciplines which interpret it – at the same time making all this scholarship accessible to the citizens of Europe. An essential part of the cultural heritage of Europe is the diverse set of languages used on the continent, in their historical, literary and spoken forms. Amongst these are the ‘hidden languages’ used by minorities but of wide interest to the general public. We take the 18 Sign Languages of the EEC – the natural languages of the deaf - as an example. Little comparative information about these is available, despite their special scientific importance, the widespread public interest and the policy implications. We propose a research project on these languages based on placing fully annotated digitized moving images of each of these languages on the internet. This requires significant development of multi-media technology which would allow distributed annotation of a central corpus, together with the development of special search techniques. The technology would have widespread application to all cultural performances recorded as sound plus moving images. Such a project captures in microcosm the essence of the ECHO proposal: cultural heritage is nothing without the humanities research which contextualizes and gives it comparative assessment; by marrying information technology to humanities research, we can bring these materials to a wider public while simultaneously boosting Europe as a research area.

Abstract

As for 'Locally-anchored spatial gestures task, version 2', a major goal of this task is to elicit locally-anchored spatial gestures across different cultures. “Locally-anchored spatial gestures” are gestures that are roughly oriented to the actual geographical direction of referents. Rather than set up an interview situation, this task involves recording informal, animated narrative delivered to a native-speaker interlocutor. Locally-anchored gestures produced in such narrative are roughly comparable to those collected in the interview task. The data collected can also be used to investigate a wide range of other topics.

Abstract

This paper describes the linguistic treatment of placement events in the Rossel Island (Papua New Guinea) language Yélî Dnye. Yélî Dnye is unusual in treating PUT and TAKE events symmetrically with a remarkable consistency. In what follows, we first provide a brief background for the language, then describe the six core PUT/TAKE verbs that were drawn upon by Yélî Dnye speakers to describe the great majority of the PUT/TAKE stimuli clips, along with some of their grammatical properties. In Section 5 we describe alternative verbs usable in particular circumstances and give an indication of the basis for variability in responses across speakers. Section 6 presents some reasons why the Yélî verb pattern for expressing PUT and TAKE events is of broad interest.

Abstract

There has been a recent spate of work on recursion as a central design feature of language and specifically of syntax. This short report points out that there is little evidence that unlimited recursion, understood as centre embedding, is typical of natural language syntax. Nevertheless, embedded pragmatic construals seem available in every language. Further, much deeper centre embedding can be found in dialogue or conversation structure than can be found in syntax. Existing accounts for the ‘performance’ limitations on centre embedding are thus thrown in doubt. Dialogue materials suggest that centre embedding is perhaps a core part of the human interaction system and is for some reason much more highly restricted in syntax than in other aspects of cognition

Abstract

This study investigates prosodic marking of focus in Bai, a Sino-Tibetan language spoken in the Southwest of China, by adopting a semi-spontaneous experimental approach. Our data show that Bai speakers increase the duration of the focused constituent and reduce the duration of the post-focus constituent to encode focus. However, duration is not used in Bai to distinguish focus types differing in size and contrastivity. Further, pitch plays no role in signaling focus and differentiating focus types. The results thus suggest that Bai uses prosody to mark focus, but to a lesser extent, compared to Mandarin Chinese, with which Bai has been in close contact for centuries, and Cantonese, to which Bai is similar in the tonal system, although Bai is similar to Cantonese in its reliance on duration in prosodic focus marking.

Abstract

Scholars disagree about the extent to which language can tell us
about conceptualisation of the world. Some believe that language
is a direct window onto concepts: Having a word ‘‘bird’’, ‘‘table’’ or
‘‘sour’’ presupposes the corresponding underlying concept, BIRD,
TABLE, SOUR. Others disagree. Words are thought to be uninformative,
or worse, misleading about our underlying conceptual representations;
after all, our mental worlds are full of ideas that we
struggle to express in language. How could this be so, argue sceptics,
if language were a direct window on our inner life? In this presentation,
I consider what language can tell us about the
conceptualisation of taste. By considering linguistic data from
twenty unrelated cultures – varying in subsistence mode (huntergatherer
to industrial), ecological zone (rainforest jungle to desert),
dwelling type (rural and urban), and so forth – I argue any single language is, indeed, impoverished about what it can reveal about
taste. But recurrent lexicalisation patterns across languages can
provide valuable insights about human taste experience. Moreover,
language patterning is part of the data that a good theory of taste
perception has to be answerable for. Taste researchers, therefore,
cannot ignore the crosslinguistic facts.

Abstract

This Research Topic explores the question: what is the relationship between representations of time and space in cultures around the world? This question touches on the broader issue of how humans come to represent and reason about abstract entities – things we cannot see or touch. Time is a particularly opportune domain to investigate this topic. Across cultures, people use spatial representations for time, for example in graphs, time-lines, clocks, sundials, hourglasses, and calendars. In language, time is also heavily related to space, with spatial terms often used to describe the order and duration of events. In English, for example, we might move a meeting forward, push a deadline back, attend a long concert or go on a short break. People also make consistent spatial gestures when talking about time, and appear to spontaneously invoke spatial representations when processing temporal language. A large body of evidence suggests a close correspondence between temporal and spatial language and thought. However, the ways that people spatialize time can differ dramatically across languages and cultures. This research topic identifies and explores some of the sources of this variation, including patterns in spatial thinking, patterns in metaphor, gesture and other cultural systems. This Research Topic explores how speakers of different languages talk about time and space and how they think about these domains, outside of language. The Research Topic invites papers exploring the following issues: 1. Do the linguistic representations of space and time share the same lexical and morphosyntactic resources? 2. To what extent does the conceptualization of time follow the conceptualization of space?

Abstract

La ciencia moderna nace de la conjunción entre postulados teóricos y el desarrollo de una infraestructura tecnológica que permite observar los hechos de manera adecuada, realizar experimentos y verificar las hipótesis. Desde Galileo, ciencia y tecnología han avanzado conjuntamente. En el mundo occidental, la ciencia ha evolucionado desde pro-puestas puramente especulativas (basadas en postulados apriorísticos) hasta el uso de métodos experimentales y estadísticos para explicar mejor nuestras observaciones. La tecnología se hermana con la ciencia facilitando al investigador una aproximación adecuada a los hechos que pretende explicar. Así, Galileo, para observar los cuerpos celestes, mejoró el utillaje óptico, lo que le permitió un acercamiento más preciso al objeto de estudio y, en consecuencia, unos fundamentos más sólidos para su propuesta teórica. De modo similar, actualmente el desarrollo tecnológico digital ha posibilitado la extracción masiva de datos y el análisis estadístico de éstos para verificar las hipótesis de partida: la lingüística no ha podido dar el paso desde la pura especulación hacia el análisis estadístico de los hechos hasta la aparición de las tecnologías digitales.

Abstract

Models are a flourishing and indispensable area of research in language evolution. Here we
highlight critical issues in using and interpreting models, and suggest viable approaches. First,
contrasting models can explain the same data and similar modelling techniques can lead to
diverging conclusions. This should act as a reminder to use the extreme malleability of
modelling parsimoniously when interpreting results. Second, quantitative techniques similar to
those used in modelling language evolution have proven themselves inadequate in other
disciplines. Cross-disciplinary fertilization is crucial to avoid mistakes which have previously
occurred in other areas. Finally, experimental validation is necessary both to sharpen models'
hypotheses, and to support their conclusions. Our belief is that models should be interpreted as
quantitative demonstrations of logical possibilities, rather than as direct sources of evidence.
Only an integration of theoretical principles, quantitative proofs and empirical validation can
allow research in the evolution of language to progress.

Abstract

Two Siberian languages, Tundra Nenets and Tundra Yukaghir, do not obey strong island constraints in questioning: any sub-constituent of a relative or adverbial clause can be questioned. We argue that this has to do with how focusing works in these languages. The focused sub-constituent remains in situ, but there is abundant morphosyntactic evidence that the focus feature is passed up to the head of the clause. The result is the formation of a complex focus structure in which both the head and non head daughter are overtly marked as focus, and they are interpreted as a pairwise list such that the focus background is applicable to this list, but not to other alternative lists

Abstract

This volume is dedicated to exploring the crossroads where complex sentences and information management – more specifically information structure (IS) and reference tracking (RT) – come together. Complex sentences are a highly relevant but understudied domain for studying notions of IS and RT. On the one hand, a complex sentence can be studied as a mini-unit of discourse consisting of two or more elements describing events, situations, or processes, with its own internal information-structural and referential organisation. On the other hand, complex sentences can be studied as parts of larger discourse structures, such as narratives or conversations, in terms of how their information-structural characteristics relate to this wider context.We first focus on the interrelatedness of IS and RT (Section 1) and then define and discuss the notion of complex sentences and their subtypes in Section 2. Section 3 surveys issues of IS in complex sentences, while Section 4 focuses on RT in complex sentences. Sections 5 and 6 briefly consider IS and RT in a wider discourse context. Section 5 discusses the interaction between IS, RT, and other discourse factors, and Section 6 focuses on ways in which a specific RT system, switch reference, can function as an RT device beyond the sentence.

Abstract

No island effects are observable in Tundra Yukaghir questions, which are possible in virtually all syntactic environments. It is argued that this feature of Tundra Yukaghir relates to its capability of explicitly marking focus domains. If a question word occurs in a syntactic island, the whole island is morphologically treated as a focus domain. In order to take scope and function as question markers, question words must remain within the focus domain, i.e. in the island clause. This syntactic configuration is reflected in the semantics of question islands, which are used to inquire about the identity of the whole island, not merely the denotation of the question word.

Abstract

The results of a study on perceptual learning are reported. Dutch subjects made lexical decisions on a list of words and nonwords. Embedded in the list were either [f]- or [s]-final words in which the final fricative had been replaced by an ambiguous sound, midway between [f] and [s]. One group of listeners heard ambiguous [f]- final Dutch words like [kara?] (based on karaf, carafe) and unambiguous [s]-final words (e.g., karkas, carcase). A second group heard the reverse (e.g., ambiguous [karka?] and unambiguous karaf). After this training phase, listeners labelled ambiguous fricatives on an [f]- [s] continuum. The subjects who had heard [?] in [f]- final words categorised these fricatives as [f] reliably more often than those who had heard [?] in [s]-final words. These results suggest that speech recognition is dynamic: the system adjusts to the constraints of each particular listening situation. The lexicon can provide this adjustment process with a training signal.

Abstract

The results of a word-spotting experiment are presented in which Dutch listeners tried to spot different types of bisyllabic Dutch words embedded in different types of nonsense contexts. Embedded verbs were not reliably harder to spot than embedded nouns; this suggests that nouns and verbs are recognised via the same basic processes. Iambic words were no harder to spot than trochaic words, suggesting that trochaic words are not in principle easier to recognise than iambic words. Words were harder to spot in consonantal contexts (i.e., contexts which themselves could not be words) than in longer contexts which contained at least one vowel (i.e., contexts which, though not words, were possible words of Dutch). A control experiment showed that this difference was not due to acoustic differences between the words in each context. The results support the claim that spoken-word recognition is sensitive to the viability of sound sequences as possible words.

Abstract

In this paper, we present two attempts to replicate a widely-cited but never fully published experiment in which German and Spanish speakers were asked to associate adjectives with nouns of masculine and feminine grammati-cal gender (Boroditsky et al. 2003). The researchers claim that speakers associ-ated more stereotypically female adjectives with grammatically feminine nouns and more stereotypically male adjectives with grammatically masculine nouns. We were not able to replicate the results either in a word association task or in an analogous primed lexical decision task. This suggests that the results of the original experiment were either an artifact of some non-documented aspect of the experimental procedure or a statistical fluke. The question whether speakers assign sex-based interpretations to grammatical gender categories at all cannot be answered definitively, as the results in the published literature vary consider-ably. However, our experiments show that if such an effect exists, it is not strong enough to be measured indirectly via the priming of adjectives by nouns.

Abstract

Our knowledge of speech perception is largely based on experiments conducted with carefully recorded clear speech presented under good listening conditions to undistracted listeners - a near-ideal situation, in other words. But the reality poses a set of different challenges. First of all, listeners may need to divide their attention between speech comprehension and another task (e.g., driving). Outside the laboratory, the speech signal is often slurred by less than careful pronunciation and the listener has to deal with background noise. Moreover, in a globalized world, listeners need to understand speech in more than their native language. Relatedly, the speakers we listen to often have a different language background so we have to deal with a foreign or regional accent we are not familiar with. Finally, outside the laboratory, speech perception is not an end in itself, but rather a mean to contribute to a conversation. Listeners do not only need to understand the speech they are hearing, they also need to use this information to plan and time their own responses. For this special topic, we invite papers that address any of these ecological aspects of speech perception.

Abstract

This paper describes the pilot study of phonetic segmentation applied to Nijmegen Corpus of Casual Czech (NCCCz). This corpus contains informal speech of strong spontaneous nature which influences the character of produced speech at various levels. This work is the part of wider research related to the analysis of pronunciation reduction in such informal speech. We present the analysis of the accuracy of phonetic segmentation when canonical or reduced pronunciation is used. The achieved accuracy of realized phonetic segmentation provides information about general accuracy of proper acoustic modelling which is supposed to be applied in spontaneous speech recognition. As a byproduct of presented spontaneous speech segmentation, this paper also describes the created lexicon with canonical pronunciations of words in NCCCz, a tool supporting pronunciation check of lexicon items, and finally also a minidatabase of selected utterances from NCCCz manually labelled on phonetic level suitable for evaluation purposes

Abstract

The central issues in the study of speech recognition by human listeners (HSR) and of automatic speech recognition (ASR) are clearly comparable; nevertheless the research communities that concern themselves with ASR and HSR are largely distinct. This paper compares the research objectives of the two fields, and attempts to draw informative lessons from one to the other.