Publications

Abstract

This paper presents research findings on manual gestures
associated with negation in Indonesian, utilizing data sourced
from talk shows available on YouTube. The study reveals that
Indonesian speakers employ six recurrent negation gestures,
which have been observed in various languages worldwide.
This suggests that gestures exhibiting a stable form-meaning
relationship and recurring frequently in relation to negation are
prevalent around the globe, although their distribution may
differ across cultures and languages. Furthermore, the paper
demonstrates that negation gestures are not strictly tied to
verbal negation. Overall, the aim of this paper is to contribute
to a deeper understanding of the conventional usage and cross-
linguistic distribution of recurrent gestures.

Abstract

Recent empirical evidence suggests that language-mediated eye gaze is partly determined by level of formal literacy training. Huettig, Singh and Mishra (2011) showed that high-literate individuals' eye gaze was closely time locked to phonological overlap between a spoken target word and items presented in a visual display. In contrast, low-literate individuals' eye gaze was not related to phonological overlap, but was instead strongly influenced by semantic relationships between items. Our present study tests the hypothesis that this behavior is an emergent property of an increased ability to extract phonological structure from the speech signal, as in the case of high-literates, with low-literates more reliant on more coarse grained structure. This hypothesis was tested using a neural network model, that integrates linguistic information extracted from the speech signal with visual and semantic information within a central resource. We demonstrate that contrasts in fixation behavior similar to those observed between high and low literates emerge when models are trained on speech signals of contrasting granularity.

Abstract

This study focuses on embodied uses of recognitional
demonstratives. While multimodal conversation analytic
studies have shown how gesture and speech interact in the
elaboration of exophoric references, little attention has been
given to the multimodal configuration of other types of
referential actions. Based on a video-recorded corpus of
professional meetings held in French, this qualitative study
shows that a subtype of deictic references, namely recognitional
references, are frequently associated with iconic gestures, thus
challenging the traditional distinction between exophoric and
endophoric uses of deixis.

Abstract

Presenting a comprehensive, state-of-the-art overview of theoretical and descriptive research in the field, The Handbook of Conversation Analysis brings together contributions by leading international experts to provide an invaluable information resource and reference for scholars of social interaction across the areas of conversation analysis, discourse analysis, linguistic anthropology, interpersonal communication, discursive psychology and sociolinguistics. Ideal as an introduction to the field for upper level undergraduates and as an in-depth review of the latest developments for graduate level students and established scholars Five sections outline the history and theory, methods, fundamental concepts, and core contexts in the study of conversation, as well as topics central to conversation analysis Written by international conversation analysis experts, the book covers a wide range of topics and disciplines, from reviewing underlying structures of conversation, to describing conversation analysis' relationship to anthropology, communication, linguistics, psychology, and sociology

Abstract

This paper describes a computational model of speech comprehension that takes the acoustic signal as input and predicts reaction times as observed in an auditory lexical decision task. By doing so, we explore a new generation of end-to-end computational models that are able to simulate the behaviour of human subjects participating in a psycholinguistic experiment. So far, nearly all computational models of speech comprehension do not start from the speech signal itself, but from abstract representations of the speech signal, while the few existing models that do start from the acoustic signal cannot directly model reaction times as obtained in comprehension experiments. The main functional components in our model are the perception stage, which is compatible with the psycholinguistic model Shortlist B and is implemented with techniques from automatic speech recognition, and the decision stage, which is based on the linear ballistic accumulation decision model. We successfully tested our model against data from 20 participants performing a largescale auditory lexical decision experiment. Analyses show that the model is a good predictor for the average judgment and reaction time for each word.

Abstract

We are often in a bilingual situation (e.g., overhearing a conversation in the train). We investigated whether first (L1) and second language (L2) phonologies are automatically activated. A masked priming paradigm was used, with Russian words as targets and either Russian or English words as primes. Event-related potentials (ERPs) were recorded while Russian (L1) – English (L2) bilinguals read aloud L1 target words (e.g. РЕЙС /reis/ ‘fl ight’) primed with either L1 (e.g. РАНА /rana/ ‘wound’) or L2 words (e.g. PACK). Target words were read faster when they were preceded by phonologically related L1 primes but not by orthographically related L2 primes. ERPs showed orthographic priming in the 125-200 ms time window. Thus, both L1 and L2 phonologies are simultaneously activated during L1 reading. The results provide support for non-selective models of bilingual reading, which assume automatic activation of the non-target language phonology even when it is not required by the task.

Abstract

This paper presents a new interdisciplinary methodology for the
analysis of future conceptualisations in big messy media data.
More specifically, it focuses on the depictions of post-Covid
futures by RT during the pandemic, i.e. on data which are of
interest not just from the perspective of academic research but
also of policy engagement. The methodology has been
developed to support the scaling up of fine-grained data-driven
analysis of discourse utterances larger than individual lexical
units which are centred around ‘will’ + the infinitive. It relies
on the true integration of manual analytical and computational
methods and tools in researching three modalities – textual,
prosodic1, and gestural. The paper describes the process of
building a computational infrastructure for the collection and
processing of video data, which aims to empower the manual
analysis. It also shows how manual analysis can motivate the
development of computational tools. The paper presents
individual computational tools to demonstrate how the
combination of human and machine approaches to analysis can
reveal new manifestations of cohesion between gesture and
prosody. To illustrate the latter, the paper shows how the
boundaries of prosodic units can work to help determine the
boundaries of gestural units for future conceptualisations.

Abstract

Over the course of a conversation, interlocutors sound more and more like each other in a process called convergence. However, the automaticity and grain size of convergence are not well established. This study therefore examined whether female native Dutch speakers converge to large yet sub-phonemic shifts in the F2 of the vowel /e/. Participants first performed a short reading task to establish baseline F2s for the vowel /e/, then shadowed 120 target words (alongside 360 fillers) which contained one instance of a manipulated vowel /e/ where the F2 had been shifted down to that of the vowel /ø/. Consistent exposure to large (sub-phonemic) downward shifts in F2 did not result in convergence. The results raise issues for theories which view convergence as a product of automatic integration between perception and production.

Abstract

Linguistic context vectors are adapted for measuring the linguistic contexts that accompany gestures and comparable co-linguistic behaviours. Focusing on gestural semiotic types, it is demonstrated that gestural linguistic context vectors carry information associated with gesture. It is suggested that these may be used to approximate gesture meaning in a similar manner to the approximation of word meaning by context vectors.

Abstract

It has been suggested that individual variation in vowel compactness of the native language (L1) and the distance between L1 vowels and vowels in the second language (L2) predict successful L2 vowel acquisition. Moreover, general articulatory skills have been proposed to account for variation in vowel compactness. In the present work, we conceptually replicate a previous study to test these hypotheses with a large sample size, a new language pair and a
new vowel pair. We find evidence that individual variation in L1 vowel compactness has opposing effects for two different vowels. We do not find evidence that individual variation in L1 compactness
is explained by general articulatory skills. We conclude that the results found previously might be specific to sub-groups of L2 learners and/or specific sub-sets of vowel pairs.