Publications

Abstract

Handedness has been studied for association with language-related disorders because of its link with language hemispheric dominance. No clear pattern has emerged, possibly because of small samples, publication bias, and heterogeneous criteria across studies. Non-right-handedness (NRH) frequency was assessed in N = 2503 cases with reading and/or language impairment and N = 4316 sex-matched controls identified from 10 distinct cohorts (age range 6–19 years old; European ethnicity) using a priori set criteria. A meta-analysis (Ncases = 1994) showed elevated NRH % in individuals with language/reading impairment compared with controls (OR = 1.21, CI = 1.06–1.39, p = .01). The association between reading/language impairments and NRH could result from shared pathways underlying brain lateralization, handedness, and cognitive functions.

Abstract

On seeing familiar persons, biographical (semantic) information is typically retrieved faster and more accurately than name information. Serial stage models explain this pattern by suggesting that access to the name follows the retrieval of semantic information. In contrast, interactive activation and competition (IAC) models hold that both processes start together but name retrieval is slower because of structural peculiarities. With a 2-choice go/no-go procedure based on a semantic and a name-related classification, the authors tested differential predictions of the 2 alternative models for reaction times (RTs) and lateralized readiness potentials (LRP). Both LRP (Experiment 1) and RT (Experiment 2) results are in line with IAC models of face identification and naming.

Abstract

In this work we perform multimodal detection and classification
of head movements from face to face video conversation data.
We have experimented with different models and feature sets
and provided some insight on the effect of independent features,
but also how their interaction can enhance a head movement
classifier. Used features include nose, neck and mid hip position
coordinates and their derivatives together with acoustic features,
namely, intensity and pitch of the speaker on focus. Results
show that when input features are sufficiently processed by in-
teracting with each other, a linear classifier can reach a similar
performance to a more complex non-linear neural model with
several hidden layers. Our best models achieve state-of-the-art
performance in the detection task, measured by macro-averaged
F1 score.

Abstract

While there are well-known demonstrations that children can use distributional information to acquire multiple components of language, the underpinnings of these achievements are unclear. In the current paper, we investigate the potential pre-requisites for a distributional learning model that can explain how children learn their first words. We review existing literature and then present the results of a series of computational simulations with Vector Space Models, a type of distributional semantic model used in Computational Linguistics, which we evaluate against vocabulary acquisition data from children. We focus on nouns and verbs, and we find that: (i) a model with flexibility to adjust for the frequency of events provides a better fit to the human data, (ii) the influence of context words is very local, especially for nouns, and (iii) words that share more contexts with other words are harder to learn.

Abstract

Children's gestures can reveal important information about their problem-solving strategies. This study investigated whether the information children express only in gesture is accessible to adults not trained in gesture coding. Twenty teachers and 20 undergraduates viewed videotaped vignettes of 12 children explaining their solutions to equations. Six children expressed the same strategy in speech and gesture, and 6 expressed different strategies. After each vignette, adults described the child's reasoning. For children who expressed different strategies in speech and gesture, both teachers and undergraduates frequently described strategies that children had not expressed in speech. These additional strategies could often be traced
to the children's gestures. Sensitivity to gesture was comparable for teachers and
undergraduates. Thus, even without training, adults glean information, not only from children's words but also from their hands.

Abstract

Different languages present a variety of ways of talking about emotional experience. Very commonly, feelings are described through the use of ‘body image constructions’ in which they are associated with processes in, or states of, specific body parts. The emotions and the body parts that are thought to be their locus and the kind of activity associated with these body parts vary cross-culturally. This study focuses on the meaning of three ‘body image constructions’ used to describe feelings similar to, but also different from, English ‘jealousy’, ‘envy’, and ‘covetousness’ in the West African language Ewe. It is demonstrated that a ‘moving body’, a pychologised eye, and red eyes are scripted for these feelings. It is argued that the expressions are not figurative and that their semantics provide good clues to understanding the cultural construction of emotions both emotions and the body.

Abstract

This paper presents the linguistic means of describing spatial relations in Ewe with particular emphasis on the grammar and meaning of adpositions. Ewe ( N iger-Congo ) has two sets of adpositions: prepositions, which have evolvedfrom verbs, and postpositions which have evolvedfrom nouns. The postpositions create places and are treated äs intrinsic parts or regions of the reference object in a spatial description. The prepositions provide the general orientation of a Figure (located object). It is demonstrated (hat spaiial relations, such äs those encapsulated in "the basic topological prepositions at, in and on" in English (Herskovits 1986: 9), are not encoded in single linguistic elements in Ewe, but are distributed over members of dijferent form classes in a syntagmatic string, The paper explores the r öle of compositionality andits interaction with pragmatics to yield understandings of spatial configurations in such a language where spatial meanings cannot he simply read off one form. The study also examines the diversity among languages in terms of the nature and obligatoriness of the coding of relational and ground Information in spatial constructions. It is argued that the ränge and type of distinctions discussed in the paper must be accountedfor in semantic typology and in the cross-linguistic investigation of spatial language and conceptualisation.

Abstract

Rhythmic patterns in interactive contexts characterize human behaviours such as conversational turn-taking. These timed patterns are also present in other animals, and often described as rhythm. Understanding fine-grained temporal adjustments in interaction requires complementary quantitative methodologies. Here, we showcase how vocal interactive rhythmicity in a non-human animal can be quantified using a multi-method approach. We record vocal interactions in harbour seal pups (Phoca vitulina) under controlled conditions. We analyse these data by combining analytical approaches, namely categorical rhythm analysis, circular statistics and time series analyses. We test whether pups' vocal rhythmicity varies across behavioural contexts depending on the absence or presence of a calling partner. Four research questions illustrate which analytical approaches are complementary versus orthogonal. For our data, circular statistics and categorical rhythms suggest that a calling partner affects a pup's call timing. Granger causality suggests that pups predictively adjust their call timing when interacting with a real partner. Lastly, the ADaptation and Anticipation Model estimates statistical parameters for a potential mechanism of temporal adaptation and anticipation. Our analytical complementary approach constitutes a proof of concept; it shows feasibility in applying typically unrelated techniques to seals to quantify vocal rhythmic interactivity across behavioural contexts.

Abstract

Recent artificial neural networks that process natural language achieve unprecedented performance in tasks requiring sentence-level understanding. As such, they could be interesting models of the integration of linguistic information in the human brain. We review works that compare these artificial language models with human brain activity and we assess the extent to which this approach has improved our understanding of the neural processes involved in natural language comprehension. Two main results emerge. First, the neural representation of word meaning aligns with the context-dependent, dense word vectors used by the artificial neural networks. Second, the processing hierarchy that emerges within artificial neural networks broadly matches the brain, but is surprisingly inconsistent across studies. We discuss current challenges in establishing artificial neural networks as process models of natural language comprehension. We suggest exploiting the highly structured representational geometry of artificial neural networks when mapping representations to brain data.

Abstract

There is a robust positive relationship between reading skills and the time to name aloud an array of letters, digits, objects, or colors as quickly as possible. A convincing and complete explanation for the direction and locus of this association remains, however, elusive. In this study we investigated rapid automatized naming (RAN) of every-day objects and basic color patches in neurotypical illiterate and literate adults. Literacy acquisition and education enhanced RAN performance for both conceptual categories but this advantage was much larger for (abstract) colors than every-day objects. This result suggests that (i) literacy/education may be causal for serial rapid naming ability of non-alphanumeric items, (ii) differences in the lexical quality of conceptual representations can underlie the reading-related differential RAN performance.

Abstract

With a long-term aim of empowering researchers everywhere to contribute to work on language development, we organized the First Truly Global /L+/ International Summer/ Winter School on Language Acquisition, a free 5-day virtual school for early career researchers. In this paper, we describe the school, our experience organizing it, and lessons learned. The school had a diverse organizer team, composed of 26 researchers (17 from under represented areas: Subsaharan Africa, South and Southeast Asia, and Central and South America); and a diverse volunteer team, with a total of 95 volunteers from 35 different countries, nearly half from under represented areas. This helped world-wide Page 5 of 5 promotion of the school, leading to 958 registrations from 88 different countries, with 300 registrants (based in 63 countries, 80% from under represented areas) selected to participate in the synchronous aspects of the event. The school employed asynchronous (pre-recorded lectures, which were close-captioned) and synchronous elements (e.g., discussions to place the recorded lectures into participants' context; networking events) across three time zones. A post-school questionnaire revealed that 99% of participants enjoyed taking part in the school. Not with standing these positive quantitative outcomes, qualitative comments suggested we fell short in several areas, including the geographic diversity among lecturers and greater customization of contents to the participants’ contexts. Although much remains to be done to promote inclusivity in linguistic research, we hope our school will contribute to empowering researchers to investigate and publish on language acquisition in their home languages, to eventually result in more representative theories and empirical generalizations

Abstract

Based on six detailed case studies of languages in which focus is marked morphosyntactically, we propose a novel formal theory of focus marking, which can capture these as well as the familiar English-type prosodic focus marking. Special attention is paid to the patterns of focus syncretism, that is, when different size and/or location of focus are indistinguishably realized by the same form.

The key ingredients to our approach are that complex constituents (not just words) may be directly focally marked, and that the choice of focal marking is governed by blocking.

Abstract

Are regular morphologically complex words stored in the mental lexicon? Answers to this question have ranged from full listing to parsing for every regular complex word. We investigated the roles of storage and parsing in the visual domain for the productive Dutch plural suffix -en.Two experiments are reported that show that storage occurs for high-frequency noun plurals. A mathematical formalization of a parallel dual-route race model is presented that accounts for the patterns in the observed reaction time data with essentially one free parameter, the speed of the parsing route. Parsing for noun plurals appears to be a time-costly process, which we attribute to the ambiguity of -en,a suffix that is predominantly used as a verbal ending. A third experiment contrasted nouns and verbs. This experiment revealed no effect of surface frequency for verbs, but again a solid effect for nouns. Together, our results suggest that many noun plurals are stored in order to avoid the time-costly resolution of the subcategorization conflict that arises when the -ensuffix is attached to nouns.

Abstract

In this paper, I present an analysis of the so-called tense forms of Biblical Hebrew. While there is fairly broad consensus on the interpretation of the yiqtol tense form, the interpretation of the qdtal tense form has led to considerable controversy. I will argue that the qātal form has no intrinsic semantic value and that it serves a pragmatic function only, namely, signaling to the hearer that the event or state expressed by the verb cannot be tightly integrated into the discourse representation of the hearer, given the speaker's estimate of their common ground.

Abstract

This paper addresses the relation between meaning, lexical productivity, and frequency of use. Using density estimation as a visualization tool, we show that differences in semantic structure can be reflected in probability density functions estimated for word frequency distributions. We call attention to an example of a bimodal density, and suggest that bimodality arises when distributions of well-entrenched lexical tems, which appear to be lognormal, are mixed with distributions of productively reated nonce formations

Abstract

As children gradually master grammatical rules, they often go through a period of producing form-meaning associations that were not observed in the input. For example, 2- to 3-year-old English-learning children use the bare form of verbs in settings that require obligatory past tense meaning while already starting to produce the grammatical –ed inflection. While many studies have focused on overgeneralization errors, fewer studies have attempted to explain the root of this earlier stage of rule acquisition. In this work, we use computational modeling to replicate children's production behavior prior to the generalization of past tense production in English. We illustrate how seemingly erroneous productions emerge in a model, without being licensed in the grammar and despite the model aiming at conforming to grammatical forms. Our results show that bare form productions stem from a tension between two factors: (1) trying to produce a less frequent meaning (the past tense) and (2) being unable to restrict the production of frequent forms (the bare form) as learning progresses. Like children, our model goes through a stage of bare form production and then converges on adult-like production of the regular past tense, showing that these different stages can be accounted for through a single learning mechanism.

Abstract

Pairwise maximum likelihood (PML) estimation is a promising method for multilevel models with discrete responses. Multilevel models take into account that units within a cluster tend to be more alike than units from different clusters. The pairwise likelihood is then obtained as the product of bivariate likelihoods for all within-cluster pairs of units and items. In this study, we investigate the PML estimation method with computationally intensive multilevel random intercept and random slope structural equation models (SEM) in discrete data. In pursuing this, we first reconsidered the general ‘wide format’ (WF) approach for SEM models and then extend the WF approach with random slopes. In a small simulation study we the determine accuracy and efficiency of the PML estimation method by varying the sample size (250, 500, 1000, 2000), response scales (two-point, four-point), and data-generating model (mediation model with three random slopes, factor model with one and two random slopes). Overall, results show that the PML estimation method is capable of estimating computationally intensive random intercept and random slopes multilevel models in the SEM framework with discrete data and many (six or more) latent variables with satisfactory accuracy and efficiency. However, the condition with 250 clusters combined with a two-point response scale shows more bias.

Abstract

Western Austronesian languages, like Tagalog, have unique, complex voice systems that require the correct combinations of verbal and nominal markers, raising many questions about their learnability. In this article, we review the experimental and observational studies on both the L1 and L2 acquisition of Tagalog. The reviewed studies reveal error patterns that reflect the complex nature of the Tagalog voice system. The main goal of the article is to present a full picture of commission errors in young Filipino children’s expression of causation and agency in Tagalog by describing patterns of nominal marking and voice marking in L1 Tagalog and L2 Tagalog. It also aims to provide an overview of existing research, as well as characterize research on nominal and verbal acquisition, specifically in terms of research problems, data sources, and methodology. Additionally, we discuss the research gaps in at least fifty years’ worth of studies in the area from the 1960’s to the present, as well as ideas for future research to advance the state of the art.

Abstract

Gliomas are brain tumors infiltrating healthy cortical and subcortical areas that may host cognitive functions, such as language. If these areas are damaged during surgery, the patient might develop word retrieval or articulation problems. For this reason, many glioma patients are operated on awake, while their language functions are tested. For this practice, quite simple tests are used, for example, picture naming. This paper describes the process and timeline of picture naming (noun retrieval) and shows the timeline and localization of the distinguished stages. This is relevant information for presurgical language testing with navigated Magnetic Stimulation (nTMS). This novel technique allows us to identify cortical involved in the language production process and, thus, guides the neurosurgeon in how to approach and remove the tumor. We argue that not only nouns, but also verbs should be tested, since sentences are built around verbs, and sentences are what we use in daily life. This approach’s relevance is illustrated by two case studies of glioma patients.

Abstract

Changes in oscillatory brain dynamics can be studied by means of induced band power (IBP) analyses, which quantify event-related changes in amplitude of frequency-specific EEG rhythms. Such analyses capture EEG phenomena that are not part of traditional event-related potential measures. The present study investigated whether IBP changes in the δ, θ, and α frequency ranges are sensitive to syntactic violations in sentences. Subjects read sentences that either were correct or contained a syntactic violation. The violations were either grammatical gender agreement violations, where a prenominal adjective was not appropriately inflected for the head noun's gender, or number agreement violations, in which a plural quantifier was combined with a singular head noun. IBP changes of the concurrently measured EEG were computed in five frequency bands of 2-Hz width, individually adjusted on the basis of subjects' α peak, ranging approximately from 2 to 12 Hz. Words constituting a syntactic violation elicited larger increases in θ power than the same words in a correct sentence context, in an interval of 300–500 ms after word onset. Of all the frequency bands studied, this was true for the θ frequency band only. The scalp topography of this effect was different for different violations: following number violations a left-hemispheric dominance was found, whereas gender violations elicited a right-hemisphere dominance of the θ power increase. Possible interpretations of this effect are considered in closing.

Abstract

Objective: To explore the reactivity of the theta and alpha rhythms during visuo-spatial working memory. Methods: One hundred and seventy-four subjects performed a delayed response task. They had to remember the spatial location of a target stimulus on a computer screen for a 1 or a 4 s retention interval. The target either remained visible throughout the entire interval (sensory trials) or disappeared after 150 ms (memory trials). Changes in induced band power (IBP) in the electroencephalogram (EEG) were analyzed in 4 narrow, individually adjusted frequency bands between 4 and 12 Hz. Results: After presentation of the target stimulus, a phasic power increase was found, irrespective of condition and delay interval, in the lower (roughly, 4–8 Hz) frequency bands, with a posterior maximum. During the retention interval, sustained occipital–parietal alpha power increase and frontal theta power decrease were found. Most importantly, the memory trials showed larger IBP decreases in the theta band over frontal electrodes than the sensory trials. Conclusions: The phasic power increase following target onset is interpreted to reflect encoding of the target location. The sustained theta decrease, which is larger for memory trials, is tentatively interpreted to reflect visuo-spatial working memory processes.

Abstract

By analyzing event-related changes in induced band power in narrow frequency bands of the human electroencephalograph, the present paper explores a possible functional role of the alpha and theta rhythms during the processing of words and of sentences. The results show a phasic power increase in the theta frequency range, together with a phasic power decrease in the alpha frequency range, following the presentation of words in a sentence. These effects may be related to word processing, either lexical or in relation to sentence context. Most importantly, there is a slow and highly frequency-specific increase in theta power as a sentence unfolds, possibly related to the formation of an episodic memory trace, or to incremental verbal working memory load.

Abstract

Previous research has suggested that the stimulus-preceding negativity (SPN) is largely independent of stimulus modality. In contrast, the scalp topography of the event related desynchronization (ERD) related to the anticipation of stimuli providing knowledge of results (KR) is modality dependent. These findings, combined with functional SPN research, lead to the hypothesis that anticipatory ERD reflects anticipatory attention, whereas the SPN mainly depends on the affective-motivational properties of the anticipated stimulus. To further investigate the prestimulus ERD, and compare this measure with the SPN, 12 participants performed a time-estimation task, and were informed about the quality of their time estimation by an auditory or a visual stimulus providing KR. The KR stimuli could be either intact or degraded. Auditory degraded KR stimuli were less effective than other KR stimuli in guiding subsequent behavior, and were preceded by a larger SPN. There were no effects of degradation on the SPN in the visual modality. Preceding auditory KR stimuli no ERD was present, whereas preceding visual stimuli an occipital ERD was found. However, contrary to expectation, the latter was larger preceding intact than preceding degraded stimuli. It is concluded that the data largely agree with an interpretation of the pre-KR SPN as a reflection of the anticipation of the affective-motivational value of KR stimuli, and of the prestimulus ERD as a perceptual anticipatory attention process.

Abstract

This book examines Latin word order, its historical origins in Proto-Indo-European and the shift in ordering patterns that took place in syntax and morphology in the history of Latin and (early) French (OV or left branching giving way to VO or right branching). Subsequently, analysis of the acquisition of ordering patterns shows that the archaic structuration—when complex—is acquired with difficulty. Diachronic and psycholinguistic analysis therefore demonstrates that the order of grammatical structures in Modern French, for example, is the result of a long-lasting development that psycholinguistic data can account for.

Abstract

While formal variation in Latin’s numerals is generally acknowledged, little is known about (relative) incidence, distribution, context, or linguistic productivity. Addressing this lacuna, this article examines “decads+” in Latin, which convey the numbers between the full decads: the teens (‘eleven’ through ‘nineteen’) as well as the numerals between the higher decads starting at ‘twenty-one’ through ‘ninety-nine’. Latin’s decads+ are compounds and prone to variation. The data, which are drawn from a variety of sources, reveal (a) substantial formal variation in Latin, both internally and typologically; (b) co-existence of several types of formation; (c) productivity of potential borrowings; (d) resilience of early formations; (e) patterns in structure and incidence that anticipate the Romance numerals; and (f) historical trends. From a typological and general linguistic perspective as well, Latin’s decads+ are most relevant because their formal variation involves sequence, connector, and arithmetical operations and because their historical depth shows a gradual shift away from widespread formal variation, eventually resulting in the relatively rigid system found in Romance. Moreover, the combined system attested in decads+ in Latin – based on a combination of inherited, innovative and borrowed patterns and reflecting different stages of development – presents a number of typological inconsistencies that require further assessment

Abstract

If a conversation is to proceed smoothly, the participants have to take turns to speak. Studies of conversation have shown that there are signals which speakers give to inform listeners that they are willing to hand over the conversational turn1−4. Some of these signals are part of the text (for example, completion of syntactic segments), some are non-verbal (such as completion of a gesture), but most are carried by the pitch, timing and intensity pattern of the speech; for example, both pitch and loudness tend to drop particularly low at the end of a speaker's turn. When one speaker interrupts another, the two can be said to be disputing who has the turn. Interruptions can occur because one participant tries to dominate or disrupt the conversation. But it could also be the case that mistakes occur in the way these subtle turn-yielding signals are transmitted and received. We demonstrate here that many interruptions in an interview with Mrs Margaret Thatcher, the British Prime Minister, occur at points where independent judges agree that her turn appears to have finished. It is suggested that she is unconsciously displaying turn-yielding cues at certain inappropriate points. The turn-yielding cues responsible are identified.

Abstract

We investigated the time course of conjunctive ''same''-''different'' judgements for visually presented object pairs by means of combined reaction time and on-line eye movement measurements. The analyses of viewing patterns, viewing times, and reaction times showed that participants engaged in a parallel self-terminating search for differences. In addition, the results obtained for objects differing in only one dimension suggest that processing times may depend on the relative codability of the stimulus dimensions. The results are reviewed in a broader framework in view of higher-order processes. We propose that overspecifications of colour, often found in object descriptions, may have an ''early'' visual rather than a ''late'' linguistic origin. In a parallel assessment of the detection materials, participants overspecified the objects' colour substantially more often than their size. We argue that referential overspecifications of colour are largely attributable to mechanisms of visual discrimination.

Abstract

In face-to-face communication, humans are faced with multiple layers of discontinuous multimodal signals, such as head, face, hand gestures, speech and non-speech sounds, which need to be interpreted as coherent and unified communicative actions. This implies a fundamental computational challenge: optimally binding only signals belonging to the same communicative action while segregating signals that are not connected by the communicative content. How do we achieve such an extraordinary feat, reliably, and efficiently? To address this question, we need to further move the study of human communication beyond speech-centred perspectives and promote a multimodal approach combined with interdisciplinary cooperation. Accordingly, we seek to reconcile two explanatory frameworks recently proposed in psycholinguistics and sensory neuroscience into a neurocognitive model of multimodal face-to-face communication. First, we introduce a psycholinguistic framework that characterises face-to-face communication at three parallel processing levels: multiplex signals, multimodal gestalts and multilevel predictions. Second, we consider the recent proposal of a lateral neural visual pathway specifically dedicated to the dynamic aspects of social perception and reconceive it from a multimodal perspective (“lateral processing pathway”). Third, we reconcile the two frameworks into a neurocognitive model that proposes how multiplex signals, multimodal gestalts, and multilevel predictions may be implemented along the lateral processing pathway. Finally, we advocate a multimodal and multidisciplinary research approach, combining state-of-the-art imaging techniques, computational modelling and artificial intelligence for future empirical testing of our model.

Abstract

Language is a universal human ability, acquired readily by young children, whootherwise struggle with many basics of survival. And yet, language ability is variableacross individuals. Naturalistic and experimental observations suggest that children’slinguistic skills vary with factors like socioeconomic status and children’s gender.But which factors really influence children’s day-to-day language use? Here, weleverage speech technology in a big-data approach to report on a unique cross-culturaland diverse data set: >2,500 d-long, child-centered audio-recordings of 1,001 2- to48-mo-olds from 12 countries spanning six continents across urban, farmer-forager,and subsistence-farming contexts. As expected, age and language-relevant clinical risksand diagnoses predicted how much speech (and speech-like vocalization) childrenproduced. Critically, so too did adult talk in children’s environments: Children whoheard more talk from adults produced more speech. In contrast to previous conclusionsbased on more limited sampling methods and a different set of language proxies,socioeconomic status (operationalized as maternal education) was not significantlyassociated with children’s productions over the first 4 y of life, and neither weregender or multilingualism. These findings from large-scale naturalistic data advanceour understanding of which factors are robust predictors of variability in the speechbehaviors of young learners in a wide range of everyday contexts

Abstract

The Basic Variety (BV) as conceived by Klein and Perdue (K&P) is a relatively stable state in the process of spontaneous (adult) second language acquisition, characterized by a small set of phrasal, semantic and pragmatic principles. These principles are derived by inductive generalization from a fairly large body of data. They are considered by K&P as roughly equivalent to those of Universal Grammar (UG) in the sense of Chomsky's Minimalist Program, with the proviso that the BV allows for only weak (or unmarked) formal features. The present article first discusses the viability of the BV principles proposed by K&P, arguing that some of them are in need of clarification with learner varieties, and that they are, in any case, not likely to be part of UG, as they exclude phenomena (e.g., so-called psych verbs) that cannot be ruled out even from the core of natural language. The article also considers the proposal that learner varieties of the BV type are completely unmarked instantiations of UG. Putting aside problems arising from the Minimalist Program, especially the question whether a grammar with only weak features would be a factual possibility and what it would look like, it is argued that the BV as characterized by K&P must be considered as the result of a process that crucially differs from first language acquisition as furnished by UG for a number of reasons, including properties of the BV itself. As a matter of fact, several of the properties claimed for the BV by K&P are more likely the result of general learning strategies than of language-specific principles. If this is correct, the characterization of the BV is a fairly interesting result, albeit of a rather different type than K&P suggest.

Abstract

The murine homologues of the loci for McLeod syndrome (XK), Dent's disease (CICN5), and synaptophysin (SYP) have been mapped to the proximal region of the mouse X chromosome and positioned with respect to other conserved loci in this region using a total of 948 progeny from two separate Mus musculus x Mus spretus backcrosses. In the mouse, the order of loci and evolutionary breakpoints (EB) has been established as centromere-(DXWas70, DXHXF34h)-EB-Clcn5-(Syp, DXMit55, DXMit26)-Tfe3-Gata1-EB-Xk-Cybb-telomere. In the proximal region of the human X chromosome short arm, the position of evolutionary breakpoints with respect to key loci has been established as DMD-EB-XK-PFC-EB-GATA1-C1CN5-EB-DXS1272E-ALAS2-E B-DXF34-centromere. These data have enabled us to construct a high-resolution genetic map for the approximately 3-cM interval between DXWas70 and Cybb on the mouse X chromosome, which encompasses 10 loci. This detailed map demonstrates the power of high-resolution genetic mapping in the mouse as a means of determining locus order in a small chromosomal region and of providing an accurate framework for the construction of physical maps.

Abstract

We know that speech planning in conversational turn-taking can happen in overlap with the previous turn and research suggests that it starts as early as possible, that is, as soon as the gist of the previous turn becomes clear. The present study aimed to investigate whether planning proceeds all the way up to the last stage of articulatory preparation (i.e., putting the articulators in place for the first phoneme of the response) and what the timing of this process is. Participants answered pre-recorded quiz questions (being under the illusion that they were asked live), while their tongue movements were measured using ultrasound. Planning could start early for some quiz questions (i.e., midway during the question), but late for others (i.e., only at the end of the question). The results showed no evidence for a difference between tongue movements in these two types of questions for at least two seconds after planning could start in early-planning questions, suggesting that speech planning in overlap with the current turn proceeds more slowly than in the clear. On the other hand, when time-locking to speech onset, tongue movements differed between the two conditions from up to two seconds before this point. This suggests that articulatory preparation can occur in advance and is not fully tied to the overt response itself.

Abstract

There are currently a number of psycholinguistic models in which processing at a particular level of representation is characterized by the generation of multiple outputs, with resolution involving the use of information from higher levels of processing. Surprisingly, models with this architecture have been characterized as autonomous within the domain of word recognition and as interactive within the domain of sentence processing. We suggest that the apparent internal confusion is not, as might be assumed, due to fundamental differences between lexical and syntactic processing. Rather, we believe that the labels in each domain were chosen in order to obtain maximal contrast between a new model and the model or models that were currently dominating the field.

Abstract

There are currently a number of psycholinguistic models in which processing at a particular level of representation is characterized by the generation of multiple outputs, with resolution - but not generation - involving the use of information from higher levels of processing. Surprisingly, models with this architecture have been characterized as autonomous within the domain of word recognition but as interactive within the domain of sentence processing. We suggest that the apparent confusion is not, as might be assumed, due to fundamental differences between lexical and syntactic processing. Rather, we believe that the labels in each domain were chosen in order to obtain maximal contrast between a new model and the model or models that were currently dominating the field. The contradiction serves to highlight the inadequacy of a simple autonomy/interaction dichotomy for characterizing the architectures of current processing models.

Abstract

While listening to meaningful speech, auditory input is processed more rapidly near the end (vs. beginning) of sentences. Although several studies have shown such word-to-word changes in auditory input processing, it is still unclear from which processing level these word-to-word dynamics originate. We investigated whether predictions derived from sentential context can result in auditory word-processing dynamics during sentence tracking. We presented healthy human participants with auditory stimuli consisting of word sequences, arranged into either predictable (coherent sentences) or less predictable (unstructured, random word sequences) 42-Hz amplitude-modulated speech, and a continuous 25-Hz amplitude-modulated distractor tone. We recorded RTs and frequency-tagged neuroelectric responses 1(auditory steady-state responses) to individual words at multiple temporal positions within the sentences, and quantified sentential context effects at each position while controlling for individual word characteristics (i.e., phonetics, frequency, and familiarity). We found that sentential context increasingly facilitates auditory word processing as evidenced by accelerated RTs and increased auditory steady-state responses to later-occurring words within sentences. These purely top–down contextually driven auditory word-processing dynamics occurred only when listeners focused their attention on the speech and did not transfer to the auditory processing of the concurrent distractor tone. These findings indicate that auditory word-processing dynamics during sentence tracking can originate from sentential predictions. The predictions depend on the listeners' attention to the speech, and affect only the processing of the parsed speech, not that of concurrently presented auditory streams.

Abstract

The notion of relational grammar is extented to ternary relations and illustrated by a fragment of English. Some of Peirce's terms for ternary relations are shown to be incorrect and corrected.

Abstract

This study explores how adults and children describe placement events (e.g., putting a book on a table) in a range of different languages (Finnish, English, German, Russian, Hindi, Tzeltal Maya, Spanish, and Turkish). Results show that the eight languages grammatically encode placement events in two main ways (Talmy, 1985, 1991), but further investigation reveals fine-grained crosslinguistic variation within each of the two groups. Children are sensitive to these finer-grained characteristics of the input language at an early age, but only when such features are perceptually salient. Our study demonstrates that a unitary notion of 'event' does not suffice to characterize complex but systematic patterns of event encoding crosslinguistically, and that children are sensitive to multiple influences, including the distributional properties of the target language, in constructing these patterns in their own speech.

Abstract

(From the chapter) the first section considers very briefly the kinds of processes that can be inferred to underlie errors that do not set in until after a period of correct usage acquisition often seems to be a more extended process than we have envisioned summarize a currently influential model of how linguistic forms, meaning, and communication are interrelated in the acquisition of language, point out some challenging problems for this model, and suggest that the notion of "meaning" in language must be reconceptualized before we can hope to solve these problems evidence from several types of late errors is marshalled in support of these arguments (From the preface) provides many examples of new errors that children introduce at relatively advanced stages of mastery of semantics and syntax Bowerman views these seemingly backwards steps as indications of definite steps forward by the child achieving reflective, flexible and integrated systems of semantics and syntax (

Abstract

Reprinted from: Bowerman, M. (1990). Mapping thematic roles onto syntactic functions: Are children helped by innate linking rules? Linguistics, 28, 1253-1289.

Abstract

Within the ISLE Metadata Initiative (IMDI) project a user-friendly editor to enter metadata descriptions and a browser operating on the linked metadata descriptions were developed. Both tools support the usage of Controlled Vocabulary (CV) repositories by means of the specification of an URL where the formal CV definition data is available.

Abstract

The recent increase in the number and complexity of the language resources available on the Internet is followed by a similar increase of available tools for linguistic analysis. Ideally the user does not need to be confronted with the question in how to match tools with resources. If resource repositories and tool repositories offer adequate metadata information and a suitable exchange protocol is developed this matching process could be performed (semi-) automatically.

Abstract

Native speakers of Dutch with English as a second language and native speakers of English participated in an English lexical decision experiment. Phonemes in real words were replaced by others from which they are hard to distinguish for Dutch listeners. Non-native listeners judged the resulting near-words more often as a word than native listeners. This not only happened when the phonemes that were exchanged did not exist as separate phonemes in the native language Dutch, but also when phoneme pairs that do exist in Dutch were used in word-final position, where they are not distinctive in Dutch. In an English bimodal priming experiment with similar groups of participants, word pairs were used which differed in one phoneme. These phonemes were hard to distinguish for the non-native listeners. Whereas in native listening both words inhibited each other, in non-native listening presentation of one word led to unresolved competition between both words. The results suggest that inaccurate phoneme processing by non-native listeners leads to the activation of spurious lexical competitors.

Abstract

In a famous paper Harvey Sacks (1974) argued that the sequential properties of greeting conventions, as well as those governing the flow of information, mean that 'everyone has to lie'. In this paper I show this dictum to be equally true in the Tzeltal Mayan community of Tenejapa, in southern Mexico, but for somewhat different reasons. The phenomenon of interest is the practice of routine fearsome threats to small children. Based on a longitudinal corpus of videotaped and tape-recorded naturally-occurring interaction between caregivers and children in five Tzeltal families, the study examines sequences of Tzeltal caregivers' speech aimed at controlling the children's behaviour and analyzes the children's developing pragmatic skills in handling such controlling utterances, from prelinguistic infants to age five and over. Infants in this society are considered to be vulnerable, easily scared or shocked into losing their 'souls', and therefore at all costs to be protected and hidden from outsiders and other dangers. Nonetheless, the chief form of control (aside from physically removing a child from danger) is to threaten, saying things like "Don't do that, or I'll take you to the clinic for an injection," These overt scare-threats - rarely actually realized - lead Tzeltal children by the age of 2;6 to 3;0 to the understanding that speech does not necessarily convey true propositions, and to a sensitivity to the underlying motivations for utterances distinct from their literal meaning. By age 4;0 children perform the same role to their younger siblings;they also begin to use more subtle non-true (e.g. ironic) utterances. The caretaker practice described here is related to adult norms of social lying, to the sociocultural context of constraints on information flow, social control through gossip, and the different notion of 'truth' that arises in the context of non-verifiability characteristic of a small-scale nonliterate society.

Abstract

This paper surveys the concept of culture as used in recent work in cognitive science, assessing the very different (and sometimes minimal) role 'culture' plays in different branches and schools of linguistics: generative approaches, descriptive/comparative linguistics, typology, cognitive linguistics, semantics, pragmatics, psycholinguistics, linguistic and cognitive anthropology. The paper then describes research on one specific topic, spatial language and conceptualization, describes a methodology for studying it cross-linguistically and cross-culturally. Finally, it considers the implications of results in this area for how we can fruitfully conceptualize 'culture', arguing for an approach which shifts back and forth between individual mind and collective representations, between universals and particulars, and ties 'culture' to our biological roots.

Abstract

How do children isolate the semantic package contained in verb roots in the Mayan language Tzeltal? One might imagine that the canonical CVC shape of roots characteristic of Mayan languages would make the job simple, but the root is normally preceded and followed by affixes which mask its identity. Pye (1983) demonstrated that, in Kiche' Mayan, prosodic salience overrides semantic salience, and children's first words in Kiche' are often composed of only the final (stressed) syllable constituted by the final consonant of the CVC root and a 'meaningless' termination suffix. Intonation thus plays a crucial role in early Kiche' morphological development. Tzeltal presents a rather different picture: The first words of children around the age of 1;6 are bare roots, children strip off all prefixes and suffixes which are obligatory in adult speech. They gradually add them, starting with the suffixes (which receive the main stress), but person prefixes are omitted in some contexts past a child's third birthday, and one obligatory aspectual prefix (x-) is systematically omitted by the four children in my longitudinal study even after they are four years old. Tzeltal children's first verbs generally show faultless isolation of the root. An account in terms of intonation or stress cannot explain this ability (the prefixes are not all syllables; the roots are not always stressed). This paper suggests that probable clues include the fact that the CVC root stays constant across contexts (with some exceptions) whereas the affixes vary, that there are some linguistic contexts where the root occurs without any prefixes (relatively frequent in the input), and that the Tzeltal discourse convention of responding by repeating with appropriate deictic alternation (e.g., "I see it." "Oh, you see it.") highlights the root.

Abstract

In this paper I take up the idea that human thinking is systematically biased in the direction of interactive thinking (E. Goody's anticipatory interactive planning), that is, that humans are peculiarly good at, and inordinately prone to, attributing intentions and goals to one other (as well as to non-humans), and that they routinely orient to presumptions about each other's intentions in what they say and do. I explore the implications of that idea for an understanding of politeness in interaction, taking as a starting point the Brown and Levinson (1987) model of politeness, which assumes interactive thinking, a notion implicit in the formulation of politeness as strategic orientation to face. Drawing on an analysis of the phenomenon of conventionalized ‘irony’ in Tzeltal, I emphasize that politeness does not inhere in linguistic form per se but is a matter of conveying a polite intention, and argue that Tzeltal irony provides a prime example of one way in which humans' highly-developed intellectual machinery for inferring alter's intentions is put to the service of social relationships.

Abstract

Two languages, historically related, both have lexical stress, with word stress distinctions signalled in each by the same suprasegmental cues. In each language, words can overlap segmentally but differ in placement of primary versus secondary stress (OCtopus, ocTOber). However, secondary stress occurs more often in the words of one language, Dutch, than in the other, English, and largely because of this, Dutch listeners find it helpful to use suprasegmental stress cues when recognising spoken words. English listeners, in contrast, do not; indeed, Dutch listeners can outdo English listeners in correctly identifying the source words of English word fragments (oc-). Here we show that Dutch-native listeners who reside in an English-speaking environment and have become dominant in English, though still maintaining their use of these stress cues in their L1, ignore the same cues in their L2 English, performing as poorly in the fragment identification task as the L1 English do.

Abstract

For multimodal annotations an exhaustive encoding system for gestures was developed to facilitate research. The structural requirements of multimodal annotations were analyzed to develop an Abstract Corpus Model which is the basis for a powerful annotation and exploitation tool for multimedia recordings and the definition of the XML-based EUDICO Annotation Format. Finally, a metadata-based data management environment has been setup to facilitate resource discovery and especially corpus management. Bt means of an appropriate digitization policy and their online availability researchers have been able to build up a large corpus covering gesture and sign language data.

Abstract

Introduction
Despite abundant research on the role of Broca's area in language processing, there is still no consensus on language specificity of this region and its connectivity network.

Methods
The present study employed the meta-analytic connectivity modeling procedure to identify and compare domain-specific (language-specific) and domain-general (shared between language and other domains) functional connectivity patterns of three subdivisions within the broadly defined Broca's area: pars opercularis (IFGop), pars triangularis (IFGtri), and pars orbitalis (IFGorb) of the left inferior frontal gyrus.

Results
The findings revealed a left-lateralized frontotemporal network for all regions of interest underlying domain-specific linguistic functions. The domain-general network, however, spanned frontoparietal regions that overlap with the multiple-demand network and subcortical regions spanning the thalamus and the basal ganglia.

Conclusions
The findings suggest that language specificity of Broca's area emerges within a left-lateralized frontotemporal network, and that domain-general resources are garnered from frontoparietal and subcortical networks when required by task demands.

Abstract

This chapter provides an introduction to the handbook. It succintly overviews the key questions in the field of L3/Ln acquisition and summarizes the scope of all the chapters included. The chapter ends by raising some outstanding questions that the field needs to address.

Abstract

In order to understand early word learning we need to better understand and quantify properties of the input that young children receive. We extended the human simulation paradigm (HSP) using egocentric videos taken from infant head-mounted cameras. The videos were further annotated with gaze information indicating in-the-moment visual attention from the infant. Our new HSP prompted participants for two types of responses, thus differentiating referential from semantic ambiguity in the learning input. Consistent with findings on visual attention in word learning, we find a strongly bimodal distribution over HSP accuracy. Even in this open-ended task, most videos only lead to a small handful of common responses. What's more, referential ambiguity was the key bottleneck to performance: participants can nearly always recover the exact word that was said if they identify the correct referent. Finally, analysis shows that adult learners relied on particular, multimodal behavioral cues to infer those target referents.

Abstract

Neuronal populations code similar concepts by similar activity patterns across the human brain's semantic networks. However, it is unclear to what extent such meaning-to-symbol mapping reflects distributional statistics, or experiential information grounded in sensorimotor and emotional knowledge. We asked whether integrating distributional and experiential data better distinguished conceptual categories than each method taken separately. We examined the similarity structure of fMRI patterns elicited by visually presented action- and object-related words using representational similarity analysis (RSA). We found that the distributional and experiential/integrative models respectively mapped the high-dimensional semantic space in left inferior frontal, anterior temporal, and in left precentral, posterior inferior/middle temporal cortex. Furthermore, results from model comparisons uncovered category-specific similarity patterns, as both distributional and experiential models matched the similarity patterns for action concepts in left fronto-temporal cortex, whilst the experiential/integrative (but not distributional) models matched the similarity patterns for object concepts in left fusiform and angular gyrus.

Abstract

Speaking requires the temporally coordinated planning of core linguistic information, from conceptual meaning to articulation. Recent neurophysiological results suggested that these operations involve a cascade of neural events with subsequent onset times, whilst competing evidence suggests early parallel neural activation. To test these hypotheses, we examined the sources of neuromagnetic activity recorded from 34 participants overtly naming 134 images from 4 object categories (animals, tools, foods and clothes). Within each category, word length and phonological neighbourhood density were co-varied to target phonological/phonetic processes. Multivariate pattern analyses (MVPA) searchlights in source space decoded object categories in occipitotemporal and middle temporal cortex, and phonological/phonetic variables in left inferior frontal (BA 44) and motor cortex early on. The findings suggest early activation of multiple variables due to intercorrelated properties and interactivity of processing, thus raising important questions about the representational properties of target words during the preparatory time enabling overt speaking.

Abstract

Eye gaze is a powerful ostensive cue in infant-caregiver interactions, with demonstrable effects on language acquisition. While the link between gaze following and later vocabulary is well-established, the effects of eye gaze on other aspects of language, such as speech processing, are less clear. In this EEG study, we examined the effects of the speaker’s eye gaze on ten-month-old infants’ neural tracking of naturalistic audiovisual speech, a marker for successful speech processing. Infants watched videos of a speaker telling stories, addressing the infant with direct or averted eye gaze. We assessed infants’ speech-brain coherence at stress (1–1.75 Hz) and syllable (2.5–3.5 Hz) rates, tested for differences in attention by comparing looking times and EEG theta power in the two conditions, and investigated whether neural tracking predicts later vocabulary. Our results showed that infants’ brains tracked the speech rhythm both at the stress and syllable rates, and that infants’ neural tracking at the syllable rate predicted later vocabulary. However, speech-brain coherence did not significantly differ between direct and averted gaze conditions and infants did not show greater attention to direct gaze. Overall, our results suggest significant neural tracking at ten months, related to vocabulary development, but not modulated by speaker’s gaze.

Abstract

Tense/aspect morphology on verbs is often thought to depend on event features like telicity, but it is not known how speakers identify these features in visual scenes. To examine this question, we asked Japanese speakers to describe computer-generated animations of simple actions with variation in visual features related to telicity. Experiments with adults and children found that they could use goal information in the animations to select appropriate past and progressive verb forms. They also produced a large number of different verb forms. To explain these findings, a deep-learning model of verb production from visual input was created that could produce a human-like distribution of verb forms. It was able to use visual cues to select appropriate tense/aspect morphology. The model predicted that video duration would be related to verb complexity, and past tense production would increase when it received the endpoint as input. These predictions were confirmed in a third study with Japanese adults. This work suggests that verb production could be tightly linked to visual heuristics that support the understanding of events.

Abstract

Two groups of listeners with Dutch and British English language backgrounds judged Dutch and British English utterances, respectively, which varied in the intonation contour on the scales EMPHATIC vs. NOT EMPHATIC and SURPRISED vs. NOT SURPRISED, two meanings derived from the Effort Code. The stimuli, which differed in sentence mode but were otherwise lexically equivalent, were varied in peak height, peak alignment, end pitch, and overall register. In both languages, there are positive correlations between peak height and degree of emphasis, between peak height and degree of surprise, between peak alignment and degree of surprise, and between pitch register and degree of surprise. However, in all these cases, Dutch stimuli lead to larger perceived meaning differences than the British English stimuli. This difference in the extent to which increased pitch height triggers increases in perceived emphasis and surprise is argued to be due to the difference in the standard pitch ranges between Dutch and British English. In addition, we found a positive correlation between pitch register and the degree of emphasis in Dutch, but a negative correlation in British English. This is an unexpected difference, which illustrates a case of ambiguity in the meaning of pitch.

Abstract

The current study investigated how the role of pitch in one’s native language and L2 experience influenced musical melodic processing by testing Turkish and Mandarin Chinese advanced and beginning learners of English as an L2. Pitch has a lower functional load and shows a simpler pattern in Turkish than in Chinese as the former only contrasts between presence and the absence of pitch elevation, while the latter makes use of four different pitch contours lexically. Using the Musical Ear Test as the tool, we found that the Chinese listeners outperformed the Turkish listeners, and the advanced L2 learners outperformed the beginning learners. The Turkish listeners were further tested on their discrimination of bisyllabic Chinese lexical tones, and again an L2 advantage was observed. No significant difference was found for working memory between the beginning and advanced L2 learners. These results suggest that richness of tonal inventory of the native language is essential for triggering a music processing advantage, and on top of the tone language advantage, the L2 experience yields a further enhancement. Yet, unlike the tone language advantage that seems to relate to pitch expertise, learning an L2 seems to improve sound discrimination in general, and such improvement exhibits in non-native lexical tone discrimination.

Abstract

In this study, we analyze one French signing and one French speaking family’s interaction during dinner. The families composed of two parents and two children aged 3 to 11 were filmed with three cameras to capture all family members’ behaviors. The three videos per dinner were synchronized and coded on ELAN. We annotated all participants’ acting, and languaging.
Our quantitative analyses show how family members collaboratively manage multiple streams of activity through the embodied performances of dining and interacting. We uncover different profiles according to participants’ modality of expression and status (focusing on the mother and the younger child). The hearing participants’ co-activity management illustrates their monitoring of dining and conversing and how they progressively master the affordances of the visual and vocal channels to maintain the simultaneity of the two activities. The deaf mother skillfully manages to alternate smoothly between dining and interacting. The deaf younger child manifests how she is in the process of developing her skills to manage multi-activity. Our qualitative analyses focus on the ecology of visual-gestural and audio-vocal languaging in the context of co-activity according to language and participant. We open new perspectives on the management of gaze and body parts in multimodal languaging.

Abstract

This study examines acoustic and aerodynamic characteristics of consonants in standard Korean and in Cheju, an endangered Korean language. The focus is on the well-known three-way distinction among voiceless stops (i.e., lenis, fortis, aspirated) and the two-way distinction between the voiceless fricatives /s/ and /s*/. While such a typologically unusual contrast among voiceless stops has long drawn the attention of phoneticians and phonologists, there is no single work in the literature that discusses a body of data representing a relatively large number of speakers. This study reports a variety of acoustic and aerodynamic measures obtained from 12 Korean speakers (four speakers of Seoul Korean and eight speakers of Cheju). Results show that, in addition to findings similar to those reported by others, there are three crucial points worth noting. Firstly, lenis, fortis, and aspirated stops are systematically differentiated from each other by the voice quality of the following vowel. Secondly, these stops are also differentiated by aerodynamic mechanisms. The aspirated and fortis stops are similar in supralaryngeal articulation, but employ a different relation between intraoral pressure and flow. Thirdly, our study suggests that the fricative /s/ is better categorized as “lenis” rather than “aspirated”. The paper concludes with a discussion of the implications of Korean data for theories of the voicing contrast and their phonological representations.

Abstract

In a semantic priming paradigm, the effects of different levels of processing on the N400 were assessed by changing the task demands. In the lexical decision task, subjects had to discriminate between words and nonwords and in the physical task, subjects had to discriminate between uppercase and lowercase letters. The proportion of related versus unrelated word pairs differed between conditions. A lexicality test on reaction times demonstrated that the physical task was performed nonlexically. Moreover, a semantic priming reaction time effect was obtained only in the lexical decision task. The level of processing clearly affected the event-related potentials. An N400 priming effect was only observed in the lexical decision task. In contrast, in the physical task a P300 effect was observed for either related or unrelated targets, depending on their frequency of occurrence. Taken together, the results indicate that an N400 priming effect is only evoked when the task performance induces the semantic aspects of words to become part of an episodic trace of the stimulus event.

Abstract

Background. Facial emotion recognition deficits are common after moderate-severe traumatic brain injury (TBI) and linked to poor social outcomes. We examine whether emotion recognition deficits extend to facial expressions depicted by emoji.
Methods. Fifty-one individuals with moderate-severe TBI (25 female) and fifty-one neurotypical peers (26 female) viewed photos of human faces and emoji. Participants selected the best-fitting label from a set of basic emotions (anger, disgust, fear, sadness, neutral, surprise, happy) or social emotions (embarrassed, remorseful, anxious, neutral, flirting, confident, proud).
Results. We analyzed the likelihood of correctly labeling an emotion by group (neurotypical, TBI), stimulus condition (basic faces, basic emoji, social emoji), sex (female, male), and their interactions. Participants with TBI did not significantly differ from neurotypical peers in overall emotion labeling accuracy. Both groups had poorer labeling accuracy for emoji compared to faces. Participants with TBI (but not neurotypical peers) had poorer accuracy for labeling social emotions depicted by emoji compared to basic emotions depicted by emoji. There were no effects of participant sex.
Discussion. Because emotion representation is more ambiguous in emoji than human faces, studying emoji use and perception in TBI is an important consideration for understanding functional communication and social participation after brain injury.

Abstract

Purpose

Real-world communication is situated in rich multimodal contexts, containing speech and gesture. Speakers often convey unique information in gesture that is not present in the speech signal (e.g., saying “He searched for a new recipe” while making a typing gesture). We examine the narrative retellings of participants with and without moderate-severe traumatic brain injury across three timepoints over two online Zoom sessions to investigate whether people with TBI can integrate information from co-occurring speech and gesture and if information from gesture persists across delays.

Methods

60 participants with TBI and 60 non-injured peers watched videos of a narrator telling four short stories. On key details, the narrator produced complementary gestures that conveyed unique information. Participants retold the stories at three timepoints: immediately after, 20-min later, and one-week later. We examined the words participants used when retelling these key details, coding them as a Speech Match (e.g., “He searched for a new recipe”), a Gesture Match (e.g., “He searched for a new recipe online), or Other (“He looked for a new recipe”). We also examined whether participants produced representative gestures themselves when retelling these details.

Results

Despite recalling fewer story details, participants with TBI were as likely as non-injured peers to report information from gesture in their narrative retellings. All participants were more likely to report information from gesture and produce representative gestures themselves one-week later compared to immediately after hearing the story.

Conclusion

We demonstrated that speech-gesture integration is intact after TBI in narrative retellings. This finding has exciting implications for the utility of gesture to support comprehension and memory after TBI and expands our understanding of naturalistic multimodal language processing in this population.

Abstract

Impaired facial affect recognition is common after traumatic brain injury (TBI) and linked to poor social outcomes. We explored whether perception of emotions depicted by emoji is also impaired after TBI. Fifty participants with TBI and 50 non-injured peers generated free-text labels to describe emotions depicted by emoji and rated their levels of valence and arousal on nine-point rating scales. We compared how the two groups’ valence and arousal ratings were clustered and examined agreement in the words participants used to describe emoji. Hierarchical clustering of affect ratings produced four emoji clusters in the non-injured group and three emoji clusters in the TBI group. Whereas the non-injured group had a strongly positive and a moderately positive cluster, the TBI group had a single positive valence cluster, undifferentiated by arousal. Despite differences in cluster numbers, hierarchical structures of the two groups’ emoji ratings were significantly correlated. Most emoji had high agreement in the words participants with and without TBI used to describe them. Participants with TBI perceived emoji similarly to non-injured peers, used similar words to describe emoji, and rated emoji similarly on the valence dimension. Individuals with TBI showed small differences in perceived arousal for a minority of emoji. Overall, results suggest that basic recognition processes do not explain challenges in computer-mediated communication reported by adults with TBI. Examining perception of emoji in context by people with TBI is an essential next step for advancing our understanding of functional communication in computer-mediated contexts after brain injury.

Abstract

Four cross-modal priming experiments and two forced-choice identification experiments investigated the use of suprasegmental cues to stress in the recognition of spoken English words, by native (English-speaking) and non- native (Dutch) listeners. Previous results had indicated that suprasegmental information was exploited in lexical access by Dutch but not by English listeners. For both listener groups, recognition of visually presented target words was faster, in comparison to a control condition, after stress-matching spoken primes, either monosyllabic (mus- from MUsic /muSEum) or bisyl labic (admi- from ADmiral/admiRAtion). For native listeners, the effect of stress-mismatching bisyllabic primes was not different from that of control primes, but mismatching monosyllabic primes produced partial facilitation. For non-native listeners, both bisyllabic and monosyllabic stress-mismatching primes produced partial facilitation. Native English listeners thus can exploit suprasegmental information in spoken-word recognition, but information from two syllables is used more effectively than information from one syllable. Dutch listeners are less proficient at using suprasegmental information in English than in their native language, but, as in their native language, use mono- and bisyllabic information to an equal extent. In forced-choice identification, Dutch listeners outperformed native listeners at correctly assigning a monosyllabic fragment (e.g., mus-) to one of two words differing in stress.