Presentations

Abstract

Nearly all human languages have grammars with complex recursive structures. These structures pose notable learning challenges. Two distributional properties of the input may facilitate learning: the presence of semantic biases (e.g. p(barks|dog) > p(talks|dog)) and the Zipf-distribution, with short sentences being extremely more frequent than longer ones. This project tested the effect of these sources of information on statistical learning of a hierarchical center-embedding grammar, using an artificial grammar learning paradigm. Semantic biases were represented by variations in transitional probabilities between words, with a biased input (p(barks|dog) > p(talks|dog)) compared to a non-biased input (p(barks|dog) = p(talks|dog)). The Zipf distribution was compared to a flat distribution, with sentences of different lengths occurring equally often. In a 2×2 factorial design, we tested for effects of biased transitional probabilities (biased/non-biased) and the distribution of sequences with varying length (Zipf distribution/flat distribution) on implicit learning and explicit ratings of grammaticality. Preliminary results show that a Zipf-shaped and semantically biased input facilitates grammar learnability. Thus, this project contributes to understanding how we learn complex structures with long-distance dependencies: learning may be sensitive to the specific distributional properties of the linguistic input, mirroring meaningful aspects of the world and favoring short utterances.

Abstract

The neural infrastructure for sentence production and comprehension has been found to be mostly shared. The same regions are engaged during speaking and listening, with some differences in their loading depending on modality (Giglio et al., 2022). In this fMRI study (n=40), we investigated whether modality affects the connectivity between inferior frontal and temporal regions, previously found to be involved in syntactic processing across modalities, and with the cerebellum, which has been historically linked with motor aspects of production. Participants produced or listened to word sequences of increasing constituent size. We found that constituent size reliably increased the connectivity between the inferior frontal gyrus and the posterior temporal lobe in both modalities. Preliminary cerebellar results suggest that different sub-regions presented different patterns of connectivity. Connectivity between Lobule VI and (pre)motor regions was increased during production relative to comprehension. Connectivity between Crus I/II and fronto-temporal regions was instead increased as a function of constituent size, and in particular during production. These results thus show that the connectivity between fronto-temporal regions is upregulated for syntactic structure building in both sentence production and comprehension, while cortico-cerebellar connectivity is enhanced both in response to syntactic processing and during production.

Abstract

Language comprehension may be facilitated by prediction, where a listener’s eye-gaze moves towards a referent before it is mentioned if the noun is predictable. Anticipatory fixations reduce when speech contains disfluencies (hesitations/repairs). Changes to the pattern of anticipatory fixations could result from a change in prediction or an attention shift. We combined EEG and eye-tracking to study the prediction of language in naturalistic, virtual environments (experiment 1 & 2) and the influence of disfluencies on predicting language (experiment 2). Participants (n=32; preliminary n=19) listened to sentences spoken by a virtual agent in various virtual scenes (e.g., office, street) while participants’ eye-movements and EEG were recorded. Spoken sentences were predictable or unpredictable, based on the verb constraints and referents were visible or absent in the scene to be congruent or incongruent with listeners’ predictions, respectively. In experiment 2, sentences were additionally either fluent or disfluent with ahesitation (uhh). Increased processing, reflected in increased theta power, was greater either at the predictive verb onset or at unpredictable noun onset in fluent sentences, but was observed at both predictable and unpredictable noun onsets in disfluent sentences. Our findings provide preliminary evidence supporting that hesitations reduce the weight listeners place on their predictions.

Abstract

Effective communication involves recognizing the disparity between our own perspective and that of the recipient, influenced by factors such as stereotypes and accents. The extent to which Theory of Mind (ToM), the ability to ascribe beliefs to others, plays a role in this process is uncertain. We anticipate individuals to speak in line with their beliefs, but what if their words conflict with our expectations? To investigate, we devised a virtual perspective-taking experiment where we manipulated a virtual agent's beliefs. Electroencephalography data were collected as participants listened to statements from the agent that either aligned or clashed with their true or false beliefs. We focused on the N400, an event-related brain component linked to word unexpectedness. As hypothesized, statements inconsistent with the agent's true beliefs triggered more pronounced N400 responses compared to matching statements. Furthermore, we anticipated that when the agent held a false belief, this knowledge would factor into interpreting their statements. Neither statements aligned with nor those diverging from the agent's false beliefs evoked N400 responses. This can be taken as evidence that participants did take the agents perspective into account. These results strongly support the role of Theory of Mind in language comprehension.

Abstract

Human communication requires interlocutors to mutually understand each other. Previous research has suggested inter-brain synchrony as an important feature of social interaction, since it has been observed during joint attention, speech interactions and cooperative tasks. Nonetheless, it is still unknown whether inter-brain synchrony is actually related to successful face-to-face communication. Here, we use dual-EEG to study if inter-brain synchrony is modulated during episodes of successful and unsuccessful communication in clear and noisy communication settings. Dyads performed a tangram-based referential communication task with and without background noise, while both their EEG and audiovisual behavior was recorded. Other-initiated repairs were annotated in the audiovisual data and were used as indexes of unsuccessful and successful communication. More specifically, we compared inter-brain synchrony during episodes of miscommunication (repair initiations) and episodes of mutual understanding (repair solutions and acceptance phases) in the clear and the noise condition. We expect that when communication is successful, inter-brain synchrony will be stronger than when communication is unsuccessful, and we expect that these patterns will be most pronounced in the noise condition. Results are currently being analyzed and will be presented and discussed with respect to the inter-brain neural signatures underlying the process of mutual understanding in face-to-face conversation.

Abstract

Monitoring auditory feedback is important for fluent speech production as it enables correction of vocalization errors. Influence of auditory feedback is best illustrated by manipulating it during speech production. A common temporal manipulation technique is delaying auditory feedback (DAF), which disrupts speech fluency, and a common spectral manipulation technique is perturbing the pitch of auditory feedback, which elicits vocal changes. We aimed to understand whether there is a correlation between sensitivity to temporal versus spectral manipulations of auditory feedback. We collected data from 40 participants. In the DAF task, participants repeated sentences, and auditory feedback was presented with 0 or 200ms delay. In the pitch perturbation task, participants phonated the vowel /a/ for 4 seconds and pitch of the auditory feedback was shifted by ±100 or ±200 cents.Voice recordings were analyzed using LME models to test the effects of feedback manipulations. We found that DAF significantly prolonged articulation duration and increased both voice pitch and intensity. Additionaly we found that large pitch shifts elicited less compensatory responses. There was a large individual variability in sensitivity to feedback manipulations for both tasks, however there was no correlation between the sensitivity profiles between tasks, suggesting that these features are processed differently.

Abstract

Phonological word forms are maintained in long-term memory and rapidly accessed during speech. However, the neural mechanisms that support the acquisition, maintenance, and recollection of word-form memories remain unclear. Starting from the hypothesis that dendrites and cell assemblies are the neural substrate for associating phoneme sequences to words, the present study investigates the dynamics of lexical access in a biologically constrained network model and compares it to the computational principles of human spoken word recognition. The model is a spiking recurrent network of dendritic neurons with realistic physiology and connectivity. The network implements unsupervised plasticity in the excitatory and inhibitory synapses, resulting in stable associations between phonemic and word representations. The activity of word assemblies indicates that, first, lexical representations are activated incrementally; second, lexical neighbors delay correct lexical access with both cohort and rhyme groups reactivated; and third, partial phonemic mismatch degrades word recognition. We also derive the model's predictions on the lexical bias effect and show that it is consistent with the hypothesis of offline feedback, providing a plausible mechanism for perceptual learning. In conclusion, we show that the dendrites could be the missing link between the perceptual space of phonemes and the mental lexicon.