(R3) poster program

(1) Asymmetrical costs of changing speaking rate indicate 'gaits' of speech

Joe Rodd (1 2), Hans Rutger Bosker (1), Mirjam Ernestus (1 2), Louis ten Bosch (2), Antje S. Meyer (1 3)

1 Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

2 Centre for Language Studies (CLS), Radboud University, Nijmegen, The Netherlands

3 Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands

 

To support the use of speech rate as a cue in recognition, precise control of speaking rate in production is required, but how speakers select and maintain their speaking rate is underexplored. A recent model of speech production, EPONA, proposes that speakers adopt qualitatively different configurations of the speech formulation system to achieve different speaking rates, analogous to the qualitatively distinct gaits (walking, running) adopted in locomotion. Switching gait is proposed to be more effortful than rate modulation within a single gait. Simulations with EPONA demonstrated the presence of gaits of speech in a sample of fast, medium and slow speech, and ruled-out the possibility that speaking rate control was purely linear.

We experimentally tested for greater difficulty in switching between speaking rates when the gait boundary must be crossed. In a multiple picture naming task, speakers were required to speak at a pre-trained rate. During the trial, the required speaking rate changed. We quantified the speed of speaking rate change. We found that speakers were slower to shift between some pairs of equally separated speaking rates than others. We interpreted these slow shifts as involving a qualitative reconfiguration of the speech production system, consistent with gaits of speech.

 

(2) Speech tempo perception and deletion: Evidence from a listening experiment

Robert Lennon (1), Rachel Smith (2), Leendert Plug (1)

(1) University of Leeds

(2) University of Glasgow

 

Studies quantifying speech tempo tend to use one of various rate measures. The relationship between these measures and perceived tempo remains poorly understood. To improve our understanding of the impact of syllable and phone deletions on tempo perception, we assess how canonical and surface syllable and phone rates map to listeners' tempo ratings (Koreman 2006; Reinisch 2016). 55 English listeners rated 180 stretches of spontaneous speech (extracted from the DyViS corpus) for tempo; we modelled ratings for stimulus samples in which canonical rate varied and surface rate was near-constant, or vice versa.

When rating stimuli that were similar in either canonical or surface syllable rate and variable in the other, listeners systematically attended to the variation, as in Koreman (2006). For phone rate, by contrast, listeners ignored variation in canonical rate when surface rate was controlled. These results suggest that when estimating tempo, listeners orient to a canonical syllable string, based on the lexical content of the incoming signal - but not a fully specified canonical phone string. Alternatively, listeners may ignore phone deletions when estimating tempo because they know these can occur at all speech rates, whereas syllable deletions mainly occur in, and therefore strongly indicate, faster speech.

 

(3) Rate and rhythm in poetry: their effects on long term memory

Sara Andreetta (1), Vezha Boboeva (2), Alessandro Treves (1)

(1) SISSA, Trieste

(2) Imperial College London

 

Poetry is a linguistic expression characterized by a special “rhythm” provided by devices of meter, linked to phonology and prosody. The intuition that rhythmic structures can help information settle down in memory dates back to the prewriting era: in every culture we find songs and poems that have been transmitted for centuries just relying on memory.

In our study we investigated the role played by distinct meter devices in long term memory. We consider these devices as examples of schemata that, by creating regularities, facilitate the recall of verses. We chose passages from the “Divina Commedia”, which employs metrical patterns familiar to Italians, and manipulated rhyme, phrasal accent or number of syllables. We used non-words to eliminate semantic effects. Eighty-five participants were involved: first, by listening several times to the passages, they encoded and retained something in memory. The following day they were asked to remember a specific non-word per passage. Only minor differences emerged among conditions. Therefore, we are replicating the study using the “Orlando Furioso”, a poem characterized by a more rigid meter.

We hope that this study, particularly once extended to ERPs, will help understand how memory for poetry works from a neurocognitive point of view.

 

(4) The effect of exposure to a regular rhythmic sequence on language processing in 4- to 6- year-old French children

Nallet Caroline (1), Eniko Ladányi (1 2), Judit Gervain (1)

(1) The Institute of Neuroscience and Cognition - Université Paris Descartes

(2) Music Cognition Lab - Vanderbilt University

 

The theory of embedded neural oscillations in speech processing (Giraud & Poeppel, 2012) and several behavioral studies about the beneficial effect of rhythmic priming on language processing (Cason & Schön, 2012 ; Chern et al., 2018) suggest that exposure and entrainment to a regular rhythm may boost language abilities. To test this hypothesis, we carried out an exploratory study to investigate the effect of listening to a regular rhythmic sequence, on the performance of 4-6-year-old children in two language processing tasks : a sentence repetition task and a non-word repetition task.

For this purpose, 60 French children (30 4-5-year-old & 30 5-6 year-old) were divided into three different experimental conditions within each class : regular, irregular and silence. Children in the first two groups were exposed to a regular or an irregular metronome beat for two-minutes priming phase before the tasks as well as for the entire duration of the tasks. Children in the third group performed the tasks in silence.

Contrary to prediction, the rhythmic regularity of the regular condition did not have a significant effect on children’s performance compared to the other conditions, regardless of task and age. However, we observed a trend in favour of rhythmic regularity for the sentence repetition task in 4-5-year-old children. The lack of significant results will be discussed in the light of previous studies that investigated the effect of regular rhythmic stimulation on language skills with other procedures.

 

(5) Neural entrainment to native linguistic rhythm

Ege Ekin Özer, Silvana Silva Pereira, Nuria Sebastian Galles

Universitat Pompeu Fabra, Center for Brain and Cognition

 

Multiple brain oscillations at different frequencies segment speech signal into its hierarchical linguistic units. Among these, theta oscillations are associated with syllable processing. This framework confines syllables into an invariant unit, disregarding the fact that syllable complexity varies according to linguistic rhythm. Linguistic rhythm is the beat of speech signal that classifies world languages into syllable-, stress- and mora-timed languages. Abundant behavioral evidence shows that segmentation strategies depend on native linguistic rhythm. Given these differences, syllable processing by neural oscillations could differ based on one’s native linguistic rhythm. In this study, neural entrainment in theta frequency range, measured by cross-correlation between audio signals and EEG signals, is predicted to be different for one’s native linguistic rhythm as opposed to other rhythms. We ensured the effect of native language by performing an experiment on two samples: native speakers of Spanish and Catalan (syllable-timed), and native speakers of English (stress-timed). While participants listened to resynthesized sentences from languages of different rhythms, their neural responses were recorded. Using machine learning algorithms, we have been able to classify between native language and non-native languages. The preliminary results suggest that syllable processing in the brain is specialized to the phonological properties of native language.

 

(6) Cross-modal entrainment to retimed speech

Alistair Beith (1), Rachel Smith (2), Dale Barr (3)

(1) School of Psychology, University of Glasgow

(2) Glasgow University Laboratory of Phonetics, University of Glasgow

(3) Institute of Neuroscience and Psychology, University of Glasgow

 

Theories of neural entrainment suggest that the temporal structure of the acoustic speech signal is crucial in speech comprehension. However, this signal is complex and apparently aperiodic. This lack of periodicity may be attributed to the demands of speech production. Therefore, digitally altering the signal to introduce periodicity may offer a rhythmic enhancement that facilitates entrainment.

In this study, speech was presented to participants over headphones concurrently with a visual transcript. Musical quantization was used to increase periodicity in the global temporal structure of the signal. This was compared to randomly retimed speech with matched local temporal distortion and an unaltered control condition. For all three conditions, synchrony between eye-movements and the timing of the speech was measured. Both retimings resulted in lower levels of synchrony compared to control and did not significantly differ from each other.

While this is consistent with previous findings in relation to distortion of the local temporal structure of the signal, the lack of a significant difference between the two retimed conditions is surprising. This suggests that any benefits of enhanced periodicity are outweighed by the costs of temporal distortion. The novel research paradigm employed in this study may be adapted to investigate this further.

 

(7) Emotional prosody: the role of pitch in time

Pol van Rijn (1), Georgios Michalareas (1), David Poeppel (1 2), Pauline Larrouy-Maestri (1)

(1) Neuroscience Department, Max Planck for Empirical Aesthetics

(2) Psychology Department, New York University

 

Speech carries information about the emotional state of a speaker. Several studies underline the importance of fundamental frequency (F0) in emotion recognition. However, current measurements of F0 are mainly limited to summary statistics (e.g., mean) that only sparsely reflect changes in pitch over time and thus are a simplification of the signal. We therefore developed three kinds of measures that describe changes in pitch (58 new features such as shape and direction of the contour or pitch distribution), applied them to a corpus of emotional prosody (30 sentences, 4 speakers, 7 emotions) and examined their contribution to emotion classification. We used SVMs and random forests to assess the relevance of the features. The new features, in addition to a standard feature set (eGeMAPS), significantly improved the classification of emotions. Results confirm that mean level of pitch is important and show that slope and distribution measures play a substantial role. In particular, the shape of the contour of the last word in the sentence appears as a good predictor of emotion classification. Currently, we are applying the features to other corpora of emotional prosody to examine the stability of our findings.

 

(8) The role of talker-specific prosody in predictive speech perception      

Giulio G.A. Severijnen, Hans Rutger Bosker, Vitória Piai, James M. McQueen

Max Planck Institute for Psycholinguistics

Donders Centre for Cognition

 

In this EEG experiment, we investigated how listeners learn about variability in suprasegmental cues between talkers to recognize spoken words. Participants learned minimal stress pairs of non-words (e.g., BOLdep/bolDEP), spoken by two different talkers. Each talker only used one acoustic cue to signal lexical stress patterns (e.g., talker A only used F0 and talker B only used amplitude). This allowed participants to learn the non-words as well as, through perceptual learning, which cues are used by each talker. At test, participants heard constraining sentences, spoken by both talkers, containing these non-words in sentence-final position. In different conditions, the sentence-final word could either be produced using the correct cues (e.g., talker A using F0; control condition) or the incorrect cues (e.g., talker A using amplitude; cue-switch condition). If participants learned about the talker-specific cues, they would be able to predict upcoming talker-matching word forms (e.g., BOLdep cued using only F0). In the cue-switch condition, we expected that these sentences would create a mismatch between predicted and perceived word forms based on the talker-specific cues, eliciting a relatively larger N200 compared to the control condition. These results would illustrate talker-specific prediction of suprasegmental cues, picked up through perceptual learning on previous encounters.

 

(9) Neural correlates of implicit rhythm and rhyme processing in Dutch infants

Laura E. Hahn, Paula Fikkert, Tineke M. Snijders

Centre for Language Studies, Radboud University, Nijmegen, The Netherlands

International Max Planck Research School for Language Sciences, Nijmegen, The Netherlands

Centre for Language Studies, Radboud University, Nijmegen, The Netherlands

Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands

 

Rhythmic speech is ubiquitous in early childhood, as caregivers often rhyme and sing for their infants. Here we report on an EEG study on rhythm and rhyme processing in a sample of 42 Dutch ten month-olds. Infants were exposed to rhythmic or non-rhythmic Dutch nursery rhymes with rhyming or non-rhyming pseudowords at the end of every line. Rhythmic verses always had alternating strong-weak syllable patterns, while for the non-rhythmic condition a one-syllable change in the lyrics disturbed the strong-weak pattern. Our preliminary ERP results indicate the rhyme response to be altered by the rhythm of the stimulus. While we observed a sustained positivity for rhyme in rhythmic verses, a negativity was observed for the non-rhythmic verses. We hypothesize rhythmic speech to lead to more speech-brain coherence than less rhythmic speech. Hence we will extend our analysis with an investigation of cortical tracking of speech rhythm in infants, and its effect on rhyme sensitivity.

 

(10) Atypical processing of phonotactic probability and syllable stress in dyslexic adults: an MMN study

Alexandra K. Emmendorfer (1 2 3), Bernadette M. Jansma (1 2), Sonja A. Kotz (3), Milene Bonte (1 2)     

1 Dept of Cognitive Neuroscience, Maastricht University

2 Maastricht Brain Imaging Center, Maastricht University

3 Dept of Neuropsychology and Psychopharmacology, Maastricht University

 

Efficient sensory processing and skill acquisition are facilitated by our experience of regularities in the timing (temporal structure) and content (formal structure) of sensory events in our environment. To examine their relevance for successful reading, we tested adults with and without dyslexia in a passive oddball paradigm. We manipulated formal and temporal predictability in bisyllabic Dutch pseudowords, where deviants differed from the standard in terms of phonotactic probability (formal deviant) or syllable stress (temporal deviant). Normally reading adults (N = 24) exhibited facilitated processing of more probable structures for both phonotactic probability and syllable stress, indexed by shorter MMN peak latencies and larger peak amplitudes, respectively. Preliminary findings in dyslexic readers (N = 6) suggest atypical processing of both deviant types. For formal deviants, we observed no MMN sensitivity to phonotactic probability, replicating previous findings in children and adults with dyslexia. For temporal deviants, the directionality of MMN modulation appears reversed, with less probable stress patterns showing a larger amplitude, and a trend towards an interaction with phonotactic probability, suggesting that dyslexic readers may rely more heavily on phonotactic probability in cases where syllable stress is irregular. Our findings imply relevance of both formal and temporal processing for reading development.

 

(11) Rhyme and metre processing in nursery rhymes: Investigating ease of processing and effect of musical aptitude

Maartje J. Stroo, Laura E. Hahn & Tineke M. Snijders

Donders Graduate School, Radboud University, Nijmegen, The Netherlands

​Centre for Language Studies (CLS), Radboud University, Nijmegen, The Netherlands

Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands

 

Everyday language is filled with rhyme and metric patterns, for example in children’s stories, songs or nursery rhymes. Obermeier et al. (2016) investigated auditory rhyme and metre processing in German poetry by means of event-related potentials (ERP). The biphasic N400-P600 interaction effect they identified in the ERP suggested meter and rhyme facilitate processing of auditorily presented speech. In the current study we aim to replicate this finding in a Dutch naturalistic stimulus, as well as assess whether musical aptitude influences sensitivity to rhyme and meter. In a passive listening paradigm twenty-eight adult participants listened to metered and non-metered nursery rhymes, with rhyming or non-rhyming pseudowords at the end of each line. Preliminary ERP results revealed a rhyme effect on the N400 and rhyme and meter effects in the P600 time-window, but no interaction between rhyme and meter. The use of pseudowords as well as the absence of a behavioural task during listening might have caused the results to differ from those of Obermeier et al (2016). At the meeting we will report the results of the pre-registered statistical analyses on the effects of rhyme and meter on the ERP, as well as the effect of musical aptitude hereon.

 

(12) Cortical tracking of speech in 7.5-month-old infants

Tineke M. Snijders & Iris Schmits

Max Planck Institute for Psycholinguistics, Nijmegen

 

Young infants are very sensitive to the rhythm of speech (Nazzi et al., 1998). Neurons in the brain process information in a rhythmical way, and can take over the speech rhythm to focus on salient aspects of the input (Lakatos et al., 2008). This cortical tracking of speech might be a possible neural mechanism through which infants can effectively use prosodic cues for early language learning. In the current study cortical tracking was assessed in 7.5-month-old infants (N=108) in an EEG experiment, in which infants listened to stretches of speech (nursery rhymes), as well as to rhythmically regular and irregular trains of complex tones (beeps). The infant brains took over the rhythm of the regular beeps, as evidenced by a larger 2.5 Hz power over bilateral frontotemporal electrodes. Furthermore, speech-brain coherence was present in response to the nursery rhymes over temporal electrodes. This speech-brain coherence was right-lateralized for stress rate (~2-3 Hz), while it was left-lateralized for the syllable rate (~6 Hz). In future analyses individual differences in infants' neural sensitivity to rhythm will be related to speech segmentation ability at 9 months and vocabulary scores at 18, 24, and 31 months.

 

(13) Examining the link between rhythm and pitch in speech and rap

Steven Gilbers (1), Nienke Hoeksema (2), Kees de Bot (1), & Wander Lowie (1)

(1) Department of Applied Linguistics, Center for Language and Cognition Groningen, Research School of Behavioural and Cognitive Neurosciences; University of Groningen, the Netherlands

(2) Neurobiology of Language Department; Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands

 

This study investigates the connection between rhythm and melody in language and music in the context of African American English (AAE) and hip-hop culture, approaching prosody from both a productional and perceptual perspective.

Looking at the speech and rap music from 16 African American rap artists from the United States’ West and East Coasts, we examine how West and East Coast AAE differ regarding prosodic rhythm and pitch, how West and East Coast rap styles (‘flows’) are distinct in terms of rhythm and pitch, and whether the patterns of regional variation observed in both domains correspond to each other. We also report on an AAE accent perception survey in which 306 participants were asked to identify whether speakers were from the West or East Coast on the basis of their accent and which accent features stood out to them the most when forming their decisions.

The results show that East Coast AAE is more isochronous and monotone than West Coast AAE. Correspondingly, East Coast rap flows were found to be less rhythmically diverse and less melodic. Moreover, prosodic cues – especially relating to rhythm – were found to be highly salient to listeners when trying to determine a speaker’s coast of origin.

 

(14) Cross-linguistic variation in local and global speech rate

Ludger Paschen (1) , Matthew Stave (2), Manfred Krifka (1), François Pellegrino (2), Susanne Fuchs (1), Frank Seifart (1)

(1) Leibniz-Zentrum Allgemeine Sprachwissenschaft

(2) Laboratoire Dynamique du Langage UMR5596, CNRS, University of Lyon

 

Cross-linguistic variation in local and global speech rate is still largely understudied. We present the development of a polyfunctional language DOcumentation REference COrpus (DoReCo) that will enable comparative research into spontaneous speech on a diverse sample of at least 50 languages. We explore this resource regarding cross-linguistic variation in two areas: 1) the (in)compressibility of certain segments (e.g., voiced vs. voiceless consonants) as affecting local speech rate (Fletcher 2010); and 2) potentially universal “attractor states” for information rate, affecting (syllable-based) global speech rate (Coupé et al. 2019). The typological diversity of the DoReCo corpora makes it possible to explore how languages manifest variation within universal patterns, based on language-specific characteristics, such as phoneme inventory or word complexity. By including diverse speaker populations, DoReCo aims at providing a more realistic window into the neuro-cognitive and physiological-articulatory bases of human language than studies focusing on individual or major languages, which are often not representative of human behavior (Henrich et al. 2010). DoReCo is designed to become a platform for easy access to over one million words of transcribed, translated, and time-aligned corpus data, thus contributing to open, reproducible science regarding global linguistic diversity and cultural heritage.

 

(15) Modelling the relationship between vision and auditory in the lexical decision

Li Lei

Centre for Language Studies, Radboud University, Nijmegen, The Netherlands

 

Reaction times (RTs) are frequently used as a direct measurement to observe human (visual or spoken) word recognition. They are difficult to interpret because RTs reflect many different processing mechanisms. Recently, studies such as that conducted by Ten Bosch et al. (2014, 2018) have shown that the removal of local speed effects, such as attention and fatigue, can increase correlations between human participants' RTs because filtered RTs are closer to a “real” description than raw RTs.

Previous studies have reported that once children have begun to receive education, they are no longer able to completely separate phonological and orthographic knowledge because the knowledge can be closely integrated within human’s mental lexicon (Tanenhaus, Flanigan & Seidenberg, 1980; Ziegler & Ferrand, 1998). Besides, a recent study by Huettig and Pickering (2019) indicated that literacy enhances people’s ability to predict spoken language. By these accounts, the orthography might be seen as a strong predictor for auditory lexical decision.

In this study, a linear mixed-effects model would be used to investigate whether RTs in the visual lexical decision could be as a strong predictor for auditory RTs.