TEMPoral Organization of Speech (TEMPOS)

Members

PI

Dr. Hans Rutger Bosker

PhD students

Ronny Bujok
Giulio Severijnen (Donders Institute)
Sophie Slaats
Orhun Ulusahin

Interns

Ivy Mok (Radboud University)
Eleni Zimianiti (Utrecht University)

As of September 2022, the TEMPOS group will transition into the 'Speech Perception in Audiovisual Communication' (SPEAC) group at the Donders Institute, Radboud University, Nijmegen. Link: https://hrbosker.github.io/

Vision

How is it possible that we can have a proper conversation with someone even if that someone is talking very fast, has a strange accent, produces uhm’s all the time, or has to shout over several other talkers in a noisy café?

Having a simple conversation often seems rather easy, yet Google, Siri, and Alexa still haven’t quite mastered it. Somehow, our brain is uniquely equipped to successfully perceive the speech of those around us – even in quite challenging listening environments. In this research group, we investigate the psychological and neurobiological mechanisms that underlie the exceptional human behavior of spoken communication. We specifically focus on the TEMPoral Organization of Speech (TEMPOS) in both the auditory and the visual modality (e.g., lips, face, hands), including such aspects as speech rate, speech rhythm, prosody, amplitude modulations, and neural oscillations.

The big questions

The work we do as part of the TEMPOS group contributes to a better understanding of how spoken communication can usually take place so smoothly. For instance, how do listeners manage to understand talkers in challenging listening conditions, such as when there are competing talkers around? What is the contribution of multimodal cues (e.g., lip movements, hand gestures) to speech perception? How do listeners ‘tune in’ to a particular talker with his or her own peculiar pronunciation habits? What is the role of context (i.e., acoustic, semantic, and situational context) in speech processing? Finally, we also develop methodological tools to facilitate research in the speech sciences.

Research projects

Some examples of ongoing research projects are:

•    Do listeners keep track of how different talkers vary in how they produce speech prosody?
Hans Rutger Bosker, Giuseppe Di Dona, James McQueen, Giulio Severijnen

•    What is the role of selective attention when listening to competing talkers (e.g., ‘cocktail party’ listening)?
Hans Rutger Bosker, Rebecca Frost, Christina Papoutsi, Eleni Zimianiti

•    How do multimodal cues, such as simple hand gestures, influence which speech sounds we hear?
Hans Rutger Bosker, Ronny Bujok, Ivy Mok, David Peeters, Wim Pouw

•    How do we 'tune in' to familiar talkers in both speech perception and production?
Hans Rutger Bosker, James McQueen, Orhun Ulusahin

•    What are the psychological and neurobiological mechanisms underlying how listeners normalize speech sounds for different speech rates?
Hans Rutger Bosker, Chris Heffner, Lars Riecke

 

How do we conduct our research?

To study speech production, we use speech elicitation paradigms, such as (multiple) picture naming, shadowing, reading out loud, Lombard tests, etc. To study speech perception, we use speech categorization experiments with manipulated speech signals (what’s this word?), speech-in-noise intelligibility experiments (what’s this sentence?), and various psycholinguistic paradigms such as repetition priming (e.g., lexical decision). We also use eye-tracking (visual world paradigm) to study the time-course of speech processing. Finally, we use neuroimaging methods (EEG, MEG, tACS) to uncover the neurobiological mechanisms involved in the temporal decoding of speech, with a particular focus on oscillatory dynamics.

 

Research output

For publications, see our scientific output.

For published research data, see the various projects in this OSF profile.

For a research example, see this clip:

In this clip, you'll see two videos from Experiment 1B in Bosker & Peeters (2021). In both videos, a talker pronounces the Dutch sentence: "Nu zeg ik het woord [pla.to]" = "Now say I the word [pla.to]". Importantly, the two videos have exactly the same audio, but the first video has a beat gesture on the first syllable /pla/, while the second video has the beat gesture on the second syllable /to/. Our research showed that the timing of the beat gesture can decide whether you hear "PLAto" (the famous Greek philosopher) or "plaTEAU" (a highland). Try it yourself!

See also https://www.youtube.com/watch?v=cW7oBD0UmMU and https://osf.io/b7kue/.

 

Practical research tools that have been developed by TEMPOS members are:

• TSR score: Token Sort Ratio, an automated metric for assessing transcript accuracy in speech intelligibility studies. See: tokensortratio.netlify.app.
  • Bosker, H. R. (2021). Using fuzzy string matching for automated assessment of listener transcripts in speech intelligibility studies. Behavior Research Methods, 53, 1945-1953. doi:10.3758/s13428-021-01542-4.

• POnSS: a tool for efficient and accurate segmentation of speech data. See: https://git.io/Jexj3.
  • Rodd. J., Decuyper, C., Bosker, H. R., & ten Bosch, L. (2021). A tool for efficient and accurate segmentation of speech data: Announcing POnSS. Behavior Research Methods 53, 744-756. doi:10.3758/s13428-020-01449-6.

 

Publicly available speech corpora developed by TEMPOS members are:

• PiNCeR: a corpus of cued-rate multiple picture naming in Dutch. See: doi:10.31234/osf.io/wyc6h.
  • Rodd, J., Bosker, H. R., Ernestus, M., Ten Bosch, L., & Meyer, A. S. (2019). PiNCeR: a corpus of cued-rate multiple picture naming in Dutch. PsyArXiv. doi:10.31234/osf.io/wyc6h.

• NiCLS: the Nijmegen Corpus of Lombard Speech, containing Dutch Lombard speech and matching plain speech. See: https://hdl.handle.net/1839/21ee5744-b5dc-4eed-9693-c37e871cdaf6.
  • Bosker, H. R., & Cooke, M. (2020). Enhanced amplitude modulations contribute to the Lombard intelligibility benefit: Evidence from the Nijmegen Corpus of Lombard Speech. The Journal of the Acoustical Society of America, 147 (2): 721-730. doi:10.1121/10.0000646.

 

Collaborators

Rebecca Frost (Edge Hill University)
Chris Heffner (University at Buffalo)
Marieke Hoetjes (Radboud University)
Lieke van Maastricht (Radboud University)
James McQueen (Radboud University)
Jonathan Peelle (Washington University in Saint Louis, USA)
David Peeters (Tilburg University)
Wim Pouw (Radboud University)
Lars Riecke (Maastricht University)

Former members

Graduated PhDs:
   Merel Maslowski (2019), "Fast speech can sound slow: effects of contextual speech rate on word recognition", fulltext.
   Joe Rodd (2020), "How speaking fast is like running: modelling control of speaking rate", fulltext.
   Greta Kaufeld (2021), "Investigating spoken language comprehension as perceptual inference", fulltext.

Interns:
   Momo Yamamura, Rik Does, Wibke Naumann, Christina Papoutsi, Anna Ravenschlag, Jeonga Kim, Marjolein Van Os, Marie Stadtbäumer, Rebecca Wogan, Vilde Reksnes, Myriam Weiss, Nora Kennis.