Hans Rutger Bosker receives ERC grant to study audiovisual communication

10 January 2022
HR Bosker ERc 2022
A European Research Council (ERC) Starting Grant has been awarded to Hans Rutger Bosker, senior investigator at the Psychology of Language Department at the Max Planck Institute for Psycholinguistics in Nijmegen. Bosker studies how people speak and listen in everyday conversations, and what role visual information plays. Does what we see influence what we hear?

Hans Rutger Bosker and his TEMPOS research group investigate human speech production and perception. His ERC project ‘HearingHands’ focuses on audiovisual communication. In face-to-face conversation, we do not only produce sound. We also produce all sorts of visual communicative cues, including hand gestures. These hand gestures are carefully timed in close synchrony with the speech. However, it remains unclear what this precise timing contributes to speech perception.

In the HearingHands project, Bosker and his team aim to demonstrate that the timing of even the simplest up-and-down hand movements serves as a form of multimodal prosody, with the potential to even guide what words you hear. The researchers will record gesture-speech synchrony in a range of different languages. In addition, they will assess how the human brain integrates visual (gestures) and auditory (speech) input. They will also test the usefulness of enriching virtual avatars with carefully timed gestures to enhance speech intelligibility.

The ERC Starting Grant awards up to €1.5 million to talented early-career researchers for a period of five years.

Share this page