What was the main question in your dissertation?

When communicating with others, we gesture all the time. Beat gestures are special kinds of gestures that are very well coordinated with the timing of our speech. In my dissertation, I aimed to understand how simple and rhythmic gestures affect the way people perceive the accompanying speech. Across various chapters, I investigated the influence of beat gestures on speech perception using different experimental designs to understand the extent of these effects and the cognitive processes underlying them. 

 

Can you explain the (theoretical) background a bit more?

Gestures are so closely linked to speech that we even gesture when we are on the phone, even though no one can see us. Most gestures we make (around 80%) are simple up-and-down or side-to-side movements of the hand. These are called beat gestures. Beat gestures are important to study because they are so common, yet we know relatively little about them, especially compared to less frequent types of gesture, such as iconic gestures that depict their meaning visually (e.g., holding an imaginary cup and moving your hand to your lips to indicate drinking). Additionally, beat gestures are also relevant because they are linked to speech rhythm and intonation, which is the way our voice rises and falls as we talk. Beat gestures can even change the word you perceive because their timing with speech can highlight specific syllables. For example, in one study I conducted, participants heard “CONtent” (the noun) when they saw a beat gesture aligned with the first syllable. But participants heard “conTENT” (adjective) when they heard the same recording but with a beat gesture aligned to the second syllable.

Why is it important to answer this question?

How and to what extent beat gestures affect speech perception is theoretically interesting because it sheds light on the multimodal nature of communication. That is, communication is not only auditory but also visual. Studying beat gestures specifically is interesting because they are inherently meaningless on their own. Only in combination with speech, they become meaningful, making them suitable for studying how speech and gestures are multimodally integrated in communication. Finally, understanding the role of beat gestures in speech perception may help us understand when people struggle in communication (e.g., in some clinical populations) and how to support them. 

Can you tell us about one particular project?

In one of my studies, I presented participants with sentences such as  “The parents bought a crib for the baby”. Participants watched videos of a speaker producing such sentences and had to decide whether the final word of each sentence (i.e., baby) was a real word. The speaker in the video either produced a beat gesture on the first syllable, the second syllable, or neither syllable of the final word. I found that participants responded faster when the word was produced together with a beat gesture. The beat gesture did not add additional information to the speech because the words were common and unambiguous. The final target word was easy to predict based on the previous information in the sentence. And yet, I found that even in these ideal conditions, beat gestures can still help a listener speed up word recognition. 

What inspired you to choose your research topic?

I think what inspired me most to study this topic was its relatability. It is not a very abstract concept, but something that everybody knows. When you start to talk about gestures and speech, everyone, even non-academics, knows what you are talking about. Everybody has their own ideas and observations. So, not only do you start to notice gestures everywhere, but the topic also comes up frequently and naturally when talking with others. This not only generates many interesting research ideas but also seeing the excitement in others is very motivating to investigate the research topic. 

Can you share a moment of significant challenge or failure during your PhD journey and how you overcame it?

My PhD started in the middle of the COVID pandemic. Therefore, it was very difficult to test participants, and we had to adapt our research plans multiple times, for example, by moving recruitment online. Every change resulted in delays, frustration, and uncertainty. Partly, I overcame these challenges by staying flexible, optimistic, and resilient. But more importantly, I was very fortunate to work together with great colleagues and supportive supervisors who helped whenever they could. 

What was the most rewarding or memorable moment during your PhD journey?

My very first manuscript was rejected multiple times, even after I ran additional experiments to address a critical reviewer's concerns. But the manuscript kept coming back. Even after submitting to a different journal, the manuscript (reviewed by the same critical reviewer) returned with reviews that made it seemingly impossible to publish. Especially at the beginning of your career, this is very discouraging. However, when I submitted the manuscript to yet another journal, it came back with minor revisions, which felt incredible after such a long struggle. It also showed that my work was worth publishing and that other researchers appreciated the effort I put into it.  

What do you want to do next?

I would like to pursue a career in research. Currently, I’m wrapping up my first postdoc at the Centre for Language Science at Radboud University, where I am developing a tool for language learners to get feedback on their L2 English prosody. Next, I already have another postdoc lined up, where I will continue my PhD work on beat gestures. During the postdoc, I aim to further understand the effect of beat gestures on speech perception in other languages. Moreover, I would like to extend this line of research by further understanding the synchrony and variability of beat gestures in production.  

Credits

• Interviewer: Bahar Tarakçı

• Interviewee: Ronny Bujok

• Dutch translation: Lieke Herraets

• German translation: Anna Serke