MLD at MMSYM 2026: Talks, Posters, Workshop and Summer School
Researchers from the Multimodal Language Department (MLD) took part in the Symposium Series on Multimodal Communication (MMSYM) 2026, held in Leuven, Belgium, from 9–11 September 2026.
MLD researchers contributed to MMSYM 2026 through talks, poster presentations, the SWiFT-MLP workshop, and the Pre-MMSYM Eye Tracking Summer School, showcasing work across gesture, prosody, facial signals, sign language learning, memory, multimodal communication, and computational methods.
Talks
Ezgi Mamus: Greater Spatial Precision in the Gesture Kinematics of Blind Versus Sighted Speakers
Ezgi Mamus presented “Greater Spatial Precision in the Gesture Kinematics of Blind Versus Sighted Speakers.” In her presentation, Mamus highlighted that blindness can enhance gesture precision, showing how our sensorimotor experiences shape how we communicate with our gestures. Using computer vision technologies, the study examines whether gesture kinematics are shaped by visuospatial experience.
The researchers assessed the gestures’ alignment with actual motion paths, along with size and duration. Blind participants produced gestures that were more precisely aligned with the motion trajectories, as well as larger and longer. These findings show that altered spatial cognition in blindness enhances gesture precision. The study underscores the role of sensorimotor experience in shaping gesture form.
Martina Mellana: Gesture-Form Similarity in Story Retelling
Martina Mellana presented a talk entitled “Gesture-Form Similarity in Story Retelling: A Comparison between Older Adults with and without Memory Impairment.”
The talk was based on a collaborative project between the Max Planck Institute for Psycholinguistics and Vanderbilt University, involving Martina Mellana, Sarah Brown-Schmidt, Melissa C. Duff, Aslı Özyürek, and Sharice Clough.
The study investigates how older adults with and without neurogenic memory impairment, including individuals with mild cognitive impairment (MCI) and Alzheimer’s disease (AD), differ in the extent to which their gesture production is influenced by previously observed gestures during narrative retelling. In particular, the study focuses on gesture-form similarity as a potential marker of gestural priming.
Sharice Clough: Quantifying Gesture and Sign Space Using Video-Based Motion Tracking
Sharice Clough presented a talk entitled “Quantifying gesture and sign space using video-based motion tracking: A case series tutorial.”
Drawing on three studies conducted within the Multimodal Language Department, the talk demonstrated how video-based motion tracking can be used to systematically quantify gesture and signing space.
The studies used a range of measures, including maximum wrist height, total distance travelled, angle deviation, 2D surface area, 3D volume, and gesture density maps. The work demonstrates how video-based motion tracking can provide detailed and systematic measures of spatial behaviour, opening new possibilities for research in psycholinguistics and cognitive science.
Poster Presentations
Loïs Dona: How Do Facial Signals Contribute to the Emergence of a Communication System?
Loïs Dona presented a poster titled “How do facial signals contribute to the emergence of a communication system?”
The poster presented findings from a language evolution communication-game experiment investigating how eyebrow movements and smiles influence the creation of shared communication systems between pairs of participants.
The study examined how facial signals contribute to interaction as participants developed and coordinated novel ways of communicating. The findings provide further insight into the role of visible facial behaviour in the emergence and coordination of shared communication systems.
Esam Ghaleb and Sho Akamine: Iconicity and the Learned Sign Space
Esam Ghaleb and Sho Akamine presented the poster “Iconicity systematically shapes the structure of the learned sign space: Novel evidence from sign embeddings.”
The work, by Dilay Z. Karadöller, Aslı Özyürek, Sho Akamine, Esam Ghaleb, and
Gerardo Ortega, investigated how hearing non-signers learn signs in Dutch Sign Language (NGT) and how iconicity influences the way these signs are learned and produced.
Using machine-learning-based sign embeddings to capture kinematic patterns in learners’ sign productions, the researchers found that iconic signs were learned more accurately, retained better over time, and showed greater consistency across learners than arbitrary signs.
The findings provide new evidence that iconicity systematically shapes the structure of learned sign representations and demonstrate how computational methods can offer new insights into sign language learning.
Beyza Nur Çay: Prosody, Gesture and Information Status in Turkish
Beyza Nur Çay presented a poster titled “Prosodic Prominence and Co-Speech Gesture as Cues to Information Status in Turkish.”
The poster presented ongoing work from her PhD project investigating how prosodic prominence and co-speech gesture are used in relation to new and given information in Turkish.
Using spontaneous narrative data, the study examines whether these cues tend to occur together when speakers introduce new information and how prosody and gesture jointly contribute to the way information is structured during communication.
The work contributes to a better understanding of how speakers combine different communicative cues across speech and gesture when organising and conveying information.
Dilay Z. Karadöller: Supporting Turkish Sign Language Learning in Deaf Children
Dilay Z. Karadöller presented the poster “An Intervention Program: Helping Deaf Children with Hearing Parents in Acquiring Turkish Sign Language” at the MMYSM symposium.
The work, by and Hilal Özdemir, Maharrem Ayar, Dilay Z. Karadöller examined whether an 11 week vocabulary intervention could support the learning and retention of Turkish Sign Language (TİD) signs among deaf children with hearing parents. The study also investigated how iconicity and phonological complexity influenced learning.
Using a research-developed web application on tablets, the children completed weekly learning and testing activities. The intervention improved sign recognition over time, while more iconic signs were learned more accurately and retained better. Phonological complexity affected performance during initial learning under some conditions but showed no association with longer-term retention.
The findings highlight the value of structured, app-supported interventions for expanding deaf children’s access to sign language vocabulary and show that iconicity can provide lasting support for lexical learning.
Link to the poster
Pre-MMSYM Eye Tracking Summer School
Martina Mellana
Martina Mellana also participated in the intensive three-day Pre-MMSYM Eye Tracking Summer School, held in the run-up to the MMSYM conference and organised by Geert Brône, Bert Oben, Eva Puimège, and Mihai Bâce from KU Leuven. The summer school provided both theoretical training and hands-on experience with mobile and remote eye tracking. As part of the programme, Martina completed a group project using remote eye tracking, which ended with a group presentation on the first day of MMSYM 2026.
The group project involved the full research process: conceptualising a research question, building the experiment in the dedicated software, collecting data, and carrying out the data analysis.
“The project concluded with a presentation on the first day of the MMSYM 2026 conference, where we presented our study to the conference audience. Our project investigated how visual attention to characters’ faces in short videos was influenced by different task instructions (focusing on emotions versus events) and by the presence or absence of audio, which could provide additional prosodic information about the characters’ emotional states.”
Together, these contributions highlight the range of research presented by MLD members at MMSYM 2026, spanning gesture and sign kinematics, facial communication, prosody, information structure, sign learning, memory, and eye tracking.
Nesrin Bulut
During the three-day Pre-MMSYM Eye Tracking Summer School at KU Leuven, Nesrin Bulut took part in theoretical and practical training on eye-tracking methods. Organised by Geert Brône, Bert Oben, Eva Puimège, and Mihai Bâce, the programme combined methodological sessions with hands-on activities using both mobile and remote eye tracking.
As part of the summer school, Nesrin collaborated on a group research project based on mobile eye-tracking data from naturalistic interactions.
“For the group project, we worked with mobile eye-tracking data from naturalistic interactions and focused on the relationship between pointing behaviour, gaze, and joint attention. With guidance from Geert Brône, Bert Oben, and Julie, we developed a research question and worked together on selecting relevant measures, analysing the data, and interpreting the observed interaction patterns.”
The group presented their project on the first day of MMSYM 2026, where they discussed their approach and findings with the conference audience.
Reflecting on the experience, Nesrin noted:
“Working directly with mobile eye-tracking data was a particularly useful opportunity for me to learn more about studying gaze and multimodal behaviour in naturalistic social interaction.”
Also at MMSYM 2026
Esam Ghaleb and Sho Akamine organised and led the hands-on SWiFT-MLP workshop: Semi-Automated Workflows for Facilitating Multimodal Language Processing on 8 September, as part of MMSYM 2026.
Read more about the SWiFT-MLP workshop in our separate news article below.
Share this page