Keynote lecture on multimodal language and cognition

GENEA Workshop 2025 is held in conjunction with ACM Multimedia 2025 in Dublin, Ireland. Her talk, titled ‘What insights can human cognition offer to enhance computational models of multimodal language generation and evaluation?’, addressed how research on human multimodal communication can inform the development of computational models for gesture and speech generation.

Aslı discussed current challenges in creating AI systems that generate contextually appropriate gestures alongside speech. She argued that incorporating insights from psycholinguistics - such as how people use gestures depending on visual processing, communicative goals, and interactional context - can help make computational models more accurate and responsive. The lecture also considered how findings from cognitive and neuroscientific studies can improve methods for evaluating these generative systems.

Recommended reading: Aslı Ozyurek (2024). A vision for the adaptive and multimodal nature of language. Topics in Cognitive Science. https://onlinelibrary.wiley.com/doi/10.1111/tops.12728