MPI colloquium: Dr. Micha Heilbron

Title

Language models for cognitive neuroscience: Two success stories and a warning

Abstract

Large Language Models (LLMs) are not only the most performant AI models for tasks expressed in language but also at predicting brain responses to language, despite not being designed with the brain in mind. This surprising alignment opens two parallel research directions. The first investigates why these brain-agnostic models can capture brain responses so well, and what it may teach us about the brain. The second explores how we can actively build cognitive principles into LLMs allowing to test cognitive hypotheses and potentially create more brain-like models. In this talk I present new work on both fronts. Results highlight that (1) high-level syntactic abstractions are a key driver of LLM-brain alignment (2) human-like fleeting memory improves language learning and brain-alignment in transformer-based language models and (3) a recent, widely-cited neural signature of next-word prediction in the brain can be fully explained by stimulus correlations, rather than next-word prediction. Together these findings show how cognitive science can both illuminate and improve language models – and provide a cautionary tale about the difficulty but vital importance of interpretation in “NeuroAI”.