Micha Heilbron joins MPI as Independent Research Group Leader

In his new role, Heilbron will investigate how insights from modern neural network language models can help us better understand human language processing. “Our goal is to understand how people process language by studying neural network language models,” he explains. “At the same time, we want to turn that around and use what we know about human language to build better language models.”

Heilbron completed his PhD under the supervision of Peter Hagoort at the Donders Institute for Brain, Cognition and Behaviour and MPI. His doctoral research focused on prediction in language processing: the idea that humans continuously anticipate upcoming words while listening or reading.
 

Studying human language through LLMs

During his PhD, the release of GPT-2 proved to be a turning point in his research. Recognising that large language models provided a working computational implementation of predictive language processing, Heilbron shifted his focus to studying human language through these models. That line of work became the most influential part of his PhD research and ultimately led to funding from the Max Planck Society to establish his new research group. The same technological developments later evolved into systems such as ChatGPT.

Reflecting on that decision, Heilbron notes that it illustrates how unpredictable scientific research can be: “When opportunities arise, it’s important to be flexible and follow them.”

Heilbron has long been fascinated by fundamental questions about language and meaning. Computational modelling, he says, allows researchers to explore these questions in a rigorous way. “It’s a bit like doing philosophy with numbers: you can work on big abstract questions about meaning or the origin of knowledge while producing precise, quantitative answers.”
 

Transforming the study of language

The opportunity to pursue these questions is what attracted him to MPI. “An entire institute dedicated to understanding language in the human mind makes it an amazing and unique place to do this type of research.”

Heilbron joins the institute at a time when advances in AI are transforming the study of language. “For decades, language seemed uniquely human, something computers struggled with. Now AI systems can process language fluently, and we are only beginning to understand what that means, both for artificial systems and for our theories of human language.”