Multimodal reference and social cognition across languages
Four-year fully funded PhD position in the Multimodal Language Department.
Doctoral supervisors: Prof. Asli Özyürek (MPI, Radboud University), Dr. Paula Rubio-Fernández (MPI), Assoc. Prof. David Peeters (Tilburg University).
Reference is one of the most basic functions of language and therefore, all languages in the world have reference systems that allow conversational partners to coordinate on an intended referent. Demonstratives, for example words like ‘this’ and ‘that’ in English, are a universal reference system that is used together with pointing to orient the addressee’s attention towards an element in the speech situation. Because of their fundamental function to establish joint attention, demonstratives are linked to human social cognition. Definite articles and pronouns, on the other hand, are used to signal referents that are familiar to the addressee (e.g., ‘We finally bought the house, but it was really expensive’). Therefore, these reference systems also rely on social cognition. This research project will investigate referential communication from a cross-linguistic perspective, adopting an integrated production and comprehension perspective and a multimodal approach that integrates speech, gesture and attention to understand which aspects of social cognition they tap into an whether and how this varies cross-linguistically.
The main questions to be addressed in the project are i) how do speakers of different languages integrate the use of reference systems and pointing gestures when referring to objects in different locations, ii) how do speakers of different languages monitor joint attention when producing/interpreting referential expressions. While the project does not aim to study any specific language, it will investigate potential cross-linguistic differences. For example, both Portuguese and Turkish have a 3-way demonstrative system that distinguishes proximal, medial and distal forms, but the Turkish medial demonstrative ‘şu’ is often used together with pointing to redirect the listener’s attention to the correct referent – unlike the medial form in Portuguese.
The above questions will be investigated using different methodologies. A corpus of naturalistic dialogue interactions will be compiled as a basis for analyzing the use of reference systems in speech as well as concurrent gestural and eye gaze behavior, for instance via kinematic and mobile eye-tracking data. This corpus will also serve as a basis for development of targeted AR and VR studies that experimentally test specific theoretical claims in mobile setups that combine experimental control with ecological validity.
The project will be led by an interdisciplinary team including Prof. Asli Özyürek
(https://www.mpi.nl/people/ozyurek-asli), an expert in human cross-linguistic multimodal language use and processing at the Multimodal Language Department of the Max Planck Institute for Psycholinguistics and Dr. Paula Rubio-Fernández (https://www.mpi.nl/department/multimodal-language-department/23/people) an expert in experimental cross-linguistic pragmatics and social cognition, and Dr. David Peeters
(https://research.tilburguniversity.edu/en/persons/david-peeters/), an expert in the study of multimodal communication using experimental methods, including virtual reality.
If you have questions about the position that you wish to discuss before you apply, please email the Department Director Prof. Asli Özyürek at asli.ozyurek [at] mpi.nl.
Candidates should have:
To apply please use our application portal here.
The deadline for application is November 1, 2023, but can be extended until a suitable candidate is found.
Short-listed candidates will be invited to participate, shortly after the deadline, in an online or in-person interview.
Applications should be in .pdf format and should include:
The Max Planck Institute (MPI) for Psycholinguistics is a world-leading research institute devoted to interdisciplinary studies of the science of language and communication, including departments on genetics, psychology, development, neurobiology and multimodality of these fundamental human abilities.
We investigate how children and adults acquire their language(s), how speaking and listening happen in real time, how the brain processes language, how the human genome contributes to building a language-ready brain, how multiple modalities (as in speech, gesture and sign) shape language and its use in diverse languages and how language is related to cognition and culture, and shaped by evolution.
We are part of the Max Planck Society, an independent non-governmental association of German-funded research institutes dedicated to fundamental research in the natural sciences, life sciences, social sciences, and the humanities.
The Max Planck Society is an equal opportunities employer. We recognize the positive value of diversity and inclusion, promote equity and challenge discrimination. We aim to provide a working environment with room for differences, where everyone feels a sense of belonging. Therefore, we welcome applications from all suitably qualified candidates.
Our institute is situated on the campus of the Radboud University and has close collaborative links with the Donders Institute for Brain, Cognition and Behavior and the Centre for Language Studies at the Radboud University. We also work closely with other child development researchers as part of the Baby & Child Research Center.
Staff and students at the MPI have access to state-of-the-art research and training facilities.
The Multimodal Language Department in particular aims to understand the cognitive and social foundations of the human ability for language and its evolution by focusing on its multimodal aspect and crosslinguistic diversity. The research at the department combines multiple methods including corpus and computational linguistics, psycho- and neuro-linguistics, machine learning, AI and virtual reality, and is concerned with various populations ranging from speakers of signed and spoken languages, young and older subjects from typical and atypical populations. The department provides opportunities for training in a range of linguistic, conversational state of the art multimodal language analysis (such as motion capture and automatic speech recognition), as well as neuropsychological, psychological methods related to multimodal language and frequent research and public engagement meetings, and support from an excellent team of researchers in linguistics and psycholinguistics.