MPI Colloquium Prof. Jordan Zlatev

Title:

Analyzing polysemiosis: Two case studies of combining language, gesture, and depiction with sand drawing

Abstract:

Human communication is by default polysemiotic: it involves the spontaneous combination of two or more semiotic systems, the most important ones in human cultures being language, gesture and depiction. First, I will formulate a cognitive-semiotic framework for the analysis of polysemiosis, contrasting this with more familiar (though ambiguous) systems based on the ambiguous term “multimodality”. Then I will describe a coding system for the analysis of polysemiotic utterances containing speech, gesture and drawing, and implemented in the ELAN video annotation software. This was developed by our research group as we analyzed video-recordings of sand drawing performances on Paama, Vanuatu and Pitjantjatjara sand stories in Central Australia. Methodologically we followed the conceptual-empirical loop of cognitive semiotics: our theoretical framework guided general considerations, such as distinguishing between the “tiers” of gesture and depiction, and the three kinds of semiotic grounds (iconic, indexical, symbolic), but the precise decisions on how to operationalize these were made only after extensive work with the material. I will describe the coding system and provide illustrative examples from the Paamese and Pitjantjatjara data, remarking on both similarities and differences in the polysemiosis of the two cultural practices.

If you would like to meet with Jordan Zlatev, please contact Limor.Raviv [at] mpi.nl