Research in the MaRCS project

is divided in two complementary subprojects,called "Speech" and "Speech+". In the Speech subproject, we focus on every detail of the speech-comprehension process to elucidate the inner workings of speech comprehension. This subproject leads to small-scale models of how speech comprehension might work.

In the other subproject, Speech+, we put these models to the test in various large-scale domains, such as spontaneous conversation or speech comprehension in the elderly.

Speech

This subproject projects focuses on the inner workings of speech comprehension. In very careful experimental approaches, each detail of the speech comprehension process is examined.

•    The speech signal carries information on different time scales. In time steps of 10^-2s, information comes in on which segments are produced (/p/, /b/, /t/ ,... ?); in time steps of 10^-1s, information comes in on speech prosody (intonation, rhythm, loudness). How are these segmental and suprasegmental properties used and integrated?

•    Comprehending speech is more than audition. When we perceive speech, we draw on prior knowledge about the words and the most likely combination of words and phonemes in our native language. Additionally, a speaker produces not only an acoustic signal but also a visual signal. How are these sources of information combined? Do lexical and visual information actually change how we perceive the acoustic signal?

•    Different speakers may pronounce the same word rather differently. How do we deal with this variability? Previous work suggests that we store instances of speech sounds and words from individual speakers in memory. How exactly is this episodic knowledge stored and how does it influence online processing?

•    Speech comprehension is not an end in itself. We recognize words to get at the intended meaning. How does this interface work? Does the recognition of a word form automatically activate all conceptual information? Conceptual information is multi-faceted and contains information about shapes, colours, smells, categories, etc. Do all these aspects become available automatically, or does access depend on the situational demands and the intentions of the listener?

Speech+

The detailed approach of the first subproject is supplemented with a second subproject in which speech comprehension as a whole is considered in a variety of domains. This also evaluates the ecological validity of our models.

•    Speech is unlike print in that it contains reduced variants of the same word (just yesterday becomes "jus yesday"). Such variations are considered as erroneous in print but occur regularly in speech. How do the different aspects of speech comprehension contribute to the recognition of such forms in natural conversation?

•    The apparent effortlessness of speech comprehension easily breaks down. How can we explain the problems in speech comprehension in a second language? What is the contribution of hearing loss and cognitive decline to speech perception problems in elderly listeners?

•    Is speech perception special? Are there commonalities with other perceptual processes in vision and audition? Is speech a cultural artefact that has adapted in a memetic evolution to our perceptual system? Or have in fact our perceptual systems evolved genetically to support the perception of speech?

•    How does speech perception relate to speech production? How are they integrated to support efficient communication in dialogue settings?