Falk Huettig

Presentations

Displaying 1 - 30 of 30
  • Hintz, F., Meyer, A. S., & Huettig, F. (2014). Mechanisms underlying predictive language processing. Talk presented at the 56. Tagung experimentell arbeitender Psychologen [TeaP, Conference on Experimental Psychology]. Giessen, Germany. 2014-03-31 - 2014-04-02.
  • Hintz, F., Meyer, A. S., & Huettig, F. (2014). The influence of verb-specific featural restrictions, word associations, and production-based mechanisms on language-mediated anticipatory eye movements. Talk presented at the 27th annual CUNY conference on human sentence processing. Ohio State University, Columbus/Ohio (US). 2014-03-13 - 2014-03-15.
  • Huettig, F., & Guerra, E. (2014). Context-dependent mapping of linguistic and color representations challenges strong forms of embodiment. Talk presented at the 20th Architectures and Mechanisms for Language Processing Conference (AMLAP 2014). Edinburgh, UK. 2014-09-03 - 2014-09-06.

    Abstract

    A central claim of embodied theories of cognition is that sensory representations are routinely activated and influence language processing even in the absence of relevant sensory input (cf. Pulvermüller, 2005; Wassenburg & Zwaan, 2010). We tested the influence of color representations during language processing in three visual world eye tracking experiments. The method is particularly well suited to investigate this issue because the availability of relevant visual input can be manipulated. We made use of the phenomena that when participants hear a word that refers to a visual object or printed word they quickly direct their eye gaze to objects or printed words which are similar (e.g. semantically or visually) to the heard word. We used a look and listen task which previously has been shown to be sensitive to such relationships between spoken words and visual items. In Experiment 1, on experimental trials, participants listened to sentences containing a critical target word associated with a prototypical color (e.g. '...spinach...') as they inspected a visual display with four words printed in black font. One of the four printed words was associated with the same prototypical color (e.g. green) as the spoken target word (e.g. FROG). On experimental trials, the spoken target word did not have a printed word counterpart (SPINACH was not present in the display). In filler trials (70% of trials) the target was present in the display and attracted significantly more overt attention than the unrelated distractors. In experimental trials color competitors were not looked at more than the distractors. In Experiment 2 the printed words were replaced with line drawings of the objects. In order to direct the attentional focus of our participants toward color features we used a within-participants counter-balanced design and alternated color and greyscale trials randomly throughout the experiment. Therefore, on one trial our participants heard a word such as 'spinach' and saw a frog (colored in green) in the visual display. On the next trial however they saw a banana (in greyscale) on hearing 'canary' (bananas and canaries are typically yellow), etc. The presence (or absence) of color was thus a salient property of the experiment. Participants looked more at color competitors than unrelated distractors on hearing the target word in the color trials but not in the greyscale trials, i.e. on hearing 'spinach' they looked at the green frog but not the greyscale frog. Experiment 3 was identical to Experiment 2, except that the visual display was removed at the sentence onset, after a longer preview. This experiment examined whether the continued presence of color in the immediate visual environment was necessary for the observation of color-mediated eye movements. Eye movements directed towards the now blank screen were recorded as the sentence unfolded (cf. Spivey & Geng, 2001). In the filler trials, participants looked significantly more at the locations where the targets, rather than the distractors, had been previously presented as the target words acoustically unfolded. In the experimental trials, the locations where the color competitors had previously been presented did not attract increased attention (neither in color nor greyscale trials). These data demonstrate that language-mediated eye movements are only influenced by color relations between spoken words and visually displayed items if color is present in the immediate visual environment. We conclude that color representations are unlikely to be routinely activated in language processing. Our findings provide strong constraints for embodied theories of cognition which assume that sensory representations influence language processing even in the absence of relevant sensory input. These results fit best with the notion that the main role of sensory representations in language processing is a different one, namely to contextualize language in the immediate environment, connecting language to the here and now.
  • Huettig, F. (2015). Does prediction in language comprehension involve language production?. Talk presented at the Comprehension=Production? workshop. Nijmegen, the Netherlands. 2015-03-26 - 2015-03-28.

    Abstract

    The notion that predicting upcoming linguistic information in language comprehension makes use of the production system has recently received much attention (e.g., Chang et al., 2006; Dell & Chang, 2014; Federmeier, 2007; Pickering & Garrod, 2007, 2013; Van Berkum et al., 2005). So far there has been little experimental evidence for a relation between prediction and production. I will discuss the results of several recent eye-tracking experiments with toddlers (Mani & Huettig, 2012) and adults (Rommers et al. submitted, Hintz et al., in prep.) which provide some support for the view that production abilities are linked to language-mediated anticipatory eye movements. These data however also indicate that production-based prediction is situation-dependent and only one of many mechanisms supporting prediction. Taken together, these results suggest that multiple-mechanism accounts are required to provide a complete picture of anticipatory language processing.
  • Huettig, F. (2014). How embodied is language processing?. Talk presented at the 2nd Attentive Listener in the Visual World workshop. Hyderabad, India. 2014-11-03 - 2014-11-05.
  • Huettig, F. (2014). How literacy acquisition affects the illiterate mind. Talk presented at the Low Educated Second Language and Literacy Acquisition (LESLLA). Nijmegen, Netherlands. 2014-08-28 - 2014-08-30.
  • Huettig, F. (2014). Literacy influences on predictive language processing and visual search. Talk presented at the Priming across Modalities: The Influence of Orthography on Sign and Spoken Language Processing workshop. Haifa, Israel. 2014-04.
  • Huettig, F. (2014). The context-dependent influence of colour representations during language-vision interactions constrains theories of conceptual processing. Talk presented at the Color in Concepts workshop. Düsseldorf, Germany. 2014-06-02 - 2014-06-03.
  • Smith, A. C., Monaghan, P., & Huettig, F. (2014). A comprehensive model of spoken word recognition must be multimodal: Evidence from studies of language mediated visual attention. Talk presented at the 36th Annual Conference of the Cognitive Science Society [CogSci 2014]. Quebec, Canada. 2014-07-23 - 2014-07-26.
  • Smith, A. C., Monaghan, P., & Huettig, F. (2014). Examining strains and symptoms of the ‘Literacy Virus’: The effects of orthographic transparency on phonological processing in a connectionist model of reading. Talk presented at the 36th Annual Conference of the Cognitive Science Society [CogSci 2014]. Quebec, Canada. 2014-07-23 - 2014-07-26.
  • Smith, A. C., Monaghan, P., & Huettig, F. (2014). Examining the effects of orthographic transparency on phonological and semantic processing within a connectionist implementation of the triangle model of reading. Talk presented at the 14th Neural Computation and Psychology Workshop [NCPW 14]. Lancaster, U.K. 2014-08-21 - 2014-08-23.
  • Huettig, F. (2013). Anticipatory eye movements and predictive language processing. Talk presented at the ZiF research group on "Competition and Priority Control in Mind and Brain. Bielefeld, Germany. 2013-07.
  • Huettig, F., Mishra, R. K., Kumar, U., Singh, J. P., Guleria, A., & Tripathi, V. (2013). Phonemic and syllabic awareness of adult literates and illiterates in an Indian alphasyllabic language. Talk presented at the Tagung experimentell arbeitender Psychologen [TeaP 2013]. Vienna, Austria. 2013-03-24 - 2013-03-27.
  • Huettig, F., Mani, N., Mishra, R. K., & Brouwer, S. (2013). Reading ability predicts anticipatory language processing in children, low literate adults, and adults with dyslexia. Talk presented at the 54th Annual Meeting of the Psychonomic Society. Toronto, Canada. 2013-11-14 - 2013-11-17.
  • Huettig, F., Mani, N., Mishra, R. K., & Brouwer, S. (2013). Reading ability predicts anticipatory language processing in children, low literate adults, and adults with dyslexia. Talk presented at the 11th International Symposium of Psycholinguistics. Tenerife, Spain. 2013-03-20 - 2013-03-23.
  • Janse, E., Huettig, F., & Jesse, A. (2013). Working memory modulates the immediate use of context for recognizing words in sentences. Talk presented at the 5th Workshop on Speech in Noise: Intelligibility and Quality. Vitoria, Spain. 2013-01-10 - 2013-01-11.
  • Mani, N., & Huettig, F. (2013). Reading ability predicts anticipatory language processing in 8 year olds. Talk presented at the Tagung experimentell arbeitender Psychologen [TeaP 2013]. Vienna, Austria. 2013-03-24 - 2013-03-27.
  • Rommers, J., Meyer, A. S., Praamstra, P., & Huettig, F. (2013). Anticipating references to objects during sentence comprehension. Talk presented at the Experimental Psychology Society meeting (EPS). Bangor, UK. 2013-07-03 - 2013-07-05.
  • Rommers, J., Meyer, A. S., Piai, V., & Huettig, F. (2013). Constraining the involvement of language production in comprehension: A comparison of object naming and object viewing in sentence context. Talk presented at the 19th Annual Conference on Architectures and Mechanisms for Language Processing [AMLaP 2013]. Marseille, France. 2013-09-02 - 2013-09-04.
  • Smith, A. C., Monaghan, P., & Huettig, F. (2013). Modelling the effect of literacy on multimodal interactions during spoken language processing in the visual world. Talk presented at Tagung experimentell arbeitender Psychologen. [TEAP 2013]. Vienna, Austria. 2013-03-24 - 2013-03-27.

    Abstract

    Recent empirical evidence suggests that language-mediated eye gaze around the visual world varies across individuals and is partly determined by their level of formal literacy training. Huettig, Singh & Mishra (2011) showed that unlike high-literate individuals, whose eye gaze was closely time locked to phonological overlap between a spoken target word and items presented in a visual display, low-literate individuals eye gaze was not tightly locked to phonological overlap in the speech signal but instead strongly influenced by semantic relationships between items. Our present study tests the hypothesis that this behaviour is an emergent property of an increased ability to extract phonological structure from the speech signal, as in the case of high-literates, with low-literates more reliant on syllabic structure. This hypothesis was tested using an emergent connectionist model, based on the Hub-and-spoke models of semantic processing (Dilkina et al, 2008), that integrates linguistic information extracted from the speech signal with visual and semantic information within a central resource. We demonstrate that contrasts in fixation behaviour similar to those observed between high and low literates emerge when the model is trained on either a speech signal segmented by phoneme (i.e. high-literates) or by syllable (i.e. low-literates).
  • Smith, A. C., Monaghan, P., & Huettig, F. (2013). Phonological grain size and general processing speed modulates language mediated visual attention – Evidence from a connectionist model. Talk presented at The 19th Annual Conference on Architectures and Mechanisms for Language Processing [AMLaP 2013]. Marseille, France. 2013-09-02 - 2013-09-04.
  • Smith, A. C., Monaghan, P., & Huettig, F. (2013). Putting rhyme in context: Visual and semantic competition eliminates phonological rhyme effects in language-mediated eye gaze. Talk presented at The 18th Conference of the European Society for Cognitive Psychology [ESCOP 2013]. Budapest, Hungary. 2013-08-29 - 2013-09-01.
  • Huettig, F. (2010). Looking, language, and memory. Talk presented at Language, Cognition, and Emotion Workshop. Delhi, India. 2010-12-06 - 2010-12-06.
  • Huettig, F., Singh, N., & Mishra, R. (2010). Language-mediated prediction is contingent upon formal literacy. Talk presented at Brain, Speech and Orthography Workshop. Brussels, Belgium. 2010-10-15 - 2010-10-16.

    Abstract

    A wealth of research has demonstrated that prediction is a core feature of human information processing. Much less is known, however, about the nature and the extent of predictive processing abilities. Here we investigated whether high levels of language expertise attained through formal literacy are related to anticipatory language-mediated visual orienting. Indian low and high literates listened to simple spoken sentences containing a target word (e.g., "door") while at the same time looking at a visual display of four objects (a target, i.e. the door, and three distractors). The spoken sentences were constructed to encourage anticipatory eye movements to visual target objects. High literates started to shift their eye gaze to the target object well before target word onset. In the low literacy group this shift of eye gaze occurred more than a second later, well after the onset of the target. Our findings suggest that formal literacy is crucial for the fine-tuning of language-mediated anticipatory mechanisms, abilities which proficient language users can then exploit for other cognitive activities such as language-mediated visual orienting.
  • Huettig, F. (2010). Toddlers’ language-mediated visual search: They need not have the words for it. Talk presented at International Conference on Cognitive Development 2010. Allahabad, India. 2010-12-10 - 2010-12-13.

    Abstract

    Eye movements made by listeners during language-mediated visual search reveal a strong link between visual processing and conceptual processing. For example, upon hearing the word for a missing referent with a characteristic colour (e.g., “strawberry”), listeners tend to fixate a colour-matched distractor (e.g., a red plane) more than a colour-mismatched distractor (e.g., a yellow plane). We ask whether these shifts in visual attention are mediated by the retrieval of lexically stored colour labels. Do children who do not yet possess verbal labels for the colour attribute that spoken and viewed objects have in common exhibit language-mediated eye movements like those made by older children and adults? That is, do toddlers look at a red plane when hearing “strawberry”? We observed that 24-month-olds lacking colour-term knowledge nonetheless recognised the perceptual-conceptual commonality between named and seen objects. This indicates that language-mediated visual search need not depend on stored labels for concepts.
  • Huettig, F., & McQueen, J. M. (2009). AM radio noise changes the dynamics of spoken word recognition. Talk presented at 15th Annual Conference on Architectures and Mechanisms for Language Processing (AMLaP 2009). Barcelona, Spain. 2009-09-09.

    Abstract

    Language processing does not take place in isolation from the sensory environment. Listeners are able to recognise spoken words in many different situations, ranging from carefully articulated and noise-free laboratory speech, through casual conversational speech in a quiet room, to degraded conversational speech in a busy train-station. For listeners to be able to recognize speech optimally in each of these listening situations, they must be able to adapt to the constraints of each situation. We investigated this flexibility by comparing the dynamics of the spoken-word recognition process in clear speech and speech disrupted by radio noise. In Experiment 1, Dutch participants listened to clearly articulated spoken Dutch sentences which each included a critical word while their eye movements to four visual objects presented on a computer screen were measured. There were two critical conditions. In the first, the objects included a cohort competitor (e.g., parachute, “parachute”) with the same onset as the critical spoken word (e.g., paraplu, “umbrella”) and three unrelated distractors. In the second condition, a rhyme competitor (e.g., hamer, “hammer”) of the critical word (e.g., kamer, “room”) was present in the display, again with three distractors. To maximize competitor effects pictures of the critical words themselves were not present in the displays on the experimental trials (e.g.,there was no umbrella in the display with the 'paraplu' sentence) and a passive listening task was used (Huettig McQueen, 2007). Experiment 2 was identical to Experiment 1 except that phonemes in the spoken sentences were replaced with radio-signal noises (as in AM radio listening conditions). In each sentence, two,three or four phonemes were replaced with noises. The sentential position of these replacements was unpredictable, but the adjustments were always made to onset phonemes. The critical words (and the immediately surrounding words) were not changed. The question was whether listeners could learn that, under these circumstances, onset information is less reliable. We predicted that participants would look less at the cohort competitors (the initial match to the competitor is less good) and more at the rhyme competitors (the initial mismatch is less bad). We observed a significant experiment by competitor type interaction. In Experiment 1 participants fixated both kinds competitors more than unrelated distractors, but there were more and earlier looks to cohort competitors than to rhyme competitors (Allopenna et al., 1998). In Experiment 2 participants still fixated cohort competitors more than rhyme competitors but the early cohort effect was reduced and the rhyme effect was stronger and occurred earlier. These results suggest that AM radio noise changes the dynamics of spoken word recognition. The well-attested finding of stronger reliance on word onset overlap in speech recognition appears to be due in part to the use of clear speech in most experiments. When onset information becomes less reliable, listeners appear to depend on it less. A core feature of the speech-recognition system thus appears to be its flexibility. Listeners are able to adjust the perceptual weight they assign to different parts of incoming spoken language.
  • Huettig, F. (2009). Language-mediated visual search. Talk presented at Invited talk at VU Amsterdam. Amsterdam.
  • Huettig, F. (2009). On the use of distributional models of semantic space to investigate human cognition. Talk presented at Distributional Semantics beyond Concrete Concepts (Workshop at Annual Meeting of the Cognitive Science Society (CogSci 2009). Amsterdam, The Netherlands. 2009-07-29 - 2009-01-08.
  • Huettig, F. (2009). The role of colour during language-vision interactions. Talk presented at International Conference on Language-Cognition Interface 2009. Allahabad, India. 2009-12-06 - 2009-12-09.
  • Huettig, F., Chen, J., Bowerman, M., & Majid, A. (2008). Linguistic relativity: Evidence from Mandarin speakers’ eye-movements. Talk presented at 14th Annual Conference on the Architectures and Mechanisms for Language Processing [AMLaP 2008]. Cambridge, UK. 2008-09-04 - 2008-09-06.

    Abstract

    If a Mandarin speaker had walked past two rivers and wished to describe how many he had seen, he would have to say “two tiao river”, where tiao designates long, rope-like objects such as rivers, snakes and legs. Tiao is one of several hundred classifiers – a grammatical category in Mandarin. In two eye-tracking studies we presented Mandarin speakers with simple Mandarin sentences through headphones while monitoring their eye-movements to objects presented on a computer monitor. The crucial question is what participants look at while listening to a pre-specified target noun. If classifier categories influence general conceptual processing then on hearing the target noun participants should look at objects that are also members of the same classifier category – even when the classifier is not explicitly present. For example, on hearing scissors, Mandarin speakers should look more at a picture of a chair than at an unrelated object because scissors and chair share the classifier ba. This would be consistent with a Strong Whorfian position, according to which language is a major determinant in shaping conceptual thought (Sapir, 1921; Whorf, 1956). A weaker influence of language-on-thought could be predicted, where language shapes cognitive processing, but only when the language-specific category is actively being processed (Slobin, 1996). According to this account, eye-movements are not necessarily drawn to chair when a participant hears scissors, but they would be on hearing ba scissors. This is because hearing ba activates the linguistic category that both scissors and chair belong to. A third logical possibility is that classifiers are purely formal markers (cf. Greenberg, 1972; Lehman, 1979) that do not influence attentional processing even when they are explicitly present. The data showed that when participants heard a spoken word from the same classifier category as a visually depicted object (e.g. scissors-chair), but the classifier was not explicitly presented in the speech, overt attention to classifier-match objects (e.g. chair) and distractor objects did not differ (Experiment 1). But when the classifier was explicitly presented (e.g. ba, Experiment 2), participants shifted overt attention significantly more to classifier-match objects (e.g. chair) than to distractors. These data are incompatible with the Strong Whorfian hypothesis. Instead the findings support the Weak Whorfian hypothesis that linguistic distinctions force attention to properties of the world but only during active linguistic processing of that distinction (cf. Slobin, 1996).

Share this page