Presentations

Abstract

Serial naming deficits have been identified as core symptoms of developmental dyslexia. A prominent hypothesis is that naming delays are due to inefficient phonological encoding, yet the exact nature of this underlying impairment remains largely underspecified. Here we used recordings of eye movements and word onset latencies to examine at what processing level the dyslexic naming deficit emerges: localized at an early stage of lexical encoding or rather later at the level of phonetic or motor planning. 23 dyslexic and 25 control adult readers were tested on a serial object naming task for 30 items and an analogous reading task, where phonological neighborhood density and word-frequency were manipulated. Results showed that both word properties influenced early stages of phonological activation (first fixation and first-pass duration) equally in both groups of participants. Moreover, in the control group any difficulty appeared to be resolved early in the reading process, while for dyslexic readers a processing disadvantage for low-frequency words and for words with sparse neighborhood also emerged in a measure that included late stages of output planning (eye-voice span). Thus, our findings suggest suboptimal phonetic and/or articulatory planning in dyslexia.

Abstract

Reading functions beyond early visual precessing are known to be lateralized to the left hemisphere, but how left-lateralization arises during literacy acquisition is an open question. Bilateral processing or rightward asymmetries have previously been associated with developmental dyslexia. However, it is unclear at present to what extent this lack of left-lateralization reflects differences in reading ability. In this study, a group of illiterate adults in rural India (N=29) participated in a literacy training program over the course of six months. fMRI measures were obtained before and after training on a number of different visual stimulus categories, including written sentences, false fonts, and object categories such as houses and faces. This training group was matched on demographic and socioeconomic variables to an illiterate no-training group and to low- and highly-literate control groups, who were also scanned twice but received no training (total N=90). In a cross-sectional analysis before training, reading ability was positively correlated with increased BOLD responses in a left-lateralized network including the dorsal and ventral visual streams for text and false fonts, but not for other types of visual stimuli. A longitudinal analysis of learning effects in the training group showed that beginning readers engage bilateral networks more than proficient readers. Lateralization of BOLD responses was further examined by calculating laterality indices in specific regions. We observed training-related changes in lateralization for processing written stimuli in a number of subregions in the dorsal and ventral visual streams, as well as in the cerebellum. Together with the cross-sectional results, these data suggest a causal relationship between reading ability and the degree of hemispheric asymmetry in processing written materials.

Abstract

Reading functions beyond early visual precessing are known to be lateralized to the left hemisphere, but how left-lateralization arises during literacy acquisition is an open question. Bilateral processing or rightward asymmetries have previously been associated with developmental dyslexia. However, it is unclear at present to what extent this lack of left-lateralization reflects differences in reading ability. In this study, a group of illiterate adults in rural India (N=29) participated in a literacy training program over the course of six months. fMRI measures were obtained before and after training on a number of different visual stimulus categories, including written sentences, false fonts, and object categories such as houses and faces. This training group was matched on demographic and socioeconomic variables to an illiterate no-training group and to low- and highly-literate control groups, who were also scanned twice but received no training (total N=90). In a cross-sectional analysis before training, reading ability was positively correlated with increased BOLD responses in a left-lateralized network including the dorsal and ventral visual streams for text and false fonts, but not for other types of visual stimuli. A longitudinal analysis of learning effects in the training group showed that beginning readers engage bilateral networks more than proficient readers. Lateralization of BOLD responses was further examined by calculating laterality indices in specific regions. We observed training-related changes in lateralization for processing written stimuli in a number of subregions in the dorsal and ventral visual streams, as well as in the cerebellum. Together with the cross-sectional results, these data suggest a causal relationship between reading ability and the degree of hemispheric asymmetry in processing written materials.

Abstract

Many psycholinguistic experiments suggest that prediction is an important characteristic of language processing. Some recent theoretical accounts in the cognitive sciences (e.g., Clark, 2013; Friston, 2010) and psycholinguistics (e.g., Dell & Chang, 2014) appear to suggest that prediction is even necessary to understand language. I will evaluate this proposal. I will first discuss several arguments that may appear to be in line with the notion that prediction is necessary for language processing. These arguments include that prediction provides a unified theoretical principle of the human mind and that it pervades cortical function. We discuss whether evidence of human abilities to detect statistical regularities is necessarily evidence for predictive processing and evaluate suggestions that prediction is necessary for language learning. Five arguments are then presented that question the claim that all language processing is predictive in nature. I point out that not all language users appear to predict language and that suboptimal input makes prediction often very challenging. Prediction, moreover, is strongly context-dependent and impeded by resource limitations. I will also argue that it may be problematic that most experimental evidence for predictive language processing comes from 'prediction-encouraging' experimental set-ups. Finally, I will discuss possible ways that may lead to a further resolution of this debate. We conclude that languages can be learned and understood in the absence of prediction. Claims that all language processing is predictive in nature are premature.

Abstract

How do human cultural inventions such as reading result in neural re-organization? In this first longitudinal study with young completely illiterate adult participants, we measured brain responses to speech, text, and other categories of visual stimuli with fMRI before and after a group of illiterate participants in India completed a literacy training program in which they learned to read and write Devanagari script. A literate and an illiterate no-training control group were matched to the training group in terms of socioeconomic background and were recruited from the same societal community in two villages of a rural area near Lucknow, India. This design permitted investigating effects of literacy cross-sectionally across groups before training (N=86) as well as longitudinally (training group N=25). The two analysis approaches yielded converging results: Literacy was associated with enhanced, left-lateralized responses to written text along the ventral stream (including lingual gyrus, fusiform gyrus, and parahippocampal gyrus), dorsal stream (intraparietal sulcus), and (pre-) motor systems (pre-central sulcus, supplementary motor area) and thalamus (pulvinar). Significantly reduced responses were observed bilaterally in the superior parietal lobe (precuneus) and in the right angular gyrus. These effects corroborate and extend previous findings from cross-sectional studies. However, effects of literacy were specific to written text and (to a lesser extent) to false fonts. We did not find any evidence for effects of literacy on responses in the auditory cortex in our Hindi-speaking participants. This raises questions about the extent to which phonological representations are altered by literacy acquisition.

Abstract

We conceptualize objects based on sensory and motor information gleaned from real-world experience. But to what extent is such conceptual information structured according to higher level linguistic features too? Here we investigate whether classifiers, a grammatical category, shape the conceptual representations of objects. In three experiments native Mandarin speakers (speakers of a classifier language) and native Dutch speakers (speakers of a language without classifiers) judged the similarity of a target object (presented as a word or picture) with four objects (presented as words or pictures). One object shared a classifier with the target, the other objects did not, serving as distractors. Across all experiments, participants judged the target object as more similar to the object with the shared classifier than distractor objects. This effect was seen in both Dutch and Mandarin speakers, and there was no difference between the two languages. Thus, even speakers of a non-classifier language are sensitive to object similarities underlying classifier systems, and using a classifier system does not exaggerate these similarities. This suggests that classifier systems simply reflect, rather than affect, conceptual structure.

Abstract

A central claim of embodied theories of cognition is that sensory representations are
routinely activated and influence language processing even in the absence of relevant
sensory input (cf. Pulvermüller, 2005; Wassenburg & Zwaan, 2010). We tested the influence
of color representations during language processing in three visual world eye tracking
experiments. The method is particularly well suited to investigate this issue because the
availability of relevant visual input can be manipulated.
We made use of the phenomena that when participants hear a word that refers to a
visual object or printed word they quickly direct their eye gaze to objects or printed words
which are similar (e.g. semantically or visually) to the heard word. We used a look and listen
task which previously has been shown to be sensitive to such relationships between spoken
words and visual items. In Experiment 1, on experimental trials, participants listened to
sentences containing a critical target word associated with a prototypical color (e.g.
'...spinach...') as they inspected a visual display with four words printed in black font. One of
the four printed words was associated with the same prototypical color (e.g. green) as the
spoken target word (e.g. FROG). On experimental trials, the spoken target word did not have
a printed word counterpart (SPINACH was not present in the display). In filler trials (70% of
trials) the target was present in the display and attracted significantly more overt attention
than the unrelated distractors. In experimental trials color competitors were not looked at
more than the distractors. In Experiment 2 the printed words were replaced with line
drawings of the objects. In order to direct the attentional focus of our participants toward
color features we used a within-participants counter-balanced design and alternated color
and greyscale trials randomly throughout the experiment. Therefore, on one trial our
participants heard a word such as 'spinach' and saw a frog (colored in green) in the visual
display. On the next trial however they saw a banana (in greyscale) on hearing 'canary'
(bananas and canaries are typically yellow), etc. The presence (or absence) of color was
thus a salient property of the experiment. Participants looked more at color competitors than
unrelated distractors on hearing the target word in the color trials but not in the greyscale
trials, i.e. on hearing 'spinach' they looked at the green frog but not the greyscale frog.
Experiment 3 was identical to Experiment 2, except that the visual display was removed at
the sentence onset, after a longer preview. This experiment examined whether the continued
presence of color in the immediate visual environment was necessary for the observation of
color-mediated eye movements. Eye movements directed towards the now blank screen
were recorded as the sentence unfolded (cf. Spivey & Geng, 2001). In the filler trials,
participants looked significantly more at the locations where the targets, rather than the
distractors, had been previously presented as the target words acoustically unfolded. In the
experimental trials, the locations where the color competitors had previously been presented
did not attract increased attention (neither in color nor greyscale trials).
These data demonstrate that language-mediated eye movements are only influenced
by color relations between spoken words and visually displayed items if color is present in the immediate visual environment. We conclude that color representations are unlikely to be
routinely activated in language processing. Our findings provide strong constraints for
embodied theories of cognition which assume that sensory representations influence language processing even in the absence of relevant sensory input. These results fit best with the notion that the main role of sensory representations in language processing is a different one, namely to contextualize language in the immediate environment, connecting language to the here and now.

Abstract

The notion that predicting upcoming linguistic information in language comprehension makes use of the production system has recently received much attention (e.g., Chang et al., 2006; Dell & Chang, 2014; Federmeier, 2007; Pickering & Garrod, 2007, 2013; Van Berkum et al., 2005). So far there has been little experimental evidence for a relation between prediction and production. I will discuss the results of several recent eye-tracking experiments with toddlers (Mani & Huettig, 2012) and adults (Rommers et al. submitted, Hintz et al., in prep.) which provide some support for the view that production abilities are linked to language-mediated anticipatory eye movements. These data however also indicate that production-based prediction is situation-dependent and only one of many mechanisms supporting prediction. Taken together, these results suggest that multiple-mechanism accounts are required to provide a complete picture of anticipatory language processing.

Abstract

If a Mandarin speaker had walked past two rivers and wished to describe how many he had seen, he would have to say “two tiao river”, where tiao designates long, rope-like objects such as rivers, snakes and legs. Tiao is one of several hundred classifiers – a grammatical category in Mandarin. In two eye-tracking studies we presented Mandarin speakers with simple Mandarin sentences through headphones while monitoring their eye-movements to objects presented on a computer monitor. The crucial question is what participants look at while listening to a pre-specified target noun. If classifier categories influence general conceptual processing then on hearing the target noun participants should look at objects that are also members of the same classifier category – even when the classifier is not explicitly present. For example, on hearing scissors, Mandarin speakers should look more at a picture of a chair than at an unrelated object because scissors and chair share the classifier ba. This would be consistent with a Strong Whorfian position, according to which language is a major determinant in shaping conceptual thought (Sapir, 1921; Whorf, 1956). A weaker influence of language-on-thought could be predicted, where language shapes cognitive processing, but only when the language-specific category is actively being processed (Slobin, 1996). According to this account, eye-movements are not necessarily drawn to chair when a participant hears scissors, but they would be on hearing ba scissors. This is because hearing ba activates the linguistic category that both scissors and chair belong to. A third logical possibility is that classifiers are purely formal markers (cf. Greenberg, 1972; Lehman, 1979) that do not influence attentional processing even when they are explicitly present. The data showed that when participants heard a spoken word from the same classifier category as a visually depicted object (e.g. scissors-chair), but the classifier was not explicitly presented in the speech, overt attention to classifier-match objects (e.g. chair) and distractor objects did not differ (Experiment 1). But when the classifier was explicitly presented (e.g. ba, Experiment 2), participants shifted overt attention significantly more to classifier-match objects (e.g. chair) than to distractors. These data are incompatible with the Strong Whorfian hypothesis. Instead the findings support the Weak Whorfian hypothesis that linguistic distinctions force attention to properties of the world but only during active linguistic processing of that distinction (cf. Slobin, 1996).