Presentations

Abstract

How do human cultural inventions such as reading result in neural re-organization? Previous cross-sectional studies have reported extensive effects of literacy on the neural systems for vision and language (Dehaene et al [2010, Science], Castro-Caldas et al [1998, Brain], Petersson et al [1998, NeuroImage], Carreiras et al [2009, Nature]). In this first longitudinal study with completely illiterate participants, we measured brain responses to speech, text, and other categories of visual stimuli with fMRI before and after a group of illiterate participants in India completed a literacy training program in which they learned to read and write Devanagari script. A literate and an illiterate no-training control group were matched to the training group in terms of socioeconomic background and were recruited from the same societal community in two villages of a rural area near Lucknow, India. This design permitted investigating effects of literacy cross-sectionally across groups before training (N=86) as well as longitudinally (training group N=25). The two analysis approaches yielded converging results: Literacy was associated with enhanced, mainly left-lateralized responses to written text along the ventral stream (including lingual gyrus, fusiform gyrus, and parahippocampal gyrus), dorsal stream (intraparietal sulcus), and (pre-) motor systems (pre-central sulcus, supplementary motor area), thalamus (pulvinar), and cerebellum. Significantly reduced responses were observed bilaterally in the superior parietal lobe (precuneus) and in the right angular gyrus. These positive effects corroborate and extend previous findings from cross-sectional studies. However, effects of literacy were specific to written text and (to a lesser extent) to false fonts. Contrary to previous research, we found no direct evidence of literacy affecting the processing of other types of visual stimuli such as faces, tools, houses, and checkerboards. Furthermore, unlike in some previous studies, we did not find any evidence for effects of literacy on responses in the auditory cortex in our Hindi-speaking participants. We conclude that learning to read has a specific and extensive effect on the processing of written text along the visual pathways, including low-level thalamic nuclei, high-level systems in the intraparietal sulcus and the fusiform gyrus, and motor areas. The absence of an effect of literacy on responses in the auditory cortex in particular raises questions about the extent to which phonological representations in the auditory cortex are altered by literacy acquisition or recruited online during reading.

Abstract

To what extent is visual attention driven by the semantics of individual objects, rather than by their visual appearance? To investigate this we continuously measured eye movements, while observers searched through displays of common objects for an aurally instructed target. On crucial trials, the target was absent, but the display contained object s that were either semantically or visually related to the target. We hypothesized that timing is crucial in the occurrence and strength of semantic influences on visual orienting, and therefore presented the target instruction either before, during, or af ter (memory - based search) picture onset. When the target instruction was presented before picture onset we found a substantial, but delayed bias in orienting towards semantically related objects as compared to visually related objects. However, this delay disappeared when the visual information was presented before the target instruction. Furthermore, the temporal dynamics of the semantic bias did not change in the absence of visual competition. These results po int to cascadic but independent influences of semantic and visual representations on attention. In addition. the results of the memory - based search studies suggest that visual and semantic biases only arise when the visual stimuli are present. Although we consistent ly found that people fixate at locat ions previously occupied by the target object (a replication of earlier findings), we did not find such biases for visually or semantically related objects. Overall, our studies show that the question whether visual orienting is driven by semantic c ontent is better rephrased as when visual orienting is driven by semantic content.

Abstract

I will discuss recent research with illiterates and individuals with dyslexia which suggests that many cognitive ‚defi ciencies‘ proposed as possible causes of dyslexia are simply a consequence of decreased reading experience. I will argue that in order to make further progress towards an understanding of the causes of dyslexia it is necessary to appropriately distinguish between cause and effect.

Abstract

How do human cultural inventions such as reading result in neural re-organization? In this first longitudinal study with young completely illiterate adult participants, we measured brain responses to speech, text, and other categories of visual stimuli with fMRI before and after a group of illiterate participants in India completed a literacy training program in which they learned to read and write Devanagari script. A literate and an illiterate no-training control group were matched to the training group in terms of socioeconomic background and were recruited from the same societal community in two villages of a rural area near Lucknow, India. This design permitted investigating effects of literacy cross-sectionally across groups before training (N=86) as well as longitudinally (training group N=25). The two analysis approaches yielded converging results: Literacy was associated with enhanced, left-lateralized responses to written text along the ventral stream (including lingual gyrus, fusiform gyrus, and parahippocampal gyrus), dorsal stream (intraparietal sulcus), and (pre-) motor systems (pre-central sulcus, supplementary motor area) and thalamus (pulvinar). Significantly reduced responses were observed bilaterally in the superior parietal lobe (precuneus) and in the right angular gyrus. These effects corroborate and extend previous findings from cross-sectional studies. However, effects of literacy were specific to written text and (to a lesser extent) to false fonts. We did not find any evidence for effects of literacy on responses in the auditory cortex in our Hindi-speaking participants. This raises questions about the extent to which phonological representations are altered by literacy acquisition.

Abstract

How do human
cultural
inventions
such as reading
result
in neural
re-organization?
In this first longitudinal
study
with young
completely
illiterate
adult
participants,
we measured
brain
responses
to speech,
text, and other
categories
of visual
stimuli
with fMRI
before
and after a group
of
illiterate
participants
in India
completed
a literacy
training
program
in which
they learned
to read and write
Devanagari
script.
A literate
and an illiterate
no-training
control
group
were
matched
to the
training
group
in terms
of socioeconomic
background
and were
recruited
from
the same
societal
community
in two villages
of a
rural area near Lucknow,
India.
This design
permitted
investigating
effects
of literacy
cross-sectionally
across
groups
before
training
(N=86)
as well as longitudinally
(training
group
N=25).
The two
analysis
approaches
yielded
converging
results:
Literacy
was
associated
with enhanced,
left-lateralized
responses
to written
text
along
the ventral
stream
(including
lingual
gyrus,
fusiform
gyrus,
and parahippocampal
gyrus),
dorsal
stream
(intraparietal
sulcus),
and (pre-)
motor
systems
(pre-central
sulcus,
supplementary
motor
area)
and thalamus
(pulvinar).
Significantly
reduced
responses
were observed
bilaterally
in the superior
parietal
lobe (precuneus)
and in the right angular
gyrus.
These
effects
corroborate
and extend
previous
findings
from
cross-sectional
studies.
However,
effects
of literacy
were
specific
to written
text and (to a lesser
extent)
to
false fonts.
We did not find any evidence
for effects
of literacy
on
responses
in the auditory
cortex
in our Hindi-speaking
participants.
This
raises
questions
about
the extent
to which
phonological
representations are altered by literacy acquisition.

Abstract

A wealth of research has demonstrated that prediction is a core feature of human information processing. Much less is known, however, about the nature and the extent of predictive processing abilities. Here we investigated whether high levels of language expertise attained through formal literacy are related to anticipatory language-mediated visual orienting. Indian low and high literates listened to simple spoken sentences containing a target word (e.g., "door") while at the same time looking at a visual display of four objects (a target, i.e. the door, and three distractors). The spoken sentences were constructed to encourage anticipatory eye movements to visual target objects. High literates started to shift their eye gaze to the target object well before target word onset. In the low literacy group this shift of eye gaze occurred more than a second later, well after the onset of the target. Our findings suggest that formal literacy is crucial for the fine-tuning of language-mediated anticipatory mechanisms, abilities which proficient language users can then exploit for other cognitive activities such as language-mediated visual orienting.

Abstract

Eye movements made by listeners during language-mediated visual search reveal a strong link between visual processing and conceptual processing. For example, upon hearing the word for a missing referent with a characteristic colour (e.g., “strawberry”), listeners tend to fixate a colour-matched distractor (e.g., a red plane) more than a colour-mismatched distractor (e.g., a yellow plane). We ask whether these shifts in visual attention are mediated by the retrieval of lexically stored colour labels. Do children who do not yet possess verbal labels for the colour attribute that spoken and viewed objects have in common exhibit language-mediated eye movements like those made by older children and adults? That is, do toddlers look at a red plane when hearing “strawberry”? We observed that 24-month-olds lacking colour-term knowledge nonetheless recognised the perceptual-conceptual commonality between named and seen objects. This indicates that language-mediated visual search need not depend on stored labels for concepts.

Abstract

If a Mandarin speaker had walked past two rivers and wished to describe how many he had seen, he would have to say “two tiao river”, where tiao designates long, rope-like objects such as rivers, snakes and legs. Tiao is one of several hundred classifiers – a grammatical category in Mandarin. In two eye-tracking studies we presented Mandarin speakers with simple Mandarin sentences through headphones while monitoring their eye-movements to objects presented on a computer monitor. The crucial question is what participants look at while listening to a pre-specified target noun. If classifier categories influence general conceptual processing then on hearing the target noun participants should look at objects that are also members of the same classifier category – even when the classifier is not explicitly present. For example, on hearing scissors, Mandarin speakers should look more at a picture of a chair than at an unrelated object because scissors and chair share the classifier ba. This would be consistent with a Strong Whorfian position, according to which language is a major determinant in shaping conceptual thought (Sapir, 1921; Whorf, 1956). A weaker influence of language-on-thought could be predicted, where language shapes cognitive processing, but only when the language-specific category is actively being processed (Slobin, 1996). According to this account, eye-movements are not necessarily drawn to chair when a participant hears scissors, but they would be on hearing ba scissors. This is because hearing ba activates the linguistic category that both scissors and chair belong to. A third logical possibility is that classifiers are purely formal markers (cf. Greenberg, 1972; Lehman, 1979) that do not influence attentional processing even when they are explicitly present. The data showed that when participants heard a spoken word from the same classifier category as a visually depicted object (e.g. scissors-chair), but the classifier was not explicitly presented in the speech, overt attention to classifier-match objects (e.g. chair) and distractor objects did not differ (Experiment 1). But when the classifier was explicitly presented (e.g. ba, Experiment 2), participants shifted overt attention significantly more to classifier-match objects (e.g. chair) than to distractors. These data are incompatible with the Strong Whorfian hypothesis. Instead the findings support the Weak Whorfian hypothesis that linguistic distinctions force attention to properties of the world but only during active linguistic processing of that distinction (cf. Slobin, 1996).