Presentations

Abstract

Individual differences in language processing are prevalent in our daily lives. However, for decades, psycholinguistic research has largely ignored variation in the normal range of abilities. Recently, scientists have begun to acknowledge the importance of inter-individual variability for a comprehensive characterization of the language system. In spite of this change of attitude, empirical research on individual differences is still sparse, which is in part due to the lack of a suitable research tool. Here, we present a novel battery of behavioral tests for assessing individual differences in language skills in younger adults. The Dutch prototype comprises 29 subtests and assesses many aspects of language knowledge (grammar and vocabulary), linguistic processing skills (word and sentence level) and general cognitive abilities involved in using language (e.g., WM, IQ). Using the battery, researchers can determine performance profiles for individuals and link them to neurobiological or genetic data.

Abstract

Theories of reading propose that the quality of word form representations affects reading comprehension. One claim is that synchronous retrieval of orthographic and phonological representations leads to better performance than asynchronous retrieval. Based on this account, one may hypothesize that synchronous rather than asynchronous presentation of orthographic and phonological forms should be beneficial when establishing the mapping between both, as it should lead to tighter couplings. We tested this hypothesis in two multi-session experiments, where participants studied isolated words of a tonal language unknown to them, Chinese. During study, written (using Pinyin transcription) and spoken word forms were presented simultaneously or in asynchronous fashion (audio-first, written-first). In both experiments, we observed an advantage for asynchronous over synchronous presentation at test, with audio-first presentation being most beneficial. These results suggest that the timing of written and spoken word forms has profound effects on the ease of learning a new tonal language.

Abstract

Prominent theories of predictive language processing assume that language production processes are used to anticipate upcoming linguistic input during comprehension (Dell & Chang, 2014; Pickering & Garrod, 2013). Here, we explored the converse case: Does a task set including production in addition to comprehension encourage prediction, compared to a task only including comprehension? To test this hypothesis, participants carried out a cross-modal naming task (Exp 1a), a self-paced reading task (Exp1 b) that did not include overt production, and a task (Exp 1c) in which naming and reading trials were evenly interleaved. We used the same predictable (N = 40) and non-predictable (N = 40) sentences in all three tasks. The sentences consisted of a fixed agent, a transitive verb and a predictable or non-predictable target word (The man breaks a glass vs. The man borrows a glass). The mean cloze probability in the predictable sentences was .39 (ranging from .06 to .8; zero in the non-predictable sentences). A total of 162 volunteers took part in the experiment which was run in a between-participants design. In Exp 1a, fifty-four participants listened to recordings of the sentences which ended right before the spoken target word. Coinciding with the end of the playback, a picture of the target word was shown which the participants were asked to name as fast as possible. Analyses of their naming latencies revealed a statistically significant naming advantage of 108 ms on predictable over non-predictable trials. Moreover, we found that the objects’ naming advantage was predicted by the target words’ cloze probability in the sentences (r = .347, p = .038). In Exp 1b, 54 participants were asked to read the same sentences in a self-paced fashion. To allow for testing of potential spillover effects, we added a neutral prepositional phrase (breaks a glass from the collection/borrows a glass from the neighbor) to each sentence. The sentences were read word-by-word, advancing by pushing the space bar. On 30% of the trials, comprehension questions were used to keep up participants' focus on comprehending the sentences. Analyses of their spillover region reading times revealed a numerical advantage (8 ms; tspillover = -1.1, n.s.) in the predictable as compared to the non-predictable condition. Importantly, the analysis of participants' responses to the comprehension questions, showed that they understood the sentences (mean accuracy = 93%). In Exp 1c, the task comprised 50% naming trials and 50% reading trials which appeared in random order. Fifty-four participants named and read the same objects and sentences as in the previous versions. The results showed a naming advantage on predictable over non-predictable items (99 ms) and a positive correlation between the items’ cloze probability and their naming advantage (r = .322, p = .055). Crucially, the post-target reading time analysis showed that with naming trials and reading trials interleaved, there was also a statistically reliable prediction effect on reading trials. Participants were 19 ms faster at reading the spillover region on predictable relative to non-predictable items (tspillover = -2.624). To summarize, although we used the same sentences in all sub-experiments, we observed effects of prediction only when the task set involved production. In the reading only experiment (Exp 1b), no evidence for anticipation was obtained although participants clearly understood the sentences and the same sentences yielded reading facilitation when interleaved with naming trials (Exp 1c). This suggests that predictive language processing can be modulated by the comprehenders’ task set. When the task set involves language production, as is often the case in natural conversation, comprehenders appear to engage in prediction to a stronger degree than in pure comprehension tasks. In our discussion, we will discuss the notion that language production may engage prediction, because being able to predict words another person is about to say might optimize the comprehension process and enable smooth turn-taking.

Abstract

A substantial body of literature has shown that readers and listeners often anticipate information. An open question concerns the mechanisms underlying predictive language processing. Multiple mechanisms have been suggested. One proposal is that comprehenders use event knowledge to predict upcoming words. Other theoretical frameworks propose that predictions are made based on simple word associations. In a recent EEG study, Metusalem and colleagues reported evidence for the modulating influence of event knowledge on prediction. They examined the degree to which event knowledge is activated during sentence comprehension. Their participants read two sentences, establishing an event scenario, which were followed by a final sentence containing one of three target words: a highly expected word, a semantically unexpected word that was related to the described event, or a semantically unexpected and event-unrelated word (see Figure, for an example). Analyses of participants’ ERPs elicited by the target words revealed a three-way split with regard to the amplitude of the N400 elicited by the different types of target: the expected targets elicited the smallest N400, the unexpected and event-unrelated targets elicited the largest N400. Importantly, the amplitude of the N400 elicited by the unexpected but event-related targets was significantly attenuated relative to the amplitude of the N400 elicited by the unexpected and event-unrelated targets. Metusalem et al. concluded that event knowledge is immediately available to constrain on-line language processing. Based on a post-hoc analysis, the authors rejected the possibility that the results could be explained by simple word associations. In the present study, we addressed the role of simple word associations in discourse comprehension more directly. Specifically, we explored the contribution of associative priming to the graded N400 pattern seen in Metusalem et al’s study. We conducted two EEG experiments. In Experiment 1, we reran Metusalem and colleagues’ context manipulation and closely replicated their results. In Experiment 2, we selected two words from the event-establishing sentences which were most strongly associated with the unexpected but event-related targets in the final sentences. Each of the two associates was then placed in a neutral carrier sentence. We controlled that none of the other words in these carrier sentences was associatively related to the target words. Importantly, the two carrier sentences did not build up a coherent event. We recorded EEG while participants read the carrier sentences followed by the same final sentences as in Experiment 1. The results showed that as in Experiment 1 the amplitude of the N400 elicited by both types of unexpected target words was larger than the N400 elicited by the highly expected target. Moreover, we found a global tendency towards the critical difference between event-related and event-unrelated unexpected targets which reached statistical significance only at parietal electrodes over the right hemisphere. Because the difference between event-related and event-unrelated conditions was larger when the sentences formed a coherent event compared to when they did not, our results suggest that associative priming alone cannot account for the N400 pattern observed in our Experiment 1 (and in the study by Metusalem et al.). However, because part of the effect remained, probably due to associative facilitation, the findings demonstrate that during discourse reading both event knowledge activation and simple word associations jointly contribute to the prediction process. The results highlight that multiple mechanisms underlie predictive language processing.

Abstract

Prominent theories of predictive language processing assume that language production processes are used to anticipate upcoming linguistic input during comprehension (Dell & Chang, 2014; Pickering & Garrod, 2013). Here, we explore the converse case: Does a task set including production in addition to comprehension encourage prediction, compared to a task only including comprehension? To test this hypothesis, we conducted a cross-modal naming experiment (Experiment 1) including an object naming task and a self-paced reading experiment (Experiment 2) that did not include overt production. We used the same predictable (N = 40) and non-predictable (N = 40) sentences in both experiments. The sentences consisted of a fixed agent, a transitive verb and a predictable or non-predictable target word (The man drinks a beer vs. The man buys a beer). Most of the empirical work on prediction used sentences in which the target words were highly predictable (often with a mean cloze probability > .8) and thus it is little surprising that participants engaged in predictive language processing very easily. In the current sentences, the mean cloze probability in the predictable sentences was .39 (ranging from .06 to .8; zero in the non-predictable sentences). If comprehenders are more likely to engage in predictive processing when the task set involves production, we should observe more pronounced effects of prediction in Experiment 1 as compared to Experiment 2. If production does not enhance prediction, we should observe similar effects of prediction in both experiments. In Experiment 1, participants (N = 54) listened to recordings of the sentences which ended right before the spoken target word. Coinciding with the end of the playback, a picture of the target word was shown which the participants were asked to name as fast as possible. Analyses of their naming latencies revealed a statistically significant naming advantage of 106 ms on predictable over non-predictable trials. Moreover, we found that the objects’ naming advantage was predicted by the target words’ cloze probability in the sentences (r = .411, p = .016). In Experiment 2, the same sentences were used in a self-paced reading experiment. To allow for testing of potential spill-over effects, we added a neutral prepositional phrase (buys a beer from the bar keeper/drinks a beer from the shop) to each sentence. Participants (N = 54) read the sentences word-by-word, advancing by pushing the space bar. On 30% of the trials, comprehension questions were used to keep up participants' focus on comprehending the sentences. Analyses of participants’ target and post-target reading times revealed numerical advantages of 6 ms and 20 ms, respectively, in the predictable as compared to the non-predictable condition. However, in both cases, this difference was not statistically reliable (t = .757, t = 1.43) and the significant positive correlation between an item’s naming advantage and its cloze probability as seen in Experiment 1 was absent (r = .037, p = .822). Importantly, the analysis of participants' responses to the comprehension questions, showed that they understood the sentences (mean accuracy = 93%). To conclude, although both experiments used the same sentences, we observed effects of prediction only when the task included production. In Experiment 2, no evidence for anticipation was found although participants clearly understood the sentences and the method has previously been shown to be sensitive to measure prediction effects (Van Berkum et al., 2005). Our results fit with a recent study by Gollan et al. (2011) who found only a small processing advantage of predictive over non-predictive sentences in reading (using highly predictable sentences with a cloze probability > . 87) but a strong prediction effect when participants read the same sentences and carried out an additional object naming task (see also Griffin & Bock, 1998). Taken together, the studies suggest that the comprehenders' task set exerts a powerful influence on the likelihood and magnitude of predictive language processing. When the task set involves language production, as is often the case in natural conversation, comprehenders might engage in prediction to a stronger degree than in pure comprehension tasks. Being able to predict words another person is about to say might optimize the comprehension process and enable smooth turn-taking.

Abstract

An important issue in current psycholinguistics is the relationship between the production and comprehension systems. It has been argued that these systems are tightly linked, and that, in particular, listeners use the speech production system to predict upcoming content. We tested this view using a novel version of the visual world paradigm. Participants heard mathematical equations and looked at a clock face showing the numbers 1 to 12. On alternating trials they either heard a complete equation (3+8=11) or they heard the first part (3+8) and had to produce the solution (11, target hereafter) themselves. Participants were encouraged to look at the relevant numbers throughout the trial. On listening trials, the participants typically looked at the target before the onset of target name, and on speaking trials they typically looked at the target before naming it. However, the timing of the looks to the targets was slightly different, with participants looking earlier at the target when they had to speak themselves than when they listened. This suggests that predicting during listening and planning to speak are indeed very similar but not identical. The further methodological and theoretical consequences of the study will be discussed.

Abstract

When processing visual objects, we integrate visual, linguistic and spatial information to form an episodic trace. Re-activating one aspect of the episodic trace of an object re-activates the entire bundle making all integrated information available. Using the blank screen paradigm [1], researchers observed that upon processing spoken linguistic input, participants tended to make eye movements on a blank screen, fixating locations that were previously occupied by objects mentioned in the linguistic utterance or were related. Ferreira and colleagues [2] suggested that 'looking at nothing' facilitated memory retrieval. However, this claim lacks convincing empirical support. In Experiment 1, Dutch participants looked at four-object-displays. Three objects were related to a spoken target word. Given the target word 'beker' (beaker), the display featured a phonological (a bear), a shape (a bobbin), a semantic (a fork) competitor, and an unrelated distractor (an umbrella). Participants were asked to name the objects as fast as possible. Subsequently, the objects disappeared. Participants fixated the center of the screen and listened to the target word. They had to carry out a semantic judgment task (indicating in which position an object had appeared that was semantically related to the objects) or a visual shape similarity judgment (indicating the position of the object similar in shape to the target). In both conditions, we observed that participants re-fixated the empty target location before responding. The set-up of Experiment 2 was identical except that we asked participants to maintain fixating the center of the screen while listening to the spoken word and responding. Performance accuracy was significantly lower in Experiment 2 than in Experiment 1. The results indicate that memory retrieval for objects is impaired when participants are not allowed to look at relevant, though empty locations. [1] Altmann, G. (2004). Language-mediated eye movements in the absence of a visual world: the 'blank screen paradigm'. Cognition, 93(2), B79-B87. [2] Ferreira, F., Apel, J., & Henderson, J. M. (2008). Taking a new look at looking at nothing. Trends Cogn Sci, 12(11), 405-410.

Abstract

Four eye-tracking experiments investigated the impact of the nature of the visual environment on the likelihood of word-object mapping taking place at a phonological level of representation during languagemediated visual search. Dutch participants heard single spoken target words while looking at four objects embedded in displays of different complexity and were asked to indicate the presence or absence of the target object. During filler trials the target objects were present, but during experimental trials they were absent and the display contained various competitor objects. For example, given the target word 'beaker', the display contained a phonological (a beaver, bever), a shape (a bobbin, klos), a semantic (a fork, vork) competitor, and an unrelated distractor (an umbrella, paraplu). When objects were embedded in semi-realistic scenes including four human-like characters (Experiment 1, 3, and 4a), there were no biases in looks to phonological competitors even when the objects' contours were highlighted (Experiment 3) and an object naming task was administered right before the eye-tracking experiment (Experiment 4a). In all three experiments however we observed evidence for inhibition in looks to phonological competitors, which suggests that the phonological forms of the objects had been retrieved. When objects were presented in simple four-object displays (Experiments 2 and 4b) there were clear attentional biases to phonological competitors replicating earlier research (Huettig & McQueen, 2007). These findings suggest that phonological word-object mapping is contingent upon the nature of the visual environment and add to a growing body of evidence that the nature of our visual surroundings induces particular modes of processing during language-mediated visual search. References Huettig, F., & McQueen, J. M. (2007). The tug of war between phonological, semantic and shape information in language-mediated visual search. Journal of Memory and Language, 57(4), 460-482. doi: 10.1016/j.jml.2007.02.001