Presentations

Abstract

It is evident that speakers can freely vary stylistic features of their speech, such as speech rate, but how they accomplish this has hardly been studied, let alone implemented in a formal model of speech production. Much as in walking and running, where qualitatively different gaits are required cover the gamut of different speeds, we might predict there to be multiple qualitatively distinct configurations, or ‘gaits’, in the speech planning system that speakers must switch between to alter their speaking rate or style. Alternatively, control might involve continuous modulation of a single ‘gait’. We investigate these possibilities by simulation of a connectionist computational model which mimics the temporal characteristics of observed speech. Different ‘regimes’ (combinations of parameter settings) can be engaged to achieve different speaking rates.

The model was trained separately for each speaking rate, by an evolutionary optimisation algorithm. The training identified parameter values that resulted in the model to best approximate syllable duration distributions characteristic of each speaking rate.

In one gait system, the regimes used to achieve fast and slow speech are qualitatively similar, but quantitatively different. In parameter space, they would be arranged along a straight line. Different points along this axis correspond to different speaking rates. In a multiple gait system, this linearity would be missing. Instead, the arrangement of the regimes would be triangular, with no obvious relationship between the regions associated with each gait, and an abrupt shift in parameter values to move from speeds associated with ‘walk-speaking’ to ‘run-speaking’.

Our model achieved good fits in all three speaking rates. In parameter space, the arrangement of the parameter settings selected for the different speaking rates is non-axial, suggesting that ‘gaits’ are present in the speech planning system.

Abstract

That speakers can vary their speaking rate is evident, but how they accomplish this has
hardly been studied. The effortful experience of deviating from one's preferred speaking rate
might result from shifting between different regimes (system configurations) of the speech
planning system. This study investigates control over speech rate through simulations of a
new connectionist computational model of the cognitive process of speech production, derived
from Dell, Burger and Svec’s (1997) model to fit the temporal characteristics of observed
speech. We draw an analogy from human movement: the selection of walking and running
gaits to achieve different movement speeds. Are the regimes of the speech production system
arranged into multiple ‘gaits’ that resemble walking and running?
During training of the model, different parameter settings are identified for different speech
rates, which can be conflated with the regimes of the speech production system. The
parameters can be considered to be dimensions of a high-dimensional ‘regime space’, in
which different regimes occupy different parts of the space.
In a single gait system, the regimes are qualitatively similar, but quantitatively different.
They are arranged along a straight line through regime space. Different points along this axis
correspond directly to different speaking rates. In a multiple gait system, the arrangement of
the regimes is more disperse, with no obvious relationship between the regions associated
with each gait.
After training, the model achieved good fits in all three speaking rates, and the parameter
settings associated with each speaking rate were different. The broad arrangement of the
parameter settings for the different speaking rates in regime space was non-axial, suggesting
that ‘gaits’ may be present in the speech planning system.

Abstract

Previous research has demonstrated that registers can differ in various ways (see Biber and Conrad 2009, for an overview).
The present study extends this finding by comparing word predictability differences between 14 speech registers ranging
from highly informal conversations to read-aloud books. To investigate word predictability differences between registers we
used statistical language models (SLMs) to compute register-specific word predictability. The reasoning behind our approach
is as follows: SLMs are sensitive to the difference between training and test sets. The accuracy of a SLM drops if the difference
between the training and test set increases. As such, SLM performance, expressed in perplexity, can be used to quantify the
similarity of different texts. We used the perplexity scores to train a register classifier and compared its performance to a
classifier trained on a random version of the corpus. We found that the speech register classifier could distinguish between the
registers perfectly (100%), while the classifier trained on the randomized corpus performed considerably worse (17%). In
addition, we validated the speech register classifier on new materials from three different corpora. These corpora were
chosen to approximate three different registers in the Spoken Dutch Corpus, namely, spontaneous conversations, news
broadcasts and read aloud books. The classifier still performed robustly (93%), which shows that the speech register classifier
is not overfitted to the Spoken Dutch Corpus. The combined results show that speech registers differ in word predictability.

Abstract

This study investigates how the comprehension of casual speech in foreign languages is affected by the phonotactic
constraints in the listener’s native language. Non-native listeners of English with different native languages heard short
English phrases produced by native speakers of English or Spanish and they indicated whether these phrases included can or
can’t. Native Mandarin listeners especially tended to interpret can’t as can. We interpret this result as a direct effect of the ban
on word-final /nt/ in Mandarin and of these listeners’ low proficiency in English. Both the native Mandarin and the native
Spanish listeners did not take full advantage of the subsegmental information in the speech signal cueing reduced can’t. This
finding is probably an indirect effect of the phonotactic constraints in their native languages: these listeners have difficulties
interpreting the subsegmental cues because these cues do not occur or have different functions in their native languages.
Dutch resembles English in the phonotactic constraints relevant to the comprehension of can’t, and native Dutch listeners
showed similar patterns in their comprehension of native and non-native English to native English listeners. This result
supports our conclusion that the major patterns in the comprehension results are driven by the phonotactic constraints in the
listeners’ native languages.
This study appeared as M. Ernestus, H. Kouwenhoven, & M. van Mulken (2017). The direct and indirect effects of the
phonotactic constraints in the listener's native language on the comprehension of reduced and unreduced word pronunciation
variants in a foreign language. Journal of Phonetics 62, 50-64.

Abstract

This study reports a detailed analysis of 159 tokens of the Dutch discourse marker eigenlijk, uttered in casual conversations.
Our data demonstrate a wide range of variation in the production of the word, ranging from trisyllabic tokens closely
resembling the word’s citation form (13% of the tokens) to phonetically minimal monosyllabic tokens consisting merely of a
vowel followed by a single obstruent consonant (36%). The full form is thus not the most frequent form. The reduced tokens
occur both in prosodically weak and strong positions, contrary to what is typically reported for reduced words. The
pronunciation variation displayed by eigenlijk is conditioned, among other factors, by the rhythm of the phrase, and shows
large differences between speakers. Importantly, a form may be reduced in one aspect, but not in another. For instance, a
bisyllabic form may be as long as a trisyllabic form and, whereas some forms still contain acoustic cues for the /l/ but not for
the fricative, this is the other way around for other forms. Generally, we found that every pronunciation variant of eigenlijk
includes two landmarks that may be considered to be the main characteristics of the word (the full vowel and a velar/uvular
consonant). These findings raise new questions about reduction, including the status of full but infrequent forms, the extent to
which reduction is an automatic process, and the role of landmarks in speech processing.
This study will appear as M. Ernestus & R. Smith (2016). Qualitative and quantitative aspects of phonetic variation in
Dutch eigenlijk. In: F. Cangemi, M. Clayards, O. Niebuhr, B. Schuppler, & M. Zellers (eds.), Rethinking reduction. Berlin/Boston:
De Gruyter Mouton.

Abstract

While most researchers now assume that the mental lexicon contains both abstract representations and exemplars, exemplar
research has barely been applied to second language (L2) learners.
We investigated the reduction phenomenon of high vowel devoicing in casual French: in a word like la cité, the /i/ can
be devoiced as voicing fails to be reestablished in time after the /s/. Our research question was: Can L2 learners show
exemplar effects (i.e. a target matching its prime is responded to faster than a mismatching target) in a lexical decision task
using devoicing? We tested 160 Dutch university students with intermediate proficiency in French.
In Exp1, we used different recordings (or tokens) for the primes and the targets. In Exp2 we used the same token (B)
for prime and target, and in Exp3 we only used token A.
We obtained exemplar effects in Exp3 but not in Exp2. Surprisingly, Exp1 yielded a significant mismatch result: the
voiced tokens were answered to significantly faster when they were primed by the devoiced tokens than the voiced ones, and
conversely.
Only token A, but not token B, as prime triggered significant matching or mismatching effects. The encoding of the
prime thus seems to be important for such effects. However, it is hard to imagine why would token A prime a mismatching
token B, but not a matching token B. Token B as target must thus be responsible for the mismatch effect found in Exp1,
although, it is unclear by which mechanism. It is clear, however, that the role of exemplars in speech comprehension is
probably much more limited than currently assumed.

Abstract

Native listeners generally understand reduced forms effortlessly, in contrast to learners of a language. We investigated which
mechanisms may underlie the poor comprehension of reduction by advanced learners. We focused on how natives and
learners process schwa reduction in French nouns (e.g., /ʀkɛ̃/ for /ʀəkɛ̃/ requin ‘shark’) occurring in sentences.
Participants performed a passive listening visual world task. They heard a sentence with either a reduced or a full
word form together with pictures representing the target word (e.g., fenêtre /fǝnɛtʁ/‘window’), a phonological competitor
(e.g., fourchette /fuʀʃɛt/ ‘fork’), and two neutral distractors. Eye movements and EEG were recorded simultaneously.
The EEG data show no N400 effect of reduction in the natives. Natives seem to activate the representations of reduced
forms as easily as unreduced forms. Unlike natives, non-natives only showed an N400 for unreduced, but not for reduced
forms. This suggests that non-natives have not activated the meaning of reduced forms. The eye tracking data reveal that the
non-natives considered competitors more seriously and for a longer stretch of time than the natives. Interestingly, when the
non-natives heard a reduced target, it was mainly the phonological competitor that was interfering with the identification of
the target word. Taken together, the data suggest that highly proficient learners suffer more from reduction than natives do
and that in learners reduction does not only affect semantic processing of words but also processing at the form level.

Abstract

Listeners in priming experiments generally recognize repeated words more quickly and/or more accurately if repetitions
share surface details (e.g., speaker voice) than if they do not. These exemplar effects, although not always replicated, suggest
that word forms are stored as clouds of exemplars. These effects arose mostly when participants relied on their episodic
memories (e.g., in old-new judgment tasks), suggesting that exemplar effects are driven by episodic memory (rather than the
mental lexicon). However, some old-new judgment experiments did not obtain exemplar effects. Possibly, their method
(behavioral responses) was not sensitive enough.
We tested two hypotheses in two experiments: whether exemplar effects are driven by episodic memory, and
whether they are better captured by EEG (electroencephalography). We repeated words in the same or a different voice
(match and mismatch), and participants engaged in old-new (Experiment 1) or animacy judgment (Experiment 2; only the
former task relying on episodic memory). We collected participants' behavioral responses and EEG. We predict larger
exemplar effects Experiment 1 and in the ERP data (event-related potentials: stimulus-locked brain potentials derived from
the EEG).
In the ERPs, an N400 brain response peaked remarkably higher for the match than the mismatch condition in
Experiment 1 only (an exemplar effect). Behaviorally, 'match' words received slightly more accurate responses in both
experiments (no interaction arose). In response times, match and mismatch did not differ in either experiment (a null result).
We thus only detected clear exemplar effects in our ERP data, indicating that exemplar effects which do not surface in
behavior may still be present in cognitive processing (and can be measured with a more sensitive method like EEG). The
effects only arose when participants had to use their episodic memories. This suggests that exemplar effects are driven by
episodic memory rather than the mental lexicon.

Abstract

This study investigates the influence of adverse listening conditions on the occurrence of exemplar effects in repetition
priming experiments that do not instruct participants to use their episodic memories. We carried out two lexical decision
experiments (N = 26 participants in each experiment), in which a prime and a target represented the same word type and
could be spoken by the same or a different speaker (but were always different tokens). Other than for instance an old-new
judgment task, a lexical decision task does not require participants' use of episodic memory. In Experiment 1, in which
participants listened to clear speech, we obtained no exemplar effects: participants recognized word repetitions spoken by the
same speaker equally fast as word repetitions spoken by different speakers. In Experiment 2, we introduced an adverse
listening condition by adding speech-shaped noise to the same stimuli (at pre-tested intermediate noise level of SNR +3 dB).
Exemplar effects arose in this experiment. Experiment 1 yielded on average longer response times than Experiment 2.
Importantly, this result contradicts the time-course hypothesis (McLennan & Luce, 2005), according to which exemplar
representations should only play a role when speech processing is relatively slow. Instead, our finding supports the
hypothesis that exemplar effects arise under adverse listening conditions, when participants are stimulated to use their
episodic memories in addition to their mental lexicons.

Abstract

Numerous experiments indicate that listeners retain detailed information on spoken words in memory, which may influence
subsequent speech processing. For instance, listeners are found to be quicker and/or more accurate to recognize repeated
words when the two presentations of a word share surface details (e.g., speaker voice, speech rate or phonemic realization)
than when they do not. However, in the literature as well in our lab approximately half of the experiments run did not find
these 'exemplar effects', raising questions about the nature and role of exemplar representations in speech comprehension.
We tried to establish the limits of the conditions under which exemplar effects arise. In a large number of priming
experiments, we manipulated listening circumstances (clean or noisy speech), listener background (native or non-native with
different mother tongues and proficiency levels), task (requiring participants' use of lexical knowledge or episodic memory),
type of surface variation (e.g., speaker voice, reduction), and type of word repetition (same token or not). We recorded
participants' behavior (response times and accuracy) and brain responses (with EEG and fMRI). We were able to show, for
example, that exemplar effects can come and go as a function of the size of the cloud of exemplars in non-native listeners.
Additionally, the occurrence of exemplar effects appears to relate to participants' use of episodic memory, suggesting that
exemplars are represented in episodic memory rather than the mental lexicon. Lastly, we will address open issues and
questions, including the lifespan of exemplar representations, the ecological validity of existing results and the role of
perceptual similarity for exemplar effects.

Abstract

Speech can be produced at different rates. The ability to produce faster or slower speech may be thought to result from executive control processes enlisted to modulate lexical selection and phonological encoding stages of speech planning.

This study used simulations of the model of serial order in language by Dell, Burger and Svec (1997, DBS) to characterise the strategies adopted by speakers when naming pictures at fast, medium and slow prescribed rates. Our new implementation of DBS was able to produce activation patterns that correlated strongly with observed syllable-level timing of disyllabic words from this task.

For each participant, different speaking rates were associated with different regions of the DBS parameter space. The precise placement of the speaking rates in the parameter space differed markedly between participants. Participants applied broadly the same parameter manipulation to accelerate their speech. This was however not the case for deceleration. Hierarchical clustering revealed two distinct patterns of parameter adjustment employed to decelerate speech, suggesting that deceleration is not necessarily achieved by the inverse process of acceleration. In addition, potential refinements to the DBS model are discussed.

Abstract

Over the last few decades, several computational models of word comprehension have been proposed (for example TRACE,
Shortlist, SpeM, Shortlist-B). These models share the concept of word activation, based on the match between speech input
and word representations, followed by a competition among activated words. Recently, DIANA has been developed with the
aim to simulate reaction times and judgments in lexical decision experiments. The model is built on three components:
Activation, Decision, and Execution. It takes the acoustic waveform as input, without any additional meta information about
segments or segmentation. The model allows a mapping directly from the signal onto word representations, which makes it a
tool to investigate reduction and the impact of fine phonetic details on sub-symbolic level.
As output, DIANA can produce a list of activated words and their activations, updated every 10 ms, during the
unfolding of the acoustic stimulus. It also produces estimations for reaction times in recognition experiments and in lexical
decision tasks. It has been tested on BALDEY, a large-scale Dutch lexical decision database, and on (pre-)MALD, a similar
database for North American.
In this talk we will present DIANA, its design, and discuss how its reaction time sequences are compared to sequences
of human reaction times. Finally, we will spend a few words on what DIANA cannot do (yet), and about our recent attempts to
include morphological structure.

Abstract

Auditory identity priming experiments generally show that participants respond more quickly and accurately to stimuli which
are repeated with similar phonetic characteristics (Goldinger, 1996). Usually, researchers separate matching and mismatching
priming conditions using a perceptual difference that spans multiple phonetic dimensions. For instance, speaker identity is not
only reflected in fundamental frequency (F0), but also involves a particular voice-quality and speaker-specific articulations.
Such complex variation cannot be precisely controlled in an experiment. Consequently, little is known about how acoustic
variation affects spoken word processing by increasing perceptual distance. Our study explores this relationship by defining
perceptual distance on a single acoustic dimension: F0.
Our first experiment investigated whether an F0 difference between otherwise identical primes and targets affects
participants’ response times in a lexical decision task (N=32). The F0 shift was implemented using PSOLA resynthesis
(Moulines & Charpentier, 1990) and was measured in ERB (Equivalent Rectangular Bandwidth; Greenwood, 1990) to retain
perceptually comparable intonation contours across repetitions. Statistical analysis showed that listeners responded
significantly faster to target stimuli when the F0 was identical to the corresponding primes.
In a second experiment we will assess the impact of increased perceptual distance between mismatching repetitions.
In a separate rating experiment (N=18) we determined the F0 difference that was perceptually double the size of the
difference between mismatching repetitions in Experiment 1. Data acquisition for the second priming experiment is currently
ongoing; results of the combined experiments will be presented at the workshop.

Abstract

A central goal of research in neurobiology of language is to discover the neural underpinning of concepts such as “phoneme”, “morpheme”, “word”, “lemma” and “phrase”, conditions such as “agreement” and operations such as “wh-movement”, which are defined in Linguistics. However, a large proportion of these concepts and operations originated as devices for meta-level discussions about sentences, ages before scientists started asking questions about the neural and cognitive processes that underlie the production and comprehension of utterances. At least some of the concepts may have no neural substance at all, even if they have been successfully invoked in explaining the results of psycholinguistic experiments. In the theory of speech comprehension it is hotly debated what the basic units of processing and representation are. The majority view still holds that the basic units are abstract phonemes or bundles of distinctive features, but there is increasing support for theories that take episodes or exemplars as the basic units. These antagonistic theories have in common that they remain extremelyvague about the details of the neural representations and the computations that are needed for a person to actually understand or produce a spoken utterance. If positions and claims are supported by computational models, it is virtually always so that those models operate on manually constructed discrete symbolic input representations, and the models make no claims about neurobiological plausibility. In the poster we will present the results of a large-scale behavioral experiment aimed at answering the question whether exemplars play a role in comprehension as well as in production. Participants were asked to shadow nonsense words of the form / CVVVVVV-CV-PV/ (Mitterer & Ernestus, 2008), where the vowel V in the central syllable could have normal or somewhat lengthened duration; also the voiceless plosive P that separates the second and third syllable can have normal duration or be lengthened. Native speakers of Dutch have several routes available for linking their perception to the ensuing articulation. At the perception side they may restrict processing to creating on-thefly exemplars without a representation in the form of discrete units, they might create a representation in the form of discrete phonemic units, or they might access their mental lexicon to find the most similar word (Roelofs, 2004). For each of these routes we construct plausible neural computational procedures that could be used to control the speech production process in the shadowing task. Using end-to-end computational models (i.e., models that take acoustic speech signals as input and produce audible speech as output) we simulate the chronometric data and the accuracy with which the stimuli were shadowed, in an attempt to explain differences between participants in terms of different routes. We will use the result to discuss potential discrepancies between representations and processes implied in functional (psycho)linguistic models of speech comprehension and production on the one hand and a detailed account of what is currently known about the neural processes that support auditory processing of speech signals and the production of spoken utterances. Holger Mitterer and Mirjam Ernestus (2008) The link between speech perception and production is phonological and abstract: Evidence from the shadowing task, Cognition 109, 168–173. Ardi Roelofs (2004) Error Biases in Spoken Word Planning and Monitoring by Aphasic and Nonaphasic Speakers: Comment on Rapp and Goldrick (2000), Psychological Review Vol. 111, No. 2, 561–572.

Abstract

Casual speech is characterized by variation in how a single
word is pronounced (e.g. ‚probably’ as ‚probably’ but also
as ‚probly’). We investigated which lexical representations
French listeners use for understanding reduced words
that are pronounced without their schwa (e.g. ‚fnêtre’ for
‚fenêtre’). Thirty-six French native speakers performed an
auditory lexical decision task with 44 words produced either
with or without schwa. In addition, they rated the relative
frequency of the reduced variant compared to the full
variant for each word. Results show that listeners recognize
a given variant more quickly if it has a higher relative
frequency. This suggests that these relative frequencies are
stored, which implies that the reduced variants are stored
as well. These findings complement results reported by
Ranbom & Connine (2007) for the perception of variants
of words containing /nt/ in American English and by Bürki
et al. (2010) for the production of reduced schwa in French.
Given that reduction processes give rise to many different
pronunciation variants for a given word (e.g. ‚fenêtre’,
‚fnêtre’, ‚fenêtr’, ‚fnêtr’, ‚fnêt’), our results raise the question
of how the activations of these different pronunciation
variants play a role in the selection of their common word
node.

Abstract

In alledaagse gesprekken spreken we veel woorden maar half uit. We zeggen bijvoorbeeld 'nutuulijk', 'ntuulk' of 'tuuk' voor 'natuurlijk' en we zeggen 'wes' voor 'wedstrijd' . Sommige woorden verkorten we, omdat we snel willen praten. Andere halve woorden hebben een speciale betekenis. Veel halve woorden zijn opgeslagen in het woordenboek in ons hoofd, maar we weten nog niet precies hoe. Moedertaalsprekers begrijpen halve woorden zonder moeite. Zij horen de halve woorden van jongs af aan en herkennen deze met behulp van de betekenis van de context en de precieze eigenschappen van het spraaksignaal. Tweede-taalleerders hebben daarentegen veel problemen met halve woorden. Waarom precies is nog onbekend. Onderzoek naar hoe moedertaalsprekers en tweede-taalleerders halve woorden verstaan, zal leiden tot meer kennis van de eigenschappen en ontwikkeling van het woordenboek in ons hoofd, tot betere onderwijsmethodes voor tweede-taalleerders en tot automatische spraakherkenners die ook spontane conversaties kunnen verstaan.

Abstract

In spontaneous conversations, words are often pronounced with fewer segments or syllables than in their citation forms. For example, the English word police can be pronounced as p’lice. Previous research has argued that adult speakers and listeners have stored lexical representations for at least some of these pronunciation variants in their mental lexicons (see Ernestus, in press), but it is still unknown how and when these representations are acquired. Previous studies have indicated that infants are frequently confronted with variability in speech caused by reduction processes (e.g. Bortfeld & Morgan, 2010; Shockey & Bond, 1980). In the present study, we investigated the perception of reduced pronunciation variants by 20- to 28-month-olds acquiring Canadian English. We focused on a common reduction process in English: schwa deletion in unstressed syllables. Children were taught to associate bisyllabic nonsense words containing a schwa in the initial syllable (e.g. satoom) with new objects. They were then tested in a preferential looking procedure on tokens of these words with schwa present, with deleted schwa and with a mispronunciation of the first consonant of the second syllable. Results will be discussed with regards to the formation of representations for reduced speech by young language learners

Abstract

Words are often shorter and contain fewer segments in casual than in formal speech. For instance, in casual Dutch, the word 'of course' is often reduced to , or even . This type of pronunciation variation is generally ascribed to general mechanisms of speech production. We investigated whether it is also conditioned by pragmatic factors. We studied the acoustic characteristics of 177 tokens of Dutch and 184 tokens of Dutch 'actually', extracted from spontaneous speech corpora. We classified their Turn Constructional Units (TCUs) as constituting responses to prior TCUs or as initiating new conversational topics. We hypothesized that and are more reduced in initiating TCUs, since in these TCUs they typically convey that aspects of the turn contain old information and violate a norm of conversation. This hypothesis was supported by our data. and are shorter in duration and in number of syllables in initiating than in responsive TCUs. Nevertheless, the syllable of is more often present in initiating than in responsive TCUs. These results show that pragmatic factors condition degree and type of reduction. Psycholinguistic models of speech production have to account for interactions between pragmatics and general production mechanisms.

Abstract

In everyday speech, words are often produced with reduced pronunciation variants, in which segments are shorter or completely absent. We investigated whether word-final /t/ reduction in Dutch past-participles is affected by the ease of planning of the preceding word, and whether previously found morphological effects may actually be planning effects. We analyzed presence of 1369 /t/s and their durations in two speech corpora representing three speech styles. /t/ appeared more often absent and shorter if the past-participle followed a word that is highly predictable given the preceding context. Furthermore, /t/ was more reduced in irregular pastparticiples with a high frequency relative to the frequencies of the other inflected forms in the verbal paradigm, that is, in past-participles that can be selected more easily, and thus planned more quickly. Both effects were more pronounced in more spontaneous speech styles, which is as expected if the effects are driven by speech planning. These planning effects have to be incorporated in psycholinguistic models of speech production. Abstractionist models could, for instance, adapt the articulation level. Exemplar-based models have to incorporate planning as a factor influencing the choice of exemplar, or assume an articulation level that can modify the selected exemplar.