It is well-established that listeners use lexical stress cues to recognize words when listening to their fellow countrymen in languages like Dutch or Spanish. Speakers of fixed-stress languages like Hungarian, however, often fail to express lexical stress correctly when speaking Dutch or Spanish. The present study examined whether Dutch listeners can adapt their perception to a Hungarian speaker’s marking of Dutch lexical stress. Native Dutch participants first listened to a short story spoken by a Hungarian learner of Dutch. One version of the story contained only words with initial stress (e.g., EEKhoorn, ‘squirrel’), the other version also contained words with stress on the second syllable (e.g., kalKOEN, ‘turkey’). The Hungarian speaker, however, erroneously marked the unstressed initial syllables with high pitch, a possible stress cue in Dutch. Subsequently, listeners’ eye movements to visually presented words were tracked while listening to the same speaker producing a new set of words. Results showed that listeners who had heard examples of non-canonically marked unstressed initial syllables were better at recognizing words when the visual target, a phonological competitor overlapping segmentally on the first syllable, or both had non-initial stress. These listeners thus had learned to interpret the speaker’s unusual use of high pitch correctly. On word pairs where target and competitor were initially stressed none of the listener groups showed an advantage. This suggests that listeners tune into speaker-specific phonetic detail not only for the recognition of the words' segmental make-up but also for the use of suprasegmental information such as lexical stress.