Human speech recognition transforms a continuous acoustic signal into categorical linguistic units, by aggregating information that is distributed in time. It has been suggested that this kind of information processing may be understood through the computations of a Recurrent Neural Network (RNN) that receives input frame by frame, linearly in time, but builds an incremental representation of this input through a continually evolving internal state. While RNNs can simulate several key behavioral observations about human speech and language processing, it is unknown whether RNNs also develop computational dynamics that resemble human neural speech processing. Here we show that the internal dynamics of long short-term memory (LSTM) RNNs, trained to recognize speech from auditory spectrograms, predict human neural population responses to the same stimuli, beyond predictions from auditory features. Variations in the RNN architecture motivated by cognitive principles further improved this predictive power. Specifically, modifications that allow more human-like phonetic competition also led to more human-like temporal dynamics. Overall, our results suggest that RNNs provide plausible computational models of the cortical processes supporting human speech recognition.

ABSTRACT Compensation for coarticulation (CfC), a context effect in which the articulatory characteristics of one segment influence the perception of a neighboring segment [1], has been a matter of considerable debate between proponents of gestural [2] and auditory theories of speech perception [3]. We set out to distinguish the two accounts by using non-native liquids (Tamil with American English listeners) that have distinct articulatory and acoustic characteristics from the native phoneme categories to which they are assimilated. We report three experiments that show that the auditory contrast account of CfC cannot explain compensatory effects with our non-native stimuli. We argue that these context effects reflect perceptual compensation for coarticulation, as predicted on a gestural account, but discuss problems for both theories. 1. INTRODUCTION Mann [1] reported that classification of members of a [da-ga] continuum shifts toward more [ga] responses following the syllable [al] and more [da] responses following [ar]. The typical explanation for this finding is that when speakers must transition from relatively front ([l]) to back ([g]) places of articulation, they will be unlikely to reach the canonical place of articulation for [g]. Thus, after [l], [g] is likely to be produced farther forward than usual due to coarticulation. After [r], with a back place of articulation, [d] is likely to be produced farther back than usual. CfC has been used to argue for gestural theories, and Fowler’s

It is often reported that for non-native listeners of a language, some native speakers' productions of non-native contrasts are easier to understand than others' (e.g., [1]).However, these effects are not well-understood, as acoustic correlates to the effects have proven difficult to establish.We report analyses of subject differences and acoustic measurements which may help to describe the acoustic phenomena underlying one class of talker effects that we have reported previously; specifically, the interaction of talker and talker condition (the number of talkers heard within a block of trials --one or several) [2].Correlations between response measures and acoustic measures suggest that when stimuli from several talkers are mixed randomly in a block of trials, subjects without well-formed categories for /r/ and /l/ attempt to use the duration of the initial steady-state portion of an /r/ or /l/ stimulus (an unreliable cue) for categorization, whereas native speakers use F3 [6].It also appears that they use this cue to establish criteria for "R"-"L" decisions, which they apply to the overall range of durations across all talkers in one block of trials.