A computational auditory scene analysis (CASA) system is described, in which sound separation according to spatial location is combined with the 'missing data' approach for automatic speech recognition. Time-frequency masks for the missing data recognizer are derived from the statistics of interaural time and level differences; these masks identify acoustic features that constitute reliable evidence of the target speech signal. It is demonstrated that this approach yields good performance in a challenging environment, in which a target voice is contaminated by another talker and reverberation. The ability of the system to generalize to source-receiver configurations that were not encountered during training is discussed

This paper describes a perceptually motivated computational auditory scene analysis (CASA) system that combines sound separation according to spatial location with the "missing data" approach for robust speech recognition in noise. Missing data time-frequency masks are created using probability distributions based on estimates of interaural time and level differences (ITD and ILD) for mixed utterances in reverberated conditions; these masks indicate which regions of the spectrum constitute reliable evidence of the target speech signal. A number of experiments compare the relative efficacy of the binaural cues when used individually and in combination. We also investigate the ability of the system to generalize to acoustic conditions not encountered during training. Performance on a continuous digit recognition task using this method is found to be good, even in a particularly challenging environment with three concurrent male talkers.

This paper describes a perceptually motivated computational auditory scene analysis (CASA) system that combines sound separation according to spatial location with 'missing data' techniques for robust speech recognition in noise. Missing data time-frequency masks are produced using cross-correlation to estimate interaural time difference (ITD) and hence spatial azimuth; this is used to determine which regions of the signal constitute reliable evidence of the target speech signal. Three experiments are performed that compare the effects of different reverberation surfaces, localisation methods and azimuth separations on recognition accuracy, together with the effects of two post-processing techniques (morphological operations and supervised learning) for improving mask estimation. Both post-processing techniques greatly improve performance; the best performance occurs using a learned mapping.

The ‘missing data ’ approach for robust speech recognition uses masks indicating which regions of an acoustic mixture provide reliable evidence of the target to be recognised. Binaural cues for spatial location were used to determine missing data masks for signals consisting of utterances from three concurrent male speakers in reverberant conditions, by deriving probability distributions from estimates of interaural time and level differences (ITD and ILD) for the mixed signals. In such a system, a decision must be made about whether the acoustic features used for decoding are selected from the left or right ear, or a combination of the two. Here, features were selected from the “better ear ” (as determined by a simple heuristic) within whole time frames, or within individual time-frequency elements. A combination of left and right ear features gave better recognition performance than using either ear alone, and the best results were obtained when selecting features within individual time-frequency elements. 1.

Abstract Unlike automatic speech recognition systems, humans canunderstand speech when other competing sounds are presentAlthough the theory of auditory scene analysis (ASA)may helpto explain this ability, some perceptual experiments show fu-sion of the speech signal under circumstances in which ASAprinciples might be expected to cause segregation. We pro-pose a model of multi-resolution ASA that uses both high- andlow- resolution representations of the auditory signal in paral-lel in order to resolve this conflict. The use of parallel repre-sentationsreduces variabilityforpattern-matching whileretain-ing the ability to identify and segregate low-level features ofthe signal. An important feature of the model is the assump-tion that features of the auditory signal are fused together un-less there is good reason to segregate them. Speech is recog-nised by matching the low-resolution representation to previ-ously learned speech templates without prior segregation of thesignal into separate perceptual streams; this contrasts with theappr oach gener a l l y used by comput at i onal m odels of A S A . Wedescribe an implementation of the multi-resolution model, us-ing hidden Markov models, that illustrates the feasibility ofthis approach and achieves much higher identification perfor-mance than standard techniques used for computer recognitionof speech mixed with other sounds.

A commonly held view of auditory scene analysis is that complex auditory environments are segregated into separate perceptual streams using primitive cues that can be attended to separately. We argue that this view is inconsistent with the majority of perceptual data reported in the literature and propose an alternative model that is based on a primary, low resolution signal representation used in a passive pattern matching stage, augmented by secondary, high resolution representations that can be used in an active pattern matching stage to formulate hypotheses about the auditory scene.