In this paper, we investigate a closed-loop auditory model and explore its potential as a feature representation for speech recognition. The closed-loop representation consists of an auditory-based, efferent-inspired feedback mechanism that regulates the operating point of a filter bank, thus enabling it to dynamically adapt to changing background noise. With dynamic adaptation, the closed-loop representation demonstrates an ability to compensate for the effects of noise on speech, and generates a consistent feature representation for speech when contaminated by different kinds of noises. Our preliminary experimental results indicate that the efferent-inspired feedback mechanism enables the closed-loop auditory model to consistently improve word recognition accuracies, when compared with an open-loop representation, for mismatched training and test noise conditions in a connected digit recognition task. Index Terms: efferent, auditory model, feature extraction

Sensory processing is associated with gamma frequency oscillations (30-80 Hz) in sensory cortices. This raises the question whether gamma oscillations can be directly involved in the representation of time-varying stimuli, including stimuli whose time scale is longer than a gamma cycle. We are interested in the ability of the system to reliably distinguish different stimuli while being robust to stimulus variations such as uniform time-warp. We address this issue with a dynamical model of spiking neurons and study the response to an asymmetric sawtooth input current over a range of shape parameters. These parameters describe how fast the input current rises and falls in time. Our network consists of inhibitory and excitatory populations that are sufficient for generating oscillations in the gamma range. The oscillations period is about one-third of the stimulus duration. Embedded in this network is a subpopulation of excitatory cells that respond to the sawtooth stimulus and a subpopulation of cells that respond to an onset cue. The intrinsic gamma oscillations generate a temporally sparse code for the external stimuli. In this code, an excitatory cell may fire a single spike during a gamma cycle, depending on its tuning properties and on the temporal structure of the specific input; the identity of the stimulus is coded by the list of excitatory cells that fire during each cycle. We quantify the properties of this representation in a series of simulations and show that the sparseness of the code makes it robust to uniform warping of the time scale. We find that resetting of the oscillation phase at stimulus onset is important for a reliable representation of the stimulus and that there is a tradeoff between the resolution of the neural representation of the stimulus and robustness to time-warp.

In the past few years, objective quality assessment models have become increasingly used for assessing or monitoring speech and audio quality. By measuring perceived quality on an easily-understood subjective scale, such as listening quality (excellent, good, fair, poor, bad), these methods provide a quick and repeatable way to estimate customer experience. Typical applications include audio quality evaluation, selection of codecs or other equipment, and measuring the quality of telephone networks. To introduce this special issue, this paper provides an overview of the field, outlining the main approaches to intrusive, nonintrusive and parametric models and discussing some of their limitations and areas of future work

The performance of large-vocabulary automatic speech recognition (ASR) systems deteriorates severely in mismatched training and testing conditions. Signal processing techniques based on the human auditory system have been proposed to improve ASR performance, especially under adverse acoustic conditions. The paper compares one such scheme, the ensemble interval histogram (EIH), with the conventional mel cepstral analysis (MEL). These two spectral feature extraction methods were implemented as front ends to a state-of-the-art continuous speech recognizer and evaluated on the TIMIT database (male). To characterize the influence of signal distortion on the representation of different sounds, phone classification experiments were conducted for three acoustic conditions-clean speech, speech through a telephone channel and speech under room reverberations (the last two are simulations). Classification was performed for static features alone and for static and dynamic features, to observe the relative contribution of time derivatives. The performance is displayed as percentage of phones correctly classified. Confusion matrices were also derived from phone classification to provide diagnostic information.

Auditory models that are capable of achieving human performance in tasks related to speech perception would provide a basis for realizing effective speech processing systems. Saving bits in speech coders, for example, relies on a perceptual tolerance to acoustic deviations from the original speech. Perceptual invariance to adverse signal conditions (noise, microphone and channel distortions, room reverberations) and to phonemic variability (due to nonuniqueness of articulatory gestures) may provide a basis for robust speech recognition. A state-of-the-art auditory model that simulates, in considerable detail, the outer parts of the auditory periphery up through the auditory nerve level is described. Speech information is extracted from the simulated auditory nerve firings, and used in place of the conventional input to several speech coding and recognition systems. The performance of these systems improves as a result of this replacement, but is still short of achieving human performance. The shortcomings occur, in particular, in tasks related to low bit-rate coding and to speech recognition. Since schemes for low bit-rate coding rely on signal manipulations that spread over durations of several tens of ms, and since schemes for speech recognition rely on phonemic/articulatory information that extend over similar time intervals, it is concluded that the shortcomings are due mainly to perceptually related rules over durations of 50-100 ms. These observations suggest a need for a study aimed at understanding how auditory nerve activity is integrated over time intervals of that duration. The author discusses preliminary experimental results that confirm human usage of such integration, with different integration rules for different time-frequency regions depending on the phoneme-discrimination task.

The author describes the closed-loop ensemble-interval-histogram (EIH) model. It is constructed by adding a feedback system to the former, open-loop, EIH model (Ghitza, Computer, speech and Language, 1(2), pp.109-130, Dec. 1986). While the open-loop EIH is a computational model based upon the ascending path of the auditory periphery, the feedback system is motivated by the descending path and attempts to capture the functional contribution of the neural feedback mechanism in the auditory periphery. The capability of the resultant closed-loop EIH to preserve relevant phonetic information in quite and in noisy acoustic environments was measured quantitatively using the model as a front-end to a dynamic time warping (DTW), speaker-dependent, isolated-word recognizer. The database consisted of a 39 word alpha-digit vocabulary spoken by two male speakers, in different levels of additive white noise. In the absence of noise the recognition scores based on the close-loop EIH are comparable to those based on the open-loop EIH. However, recognition performance based on the closed-loop EIH does not decline as much as with the open-loop EIH at low signal-to-noise ratios. At SNR of 6 dB, the average correct-recognition score with the closed-loop EIH is 82%. This is equivalent to the recognition score obtained with the open-loop EIH at 10 dB SNR, a gain of 4 dB.>

In a previous report (Ghitza, 1987, [1]) we described a computational model based upon the temporal characteristics of the information in the auditory nerve fiber firing patterns, which produced an "auditory" spectral representation (the EIH) of the input signal. We also demonstrated that for speech recognition purposes, the EIH is more robust against noise compared to the traditional Fourier power spectrum. This paper reports on the first step towards understanding the role of different parameters in the EIH in achieving this performance. Both, the Fourier power spectrum measurement and the EIH measurement can be partitioned into two parts, a filter-bank followed by feature analyzer. In the Fourier power spectrum, the filter bank consists of uniformly shaped Hamming filters and the analyzer is based on power measurements. In the EIH, the filter bank consists of the cochlear filters and the analyzer is based on timing-synchrony measurements. The present study examines the relative importance of the filter-bank properties as compared to the analysis principle. For this purpose a modified EIH model has been created in which the cochlear filters have been replaced by the uniformly shaped Hamming filters. The output of the filter bank is processed by the timing-synchrony analyzer, as with the original EIH. The modified EIH and the Fourier power spectrum differs, therefor, only in the kind of analysis performed on the filter bank output. The modified EIH has been used as a front-end to a Dynamic Time Warp (DTW), using the same set-up as in Ghitza, 1987, [1]. A speaker dependent, isolated word recognition test has been conducted, on a database consisted of a 39 word alpha-digits vocabulary spoken by two male and two female speakers, in different levels of additive white noise. Compared to the Fourier-based front-end, the recognition scores have been slightly improved in clean environment but significantly improved in noisy environments. Furthermore, compared to the original EIH, the recognition scores have also been improved, both in clean and in noisy environments. These results demonstrate that the timing-synchrony measurement is significantly more robust against noise compared to the power measurement. They also show that the robustness is due to the timing-synchrony analyzer and not to the unique shape of the cochlear filters.

Traditional speech analysis/synthesis techniques are designed to produce synthesized speech with a spectrum (or waveform) which is as close as possible to the original. It is suggested, instead, to match the in-synchrony-bands spectrum measures (Ghitza, ICASSP-85, Tampa FL., Vol.2, p. 505) of the synthetic and the original speech. This concept has been used in conjunction with a sinusoidal representation type of speech analysis/synthesis (McAulay and Quatieri, Lincoln Laboratory Technical Report 693, May 1985). Based on informal listening, the resulting speech is natural (with some tonal artifact) and highly intelligible both in quiet and noisy environments. The same performance is obtained with two overlapping superposed speech waveforms, music waveforms, and speech in musical background. These results demonstrate the adequacy of the in-synchrony-bands measure in selecting the perceptually meaningful frequency regions of the stimulus spectra. Moreover, the inherent dominance property of this measure significantly reduces the number of sinusoidal components needed for synthesis by approximately 70 percent, offering the potential for reduced data-rate.

A speech spectrum intensity measure based on temporal non-place modeling of the cat's auditory nerve firing patterns is introduced where the spectrum intensity values are estimated using timing-synchrony measurements only. The ability of this measure to serve as a speech information carrier was tested psychoacoustically, by integrating the proposed measure into a buzz-hiss type analysis-synthesis vocoder. Informal listening suggests that considerable speech information is preserved when this new measure is used to replace the spectrum analyzer portion of the vocoder.