In this paper, we address the generalization of deep neural network (DNN) based speech enhancement to unseen noise conditions for the case that training data is limited in size and diversity. To gain more insights, we analyze the generalization with respect to (1) the size and diversity of the training data, (2) different network architectures, and (3) the chosen features. To address (1), we train networks on the Hu noise corpus (limited size), the CHiME 3 noise corpus (limited diversity) and also propose a large and diverse dataset collected based on freely available sounds. To address (2), we compare a fully-connected feed-forward and a long short-term memory (LSTM) architecture. To address (3), we compare three input features, namely logarithmized noisy periodograms, noise aware training (NAT) and the proposed signal-to-noise ratio based noise aware training (SNR-NAT). We confirm that rich training data and improved network architectures help DNNs to generalize. Furthermore, we show via experimental results and an analysis using t-distributed stochastic neighbor embedding (t-SNE) that the proposed SNR-NAT features yield robust and level independent results in unseen noise even with simple network architectures and when trained on only small datasets, which is the key contribution of this paper.

In this work, the generalization of speech enhancement algorithms based on deep neural networks (DNNs) for training datasets that differ in size and diversity is analyzed. For this, we compare noise aware training (NAT) features and signal-to-noise ratio (SNR) based noise aware training (SNR-NAT) features. NAT appends an estimate of the noise power spectral density (PSD) to a noisy periodogram input feature, whereas SNR-NAT uses the noise PSD for normalization. We show that the Hu noise corpus (limited size) and the CHiME 3 noise corpus (limited diversity) may result in DNNs which do not generalize well to unseen noises. We construct a large and diverse dataset from freely available data and show that it helps DNNs to generalize. However, we also show that with SNR-NAT features, the trained models are more robust even if a small or less diverse training set is employed. Using t-distributed stochastic neighbor embedding (t-SNE), we demonstrate that using SNR-NAT both the features and the resulting internal representation of the DNN are less dependent on the background noise which facilitates the generalization to unseen noise types.

The majority of multichannel speech enhancement algorithms are two-step procedures that first apply a linear spatial filter, a so-called beamformer, and combine it with a single-channel approach for postprocessing.However, the serial concatenation of a linear spatial filter and a postfilter is not generally optimal in the minimum mean square error (MMSE) sense for noise distributions other than a Gaussian distribution.Rather, the MMSE optimal filter is a joint spatial and spectral nonlinear function.While estimating the parameters of such a filter with traditional methods is challenging, modern neural networks may provide an efficient way to learn the nonlinear function directly from data.To see if further research in this direction is worthwhile, in this work we examine the potential performance benefit of replacing the common two-step procedure with a joint spatial and spectral nonlinear filter.We analyze three different forms of non-Gaussianity: First, we evaluate on super-Gaussian noise with a high kurtosis.Second, we evaluate on inhomogeneous noise fields created by five interfering sources using two microphones, and third, we evaluate on realworld recordings from the CHiME3 database.In all scenarios, considerable improvements may be obtained.Most prominently, our analyses show that a nonlinear spatial filter uses the available spatial information more effectively than a linear spatial filter as it is capable of suppressing more than D -1 directional interfering sources with a D-dimensional microphone array without spatial adaptation.

For enhancing noisy signals, machine-learning based single-channel speech enhancement schemes exploit prior knowledge about typical speech spectral structures. To ensure a good generalization and to meet requirements in terms of computational complexity and memory consumption, certain methods restrict themselves to learning speech spectral envelopes. We refer to these approaches as machine-learning spectral envelope (MLSE)-based approaches. In this paper, we show by means of theoretical and experimental analyses that for MLSE-based approaches, super-Gaussian priors allow for a reduction of noise between speech spectral harmonics which is not achievable using Gaussian estimators such as the Wiener filter. For the evaluation, we use a deep neural network based phoneme classifier and a low-rank nonnegative matrix factorization framework as examples of MLSE-based approaches. A listening experiment and instrumental measures confirm that while super-Gaussian priors yield only moderate improvements for classic enhancement schemes, for MLSE-based approaches super-Gaussian priors clearly make an important difference and significantly outperform Gaussian priors.

First-order recursive smoothing filters using a fixed smoothing constant are in general unbiased estimators of the mean of a random process. Due to their efficiency in terms of memory consumption and computational complexity, they are of high practical relevance and are also often used to track the first-order moment of nonstationary random processes. However, in single-channel speech-enhancement applications, e.g., for the estimation of the noise power spectral density, an adaptively changing smoothing factor is often employed. Here, the adaptivity is used to avoid speech leakage by raising the smoothing factor when speech is likely to be present. In this paper, we investigate the properties of adaptive first-order recursive smoothing factors applied to noise power spectral density estimators. We show that in contrast to a smoothing with fixed smoothing factors, adaptive smoothing is in general biased. We propose different methods to quantify and to compensate for the bias. We demonstrate that the proposed correction methods reduce the estimation error and increases the perceptual evaluation of speech quality scores in a speech enhancement framework.

Due to the low computational complexity and the low memory consumption, first-order recursive smoothing is a technique often applied to estimate the mean of a random process. For instance, recursive smoothing is used in noise power estimators where adaptively changing smoothing factors are used instead of fixed ones to prevent the speech power from leaking into the noise estimate. However, in general, the usage of adaptive smoothing factors leads to a biased estimate of the mean. In this paper, we propose a novel method to correct the bias evoked by adaptive smoothing factors. We compare this method to a recently proposed compensation method in terms of the log-error distortion using real world signals for two noise power estimators. We show that both corrections reduce the distortion measure in noisy speech while the novel method has the advantage that no iteration is required for determining the correction factor.

The robustness of speech recognizers towards noise can be increased by normalizing the statistical moments of the Mel-frequency cepstral coefficients (MFCCs), e. g. by using cepstral mean normalization (CMN) or cepstral mean and variance normalization (CMVN). The necessary statistics are estimated over a long time window and often, a complete utterance is chosen. Consequently, changes in the background noise can only be tracked to a limited extent which poses a restriction to the performance gain that can be achieved by these techniques. In contrast, algorithms recently developed for single-channel speech enhancement allow to track the background noise quickly. In this paper, we aim at combining speech enhancement techniques and feature normalization methods. For this, we propose to transform an estimate of the noise power spectral density to the MFCC domain, where we subtract it from the noisy MFCCs. This is followed by a conventional CMVN. For background noises that are too instationary for CMVN but can be tracked by the noise estimator, we show that this processing leads to an improvement in comparison to the sole application of CMVN. The observed performance gain emerges especially in low signal-to-noise-ratios.

In this paper, we focus on methods for estimating the a posteriori probability of a signal segment being voiced which employ a harmonic signal model. Fisher et al. [1] present two likelihood functions for voiced and unvoiced speech from which the posterior probability can be derived. However, due to the chosen models, the a posteriori probability of a signal segment being voiced does not go to 0 % in unvoiced speech. Thus, a novel algorithm is proposed, which incorporates the expected unvoiced speech energy and allows for obtaining low probabilities. Further, it explicitly models the statistics of the segment energy and employs a state-of-the-art noise tracker. Experiments which were conducted on the TIMIT database for different noise types and noise levels show that the proposed method results in lower over-estimation and under-estimation of the voicing probability as compared to [1].