In this letter, we introduce a novel approach for nonlinear acoustic echo cancellation. The proposed approach uses the principle of transfer learning to train a neural network that approximates the nonlinear function responsible for the nonlinear distortions and generalizes this network to different acoustic conditions. The topology of the proposed network is inspired by the conventional adaptive filtering approaches for nonlinear acoustic echo cancellation. The network is trained to model the nonlinear distortions using the conventional error backpropagation algorithm. In deployment, and in order to account for any variation or discrepancy between training and deployment conditions, only a subset of the network's parameters is adapted using the significance-aware elitist resampling particle filter. The proposed approach is evaluated and verified using synthesized nonlinear distortions and real nonlinear distortions recorded by a commercial mobile phone.

This paper considers an effective method for nonlinear acoustic echo cancellation (NL-AEC). More specifically, we model the nonlinear echo path by a latent state vector capturing the coefficients of a memoryless processor and a linear finite impulse response filter. To estimate the posterior probability distribution of the state vector, an elitist particle filter based on evolutionary strategies (EPFES) has been proposed, which evaluates realizations of the latent state vector based on long-term fitness measures. This method includes a manually-tuned recursive calculation of the probabilities that the observation has been produced by the state-vector realizations. For avoiding this manual tuning, we introduce a new approach denoted as Elitist Resampling Particle Filtering (ERPF) which can also be shown to combine the advantages of the Sequential Importance Sampling Particle Filter (SIS-PF) and the Sequential Importance Sampling/Resampling Particle Filter (SIR-PF). This new approach allows universal use and leads to superior system identification performance compared to both the original EPFES as well as the SIR-PF, as verified for a simulated scenario and a real smartphone recording.

The adaptation of automatic speech recognition systems to a speaker or an environment is important if we are to achieve high speech recognition performance ubiquitously. Recently, deep neural network (DNN) based acoustic models have been made adaptive to speakers or environments by the addition of an auxiliary feature representing the acoustic context information such as speaker or noise characteristics to the network input. The addition of such auxiliary features to the input realizes only the adaptation of the bias term of the input layer. In this paper, we introduce “context adaptive neural networks,” which are an alternative approach for exploiting auxiliary features that can achieve adaptation of all the parameters of a layer including the linear transformation matrices and the bias terms. A context adaptive neural network is a neural network with one of its layers factorized into sublayers, each associated with an acoustic context class representing a class of speakers or noise conditions. The output of the factorized layer is obtained as a weighted sum of the contributions of all of the sublayers. The weighting coefficients, or context class weights, are derived from the auxiliary features, by transforming them through an auxiliary network. The auxiliary network and the main network can be trained jointly, which enables the context classes that optimize the training criterion to be learned automatically. We perform experiments on three tasks, i.e., two speaker adaptation experiments using DNN models with medium-sized (Wall Street Journal) and large (Continuous Spontaneous Japanese) training datasets, and one environmental adaptation of a convolutional neural network based acoustic model with CHiME3 data. These experiments confirm the potential of the proposed approach in various settings.

In this paper, we present the elitist particle filter based on evolutionary strategies (EPFES) as an efficient approach to estimate the statistics of a latent state vector capturing the relevant information of a nonlinear system. Similar to classical particle filtering, the EPFES consists of a set of particles and respective weights which represent different realizations of the latent state vector and their likelihood of being the solution of the optimization problem. As main innovation, the EPFES includes an evolutionary elitist-particle selection scheme which combines long-term information with instantaneous sampling from an approximated continuous posterior distribution. In this paper, we propose two advancements of the previously published elitist-particle selection process. Further, the EPFES is shown to be a generalization of the widely-used Gaussian particle filter and thus evaluated with respect to the latter: First, we consider the univariate nonstationary growth model with time-variant latent state variable to evaluate the tracking capabilities of the EPFES for instantaneously calculated particle weights. This is followed by addressing the problem of single-channel nonlinear acoustic echo cancellation as a challenging benchmark task for identifying an unknown system of large search space: the nonlinear acoustic echo path is modeled by a cascade of a parameterized preprocessor (to model the loudspeaker signal distortions) and a linear FIR filter (to model the sound wave propagation and the microphone). By using long-term information, we highlight the efficacy of the well-generalizing EPFES in estimating the preprocessor parameters for a simulated scenario and a real smartphone recording. Finally, we illustrate similarities between the EPFES and evolutionary algorithms to outline future improvements by fusing the achievements of both fields of research.

We propose a new concept for adapting CNN-based acoustic models using spatial diffuseness features as auxiliary information about the acoustic environment: the spatial diffuseness features are simultaneously employed as acoustic-model input features and to estimate environmental cues for context adaptation, where one convolutional layer is factorized into several sub-layers to represent different acoustic conditions. This context-adaptive CNN-based acoustic model facilitates an online environmental adaptation and is experimentally verified for the real-world recordings provided by the CHiME-3 task. The best performing setup reduces the average word error rate scores achieved by the baseline system (without using spatial diffuseness features) from 19.4% to 15.9% and 12.2% to 10.7% considering two experimental setups with and without front-end signal enhancement, respectively.

The use of auxiliary features is an effective way to improve the performance of deep neural network (DNN)-based acoustic models. Most approaches use auxiliary features that represent the speaker or the environment. These auxiliary features are usually computed independently of the acoustic model. This paper investigates a types of auxiliary features obtained from the output of a hidden layer that feeds back to the input layer of the network. Since the auxiliary features are extracted from the hidden layer of the network no external information is required such as the speaker or the environment. Experimentally, by forcing the extraction of the auxiliary features from the same networks, we can further improve the performance of the overall network and reduce the total number of parameters used. We tested this approach with different deep neural network architectures including: deep neural networks, convolutional neural networks and unfolded recurrent convolutional networks. We confirmed the effectiveness of this approach on the CHiME3 dataset.

Uncertainty decoding combines a probabilistic feature description with the acoustic model of a speech recognition system. For DNN-HMM hybrid systems, this can be realized by averaging the DNN outputs produced by a finite set of feature samples (drawn from an estimated probability distribution). In this article, we employ this sampling approach in combination with a multi-microphone speech enhancement system. We propose a new strategy for generating feature samples from multichannel signals, based on modeling the spatial coherence estimates between different microphone pairs as realizations of a latent random variable. From each coherence estimate, a spectral enhancement gain is computed and an enhanced feature vector is obtained, thus producing a finite set of feature samples, of which we average the respective DNN outputs. In the experimental part, this new uncertainty decoding strategy is shown to consistently improve the recognition accuracy of a DNN-HMM hybrid system for the 8-channel REVERB Challenge task.

We propose a spatial diffuseness feature for deep neural network (DNN)-based automatic speech recognition to improve recognition accuracy in reverberant and noisy environments. The feature is computed in real-time from multiple microphone signals without requiring knowledge or estimation of the direction of arrival, and represents the relative amount of diffuse noise in each time and frequency bin. It is shown that using the diffuseness feature as an additional input to a DNN-based acoustic model leads to a reduced word error rate for the REVERB challenge corpus, both compared to logmelspec features extracted from noisy signals, and features enhanced by spectral subtraction.

In this letter, we derive a new stepsize adaptation for the normalized least mean square algorithm (NLMS) by describing the task of linear acoustic echo cancellation from a Bayesian network perspective. Similar to the well-known Kalman filter equations, we model the acoustic wave propagation from the loudspeaker to the microphone by a latent state vector and define a linear observation equation (to model the relation between the state vector and the observation) as well as a linear process equation (to model the temporal progress of the state vector). Based on additional assumptions on the statistics of the random variables in observation and process equation, we apply the expectation-maximization (EM) algorithm to derive an NLMS-like filter adaptation. By exploiting the conditional independence rules for Bayesian networks, we reveal that the resulting EM-NLMS algorithm has a stepsize update equivalent to the optimal-stepsize calculation proposed by Yamamoto and Kitayama in 1982, which has been adopted in many textbooks. As main difference, the instantaneous stepsize value is estimated in the M step of the EM algorithm (instead of being approximated by artificially extending the acoustic echo path). The EM-NLMS algorithm is experimentally verified for synthesized scenarios with both, white noise and male speech as input signal.

In this work, a novel approach for nonlinear acoustic echo cancellation is proposed. The main innovative idea of the proposed method is to model only the small region of the echo path around the direct path by a group of parallel Hammerstein models, to estimate a nonlinear preprocessor by correlations between the linear kernels of the Hammerstein submodels, and to describe the remaining echo path by a simple Hammerstein model with the preprocessor determined in the aforementioned way. While the computational complexity of such a system increases only slightly in comparison to a linear echo canceller, experiments with speech recordings from a smartphone in different environments confirm a significantly increased echo cancellation performance.

In this article, we introduce a novel approach for nonlinear acoustic echo cancellation based on a combination of particle filtering and evolutionary strategies. The nonlinear echo path is modeled as a state vector with non-Gaussian probability distribution and the relation to the observed signals and near-end interferences are captured by nonlinear functions. To estimate the probability distribution of the state vector and the model parameters, we apply the numerical sampling method of particle filtering, where each set of particles represents different realizations of the nonlinear echo path. While the classical particle-filter approach is unsuitable for system identification with large search spaces, we introduce a modified particle filter to select elitist particles based on long-term fitness measures and to create new particles based on the approximated probability distribution of the state vector. The validity of the novel approach is experimentally verified with real recordings for a nonlinear echo path stemming from a commercial smartphone.

For applications like acoustic echo cancellation (AEC) or listening room equalization (LRE), a loudspeaker-enclosure-microphone system (LEMS) must be identified. When using a large number of reproduction channels, as, e. g., for wave field synthesis (WFS) or Higher-Order Ambisonics (HOA), the strong correlation of the loud-speaker signals will hamper a unique identification. A state-of-the-art remedy against this so-called nonuniqueness problem is a decorrelation of the loudspeaker signals, which facilitates a unique identification. However, most of the known approaches are not suitable for acoustic wave field reproduction schemes, as they would distort the reproduced wave field in an uncontrolled manner or degrade the audio quality. In this contribution, we propose a wave-domain time-varying filtering of the loudspeaker signals, so that the reproduced wave field is rotated within a perceptually acceptable range, while preserving its shape.