Deep neural networks (DNNs) have proven themselves as an effective means to separate clean speech from noisy mixtures. When there are multiple concurrent talkers, however, unambiguously defining the target output is not trivial, especially if the mixture is single-channel and the talkers are not known in advance. Although this problem can be addressed with permutation invariant training or deep clustering, the performance still suffers in this case. Approaches for compact arrays of multiple microphones can exploit spatial diversity to resolve the ambiguity: a separate output may be generated for each direction of arrival (DOA), or the speaker assignment can be controlled with a location-based training (LBT). Alternatively, we can narrow down the target definition at the input, to perform a spatially selective speaker separation instead of separating all speakers simultaneously. This is achieved by specifying freely adjustable target DOAs. On the one hand, these can be integrated as location-based input features (LBI). On the other hand, the main contribution of this work is a location dependent feature extraction (LDE): we implicitly introduce a DOA dependence in a small part of the DNN by optimizing its parameters for each DOA separately. Experiments demonstrate that LDE outperforms LBT and LBI in terms of instrumental metrics and speech recognition results. A representative audio example is presented for a qualitative impression. An analysis of the spatial selectivity reveals that target and nontarget directions can be distinguished quite well with LDE, which is also verified by recordings of real moving talkers.

Supervised learning methods are a powerful tool for direction of arrival (DOA) estimation because they can cope with adverse conditions where simplified models fail. In this work, we consider a previously proposed convolutional neural network (CNN) approach that estimates the DOAs for multiple sources from the phase spectra of the microphones. For speech, specifically, the approach was shown to work well even when trained entirely on synthetically generated data. However, as each frame is processed separately, temporal context cannot be taken into account. This prevents the exploitation of interframe signal correlations, and the fact that DOAs do not change arbitrarily over time. We therefore consider two different extensions of the CNN: the integration of a long short-term memory (LSTM) layer, or of a temporal convolutional network (TCN). In order to accommodate the incorporation of temporal context, the training data generation framework needs to be adjusted. To obtain an easily parameterizable model, we propose to employ Markov chains to realize a gradual evolution of the source activity at different times, frequencies, and directions, throughout a training sequence. A thorough evaluation demonstrates that the proposed configuration for generating training data is suitable for the tasks of single-, and multi-talker localization. In particular, we note that with temporal context, it is important to use speech, or realistic signals in general, for the sources. Experiments with recorded impulse responses and noise reveal that the CNN with the LSTM extension outperforms all other considered approaches, including the plain CNN, and the TCN extension.

The weighted least-squares (WLS) direction-of-arrival estimator that minimizes an error based on interchannel phase differences is both computationally simple and flexible. However, the approach has several limitations, including an inability to cope with spatial aliasing and a sensitivity to phase wrapping. The recently proposed phase wrapping robust (PWR)-WLS estimator addresses the latter of these issues, but requires solving a nonconvex optimization problem. In this contribution, we focus on both of the described shortcomings. First, a conceptually simpler alternative to PWR is presented that performs comparably given a good initial estimate. This newly proposed method relies on an unwrapping of the phase differences vector. Secondly, it is demonstrated that all microphone pairs can be utilized at all frequencies with both estimators. When incorporating information from other frequency bins, this permits a localization above the spatial aliasing frequency of the array. Experimental results show that a considerable performance improvement is possible, particularly for arrays with a large microphone spacing.

For blind speech dereverberation, two frameworks are commonly used: on the one hand, the multi-channel linear prediction (MCLP) framework, and on the other hand, data-dependent beamforming, e.g., the generalized sidelobe canceler (GSC) framework. The MCLP framework is designed to perform deconvolution and hence has gained increased prominence in blind speech dereverberation. The GSC framework is commonly used for noise reduction, but may be applied for dereverberation as well. In previous work, we have shown that for the noiseless case, MCLP and the GSC yield in theory mathematically equivalent results in terms of dereverberation. In this paper, we assume additional coherent as well as incoherent-noise components and formally analyze and compare both frameworks in terms of dereverberation and noise reduction performance. Both the theoretical analysis and time domain simulation results demonstrate that unlike the GSC, MCLP expectably shows limited performance in terms of noise reduction, while both perform equally well in terms of dereverberation, provided that the GSC blocking matrix achieves complete blocking of the early reverberant-speech component and sufficiently many microphones are available. In case of incomplete blocking, however, the GSC performs inferior to MCLP in terms of dereverberation, as shown in short-time Fourier transform domain simulations.

Clipping or saturation in audio signals is a very common problem in signal processing, for which, in the severe case, there is still no satisfactory solution. In such case, there is a tremendous loss of information, and traditional methods fail to appropriately recover the signal. We propose a novel approach for this signal restoration problem based on the framework of Iterative Hard Thresholding. This approach, which enforces the consistency of the reconstructed signal with the clipped observations, shows superior performance in comparison to the state-of-the-art declipping algorithms. This is confirmed on synthetic and on actual high-dimensional audio data processing, both on SNR and on subjective user listening evaluations.

Whereas state-of-the-art single-channel noise reduction algorithms for auditory prostheses demonstrate an appreciable suppression of the noise and improved speech quality, they are unable, thus far, to improve the intelligibility of noise-degraded speech signals. Alternative approaches to speech enhancement using a binary time-frequency mask have demonstrated substantial intelligibility improvements in low signal-to-noise-ratio (SNR) conditions under ideal settings, making this a promising research direction for auditory prostheses. These approaches exploit the sparsity and disjoint-ness of speech spectra in their short-time-frequency representation to preserve only the target-dominant time-frequency regions in the processed output. State-of-the-art noise reduction algorithms in contrast are soft-decision approaches which weight each time-frequency region in proportion to the prevailing SNR. However, the potential for intelligibility improvement using these approaches has not been examined systematically vis-à-vis the binary mask alternative. This contribution compares the performance of an ideal soft-decision system, exemplified by the ideal Wiener filter (IWF), and the ideal binary mask (IBM) for single-channel speech enhancement for auditory prostheses. To obtain results relevant to this application area, a (relatively) low spectral resolution, modelled using the Bark-spectrum scale, is used for both the IWF and the IBM. This spectral resolution is comparable to that being used in commercial hearing instruments. The comparison is in terms of potential for intelligibility improvement and resulting signal quality. Intelligibility tests carried out under various noise conditions and SNRs show that the IWF leads to higher intelligibility scores than the IBM in low SNR conditions. Under non-ideal parameter estimates, it is demonstrated that the IWF approach is also much less sensitive to estimation errors. Quality-wise, a preference for the IWF exists. This was evaluated using a two-stage, pair-wise preference-rating test.

In this paper, solutions for combined feedback and noise suppression in hearing aids are developed. The techniques presented are based on the generalized sidelobe canceller (GSC) and adaptive feedback canceller (AFC), with a prediction error method (PEM) adaptation to avoid speech distortion. Two possible cascades of GSC-based noise reduction and AFC, namely an ldquoAFC firstrdquo and a ldquoGSC first,rdquo as well as a truly integrated solution that jointly suppresses feedback and noise are discussed. The integrated solution (called PEM-GFIC) achieves optimum synergies between noise and feedback suppression at the lowest computational cost. In addition, it cancels more feedback than the generalized echo and interference canceller, a joint solution for echo and noise suppression. In the cascaded solutions, the feedback and noise suppression filters are not always optimally exploited. For high input SNRs, ldquoAFC firstrdquo scheme generally may achieve better feedback cancellation because of its larger number of degrees of freedom. However, noise reduction by the GSC-stage seriously affects the feedback cancellation performance. At low SNRs, ldquoGSC firstrdquo generally achieves more feedback cancellation than PEM-GFIC at the expense of worse noise reduction. At high hearing aid gains and/or large SNRs, the noise reduction stage however negatively affects the performance of the feedback cancellation filter, resulting in a worse feedback and noise suppression compared to PEM-GFIC.

We propose an instrumental variable method for adaptive feedback cancellation (IV-AFC) in hearing aids that is based on the autoregressive modelling of the desired signal. The IV-AFC offers better feedback suppression for spectrally colored signals than the standard continuous adaptation feedback cancellers. In contrast to a previously proposed prediction error method based feedback canceller, the IV-AFC does not suffer from stability problems when the adaptive feedback canceller is highly time-varying.

For small-sized arrays such as hearing aids, noise reduction is obtained at the expense of an increased sensitivity to errors in the assumed signal model, i.e., microphone mismatch, variations in speaker and microphone positions, reverberation. However, the noise reduction algorithm should still be robust, i.e., insensitive to small signal model errors. In this paper, we evaluate the robustness of the Generalized Sidelobe Canceller (GSC) and a recently developed Multichannel Wiener Filtering (MWF) technique for hearing aid applications both analytically and experimentally. The analysis reveals that robustness of the GSC is especially crucial in complicated noise scenarios and that microphone mismatch is particularly harmful to the GSC, even when the adaptive noise canceller is adapted during noise only. Hence, a constraint on the noise sensitivity of the GSC is essential, at the expense of less noise reduction. The MWF on the other hand, is not affected by microphone mismatch and has a potential benefit over the robust GSC with noise sensitivity constraint. However, the MWF is sensitive to the estimation accuracy of the second order statistics of speech and noise so that its benefit may be lost in nonstationary noise scenarios.

Recently, a generalized noise reduction scheme was proposed, called the spatially preprocessed speech distortion weighted multi-channel Wiener filter (SP-SDW-MWF). Compared to GSC with quadratic inequality constraint (QIC-GSC), the SP-SDW-MWF reduces more noise, for a given maximum speech distortion level. We develop time-domain and frequency-domain stochastic gradient implementations of the SP-SDW-MWF. Experimental results with a hearing aid show that the proposed stochastic gradient algorithm preserves the benefit of the SP-SDW-MWF over the QIC-GSC, while its computational cost is comparable to the NLMS based scaled projection algorithm (SPA) for QIC-GSC.

The noise reduction performance of the generalized sidelobe canceller (GSC) depends on the validity of a priori assumptions about the signal model, whereas the recently developed multi-channel Wiener filter (MWF) techniques do not, hence, their potential benefit. However, both techniques rely on a speech detection algorithm. In this paper, we analyze the average effect of speech detection errors on the performance of the GSC and MWF both theoretically and experimentally. It is shown that the MWF preserves its benefit over the GSC for a reasonable speech detection error rate of 20% or less, even when the GSC is supplied with a robustness constraint.