In recent years, several supervised learning-based approaches have been proposed for estimating the direction of arrival (DOA) of a single talker in noisy and reverberant environments. In the absence of auxiliary information, such as a voice activity detector (VAD), the estimated DOA may be erroneous due to speech pauses or noise dominance. In this paper, we consider a speech-aware DOA estimation system for binaural hearing aids, which does not require a separate VAD. This system utilizes a combination of spatial features with an auditory-inspired periodicity feature called periodicity degree (PD) as input features of a convolutional neural network (CNN). Using speech and non-speech signals during the training, the CNN can capture the harmonic structure encoded in the PD features, thereby distinguishing speech from non-speech portions and simultaneously mapping spatial features to sound source DOA upon speech detection. To investigate the benefit of using PD features for speech-aware DOA estimation, we evaluated the performance of speech-aware systems that utilized either broadband or narrowband feature combinations compared to baseline systems. We propose to use a novel narrowband feature combination consisting of the narrowband cross-power spectrum (CPS) as the spatial feature and a new subband-averaged representation of PD features. The broadband feature combination consisted of the generalized cross-correlation with phase transform (GCC-PHAT) and the broadband PD features. The baseline systems considered in this work consisted of a CNN that exploits only a spatial feature, cascaded with a VAD. Evaluations in reverberant environments with different background noises for both static and dynamic single-talker scenarios demonstrate that incorporating the PD feature in conjunction with any type of spatial feature provides an advantage for binaural DOA estimation in terms of accuracy and angular error.

The approach to directional speech enhancement described and evaluated herein involves combining adaptive beamforming and spatial spectro-temporal post-filtering using a small dual-microphone end-fire array, as is used in modern digital hearing aids. An established and commonly used hearing aid algorithm is used for the adaptive beamforming while our Short-Time Target Cancellation (STTC) processing, which leverages the computational efficiency of the Short-Time Fourier Transform (STFT), is used for the post-filtering. Our STTC processing is notable for effecting real-time low-latency (≤20 ms) spatial spectro-temporal filtering with causal, efficient and minimalist STFT-based processing that has low requirements in terms of energy consumption, memory and computational power. The evaluation is carried out in both semi-anechoic and reverberant rooms using instrumental measures of speech enhancement. As demonstrated in this brief study, our STTC spatial spectro-temporal filtering can be implemented with hearing aid microphones and can be used to enhance established hearing aid processing.

While many algorithms deal with direction of arrival (DOA) estimation and voice activity detection (VAD) as two separate tasks, only a small number of data-driven methods have addressed these two tasks jointly. In this paper, a multi-input single-output convolutional neural network (CNN) is proposed which exploits a novel feature combination for joint DOA estimation and VAD in the context of binaural hearing aids. In addition to the well-known generalized cross correlation with phase transform (GCC-PHAT) feature, the network uses an auditory-inspired feature called periodicity degree (PD), which provides a broadband representation of the periodic structure of the signal. The proposed CNN has been trained in a multi-conditional training scheme across different signal-to-noise ratios. Experimental results for a single-talker scenario in reverberant environments show that by exploiting the PD feature, the proposed CNN is able to distinguish speech from non-speech signal blocks, thereby outperforming the baseline CNN in terms of DOA estimation accuracy. In addition, the results show that the proposed method is able to adapt to different unseen acoustic conditions and background noises.

Recent studies of hearing aid benefits indicate that head movement behavior influences performance. To systematically assess these effects, movement behavior must be measured in realistic communication conditions. For this, the use of virtual audiovisual environments with animated characters as visual stimuli has been proposed. It is unclear, however, how these animations influence the head- and eye-movement behavior of subjects. Here, two listening tasks were carried out with a group of 14 young normal hearing subjects to investigate the influence of visual cues on head- and eye-movement behavior; on combined localization and speech intelligibility task performance; as well as on perceived speech intelligibility, perceived listening effort and the general impression of the audiovisual environments. Animated characters with different lip-syncing and gaze patterns were compared to an audio-only condition and to a video of real persons. Results show that movement behavior, task performance, and perception were all influenced by visual cues. The movement behavior of young normal hearing listeners in animation conditions with lip-syncing was similar to that in the video condition. These results in young normal hearing listeners are a first step towards using the animated characters to assess the influence of head movement behavior on hearing aid performance.

Sensorineural hearing loss typically results in elevated thresholds and steepened loudness growth significantly conditioned by a damage of outer hair cells (OHC). In hearing aids, amplification and dynamic compression aim at widening the limited available dynamic range. However, speech perception particularly in complex acoustic scenes often remains difficult. Here, a physiologically motivated, fast acting, model-based dynamic compression algorithm (MDC) is introduced which aims at restoring the behaviorally estimated basilar membrane input-output (BM I/O) function in normal-hearing listeners. A system-specific gain prescription rule is suggested, based on the same model BM I/O function and a behavioral estimate of the individual OHC loss. Cochlear off-frequency component suppression is mimicked using an instantaneous frequency (IF) estimate. Increased loudness as a consequence of widened filters in the impaired system is considered in a further compensation stage. In an extended version, a subband estimate of the speech presence probability (MDC+SPP) additionally provides speech-selective amplification in stationary noise. Instrumental evaluation revealed that the IF control enhances the spectral contrast of vowels and benefits in quality predictions at higher signal-to-noise ratios (SNRs) were observed. Compared with a conventional multiband dynamic compressor, MDC achieved objective quality and intelligibility benefits for a competing talker at lower SNRs. MDC+SPP outperformed the conventional compressor in the quality predictions and reached comparable instrumental speech intelligibility as achieved with linear amplification. The proposed algorithm provides a first promising basis for auditory model-based compression with signal-type- and bandwidth-dependent gains.

Besides noise reduction an important objective of binaural speech enhancement algorithms is the preservation of the binaural cues of both desired and undesired sound sources. Recently an extension of the binaural Multi-channel Wiener filter (MWF), namely the MWF-IC, has been presented which aims to preserve the Interaural Coherence (IC) of the noise component. Since for the MWF-IC no closed-form solution exists, in this paper we propose to preserve the IC using the binaural MWF with partial noise estimation (MWF-N), for which a closed-form solution exists. Furthermore, we derive a closed-form expression for the trade-off parameter in the MWF-N yielding a predefined IC at the filter output. Experimental results in a diffuse noise scenario show that both the MWF-IC and the MWF-N preserve the IC of the output noise component. However, the MWF-IC yields a better noise reduction performance whereas the MWF-N introduces less speech distortion.

This paper describes an online algorithm for enhancing monaural noisy speech. First, a novel phase-corrected low-delay gammatone filterbank is derived for signal subband decomposition and resynthesis; the subband signals are then analyzed frame by frame. Second, a novel feature named periodicity degree (PD) is proposed to be used for detecting and estimating the fundamental period ( P0) in each frame and for estimating the signal-to-noise ratio (SNR) in each frame-subband signal unit. The PD is calculated in each unit as the multiplication of the normalized autocorrelation and the comb filter ratio, and shown to be robust in various low-SNR conditions. Third, the noise energy level in each signal unit is estimated recursively based on the estimated SNR for units with high PD and based on the noisy signal energy level for units with low PD. Then the a priori SNR is estimated using a decision-directed approach with the estimated noise level. Finally, a revised Wiener gain is calculated, smoothed, and applied to each unit; the processed units are summed across subbands and frames to form the enhanced signal. The P 0 detection accuracy of the algorithm was evaluated on two corpora and showed comparable performance on one corpus and better performance on the other corpus when compared to a recently published pitch detection algorithm. The speech enhancement effect of the algorithm was evaluated on one corpus with two objective criteria and showed better performance in one highly non-stationary noise and comparable performance in two other noises when compared to a state-of-the-art statistical-model based algorithm.

Besides noise reduction an important objective of binaural speech enhancement algorithms is the preservation of the binaural cues of all sound sources. To this end, an extension of the binaural multi-channel Wiener filter (MWF), namely the MWF-ITF, has been proposed, which aims to preserve the Interaural Transfer Function (ITF) of the noise sources. However, the MWF-ITF is well-suited only for directional noise sources but not for, e.g., spatially isotropic noise, whose spatial characteristics cannot be properly described by the ITF but rather by the Interaural Coherence (IC). Hence, another extension of the binaural MWF, namely the MWF-IC, has been recently proposed, which aims to preserve the IC of the noise component. Since for the MWF-IC a substantial tradeoff between noise reduction and IC preservation exists, in this paper we propose a perceptually constrained version of the MWF-IC, where the amount of IC preservation is controlled based on the IC discrimination ability of the human auditory system. In addition, a theoretical analysis of the binaural cue preservation capabilities of the binaural MWF and the MWF-ITF for spatially isotropic noise fields is provided. Several simulations in diffuse noise scenarios show that the perceptually constrained MWF-IC yields a controllable preservation of the IC without significantly degrading the output SNR compared to the binaural MWF and the MWF-ITF. Furthermore, contrary to the binaural MWF and MWF-ITF, the proposed algorithm retains the spatial separation between the output speech and noise components while the binaural cues of the speech component are only slightly distorted, such that the binaural hearing advantage for speech intelligibility can still be exploited.

Besides noise reduction an important objective of binaural speech enhancement algorithms is the preservation of the binaural cues of both desired and undesired sound sources. Recently, an extension of the binaural Multi-channel Wiener filter (MWF), namely the MWF-IC, has been presented, which aims to preserve the Interaural Coherence (IC) of the noise component. Since for the MWF-IC a substantial trade-off between noise reduction and IC preservation exists, in this paper we propose a perceptually constrained version of the MWF-IC, where the amount of IC preservation is controlled based on psychoacoustic criterias of the IC discrimination ability of the human auditory system. In addition, we present a simplified version of the MWF-IC, resulting in a decrease of computational complexity. Experimental results show that the perceptually motivated MWF-IC and its simplified version yield a very similar performance and the loss in intelligibility weighted output SNR compared to the binaural MWF can be limited to 0.5 dB, whereas the spatial separation between the output speech and noise component is increased leading to better perceptual results.

Besides noise reduction an important objective of binaural speech enhancement algorithms is the preservation of the binaural cues, i.e. the Interaural Level Difference and the Interaural Time Difference of all sound sources. Recently, extensions of the binaural Multi-channel Wiener filter (MWF) have been presented, which aim to preserve the binaural cues of the residual noise component. However, since these algorithms aim to preserve the Interaural Transfer Function (ITF), they are well-suited only for directional noise sources but not for, e.g. spatially isotropic noise, which can not be fully described by the ITF. In this paper, we present an extension of the binaural MWF, aiming to preserve the Interaural Coherence of the residual noise component. Experimental results using spatially isotropic noise show that the proposed algorithm yields a good preservation of the Interaural Coherence without significantly degrading the output SNR compared to the binaural MWF and the binaural MWF with ITF preservation.

The segregation of concurrent speakers and other sound sources is an important aspect of the human auditory system but is missing in most current systems for automatic speech recognition (ASR), resulting in a large gap between human and machine performance. The present study uses a physiologically-motivated model of binaural hearing to estimate the position of moving speakers in a noisy environment by combining methods from Computational Auditory Scene Analysis (CASA) and ASR. The binaural model is paired with a particle filter and a beamformer to enhance spoken sentences that are transcribed by the ASR system. Results based on an evaluation in clean, anechoic two-speaker condition shows the word recognition rates to be increased from 30.8% to 72.6%, demonstrating the potential of the CASA-based approach. In different noisy environments, improvements were also observed for SNRs of 5 dB and above, which was attributed to the average tracking errors that were consistent over a wide range of SNRs.

An important objective of binaural noise reduction algorithms is the preservation of the binaural cues. In this paper an extension of the Multi-channel Wiener filter with binaural cue preservation (MWF-ITF) is presented, where the average noise ITF preservation term is replaced by an instantaneous noise ITF preservation term. This framework in addition allows to impose perfect ITF preservation, leading to a hard-constraint formulation. Experimental results show that the proposed technique yields a better performance in preserving the binaural cues of both the noise component and the speech component compared to the MWF-ITF, without degrading the output SNR.

We aim to assess the perceived quality of estimated source signals in the context of audio source separation. These signals may involve one or more kinds of distortions, including distortion of the target source, interference from the other sources or musical noise artifacts. We propose a subjective test protocol to assess the perceived quality with respect to each kind of distortion and collect the scores of 20 subjects over 80 sounds. We then propose a family of objective measures aiming to predict these subjective scores based on the decomposition of the estimation error into several distortion components and on the use of the PEMO-Q perceptual salience measure to provide multiple features that are then combined. These measures increase correlation with subjective scores up to 0.5 compared to nonlinear mapping of individual state-of-the-art source separation measures. Finally, we released the data and code presented in this paper in a freely available toolkit called PEASS.

The effect of a binaural coherence-based noise reduction scheme on the feedback stability margin and sound quality in hearing aids has been analyzed. For comparison, a conventional adaptive feedback canceler (AFC) and the combination of the adaptive filter with the binaural coherence filter have been tested. The observed quantities are feedback stability and target signal attenuation. An objective measure of feedback stability, i.e., the added stable gain (ASG) was obtained for a number of algorithmic settings and compared to a subjective measure of feedback stability, the added tolerable gain (ATG). In an attempt to eliminate the subjective bias in estimating the ATG, the ldquounbiased added gainrdquo (UAG) is introduced as a new method. Both, objective and subjective measures give similar results for feedback stability. This allows for a valid comparison across different feedback reduction schemes both in isolation and in combination: whereas the ASG of the coherence filter without combination with AFC is negligible, the results indicate that a robust feedback suppression in hearing aids can be achieved if the benefit of de-correlation and the head-shadow effect in binaural hearing aids is used in an advantageous way. The ASG reaches 23 dB for the best combination at the expense of an average target signal attenuation of 15 dB at critical frequencies. The attribute of the coherence filter is that it adaptively limits the maximum gain before feedback becomes audible. The UAG analysis revealed that subjects used a stable quality criterion across the conditions tested and that the group of subjects covered a large range of individual quality criteria.

In this study a self-steering beamformer with binaural output for a head-worn microphone array is investigated in simulated and real- world conditions. The influence of the underlying sound propagation model on the estimation accuracy of the direction of arrival (DOA) estimation algorithm and the overall performance of the combined DOA-beamformer-system is evaluated. For this, technical performance measures as well as objective quality measures based on perceptual models of the auditory system are used. The self-steering beamformer showed better performance than a beamformer with fixed look-direction for SNR values above -2 dB if the propagation model includes at least a coarse head model.

A key question for speech enhancement and simulations of auditory scene analysis in high levels of nonstationary noise is how to combine principles of auditory grouping and to integrate several noise-perturbed acoustical cues in a robust way. We present an application of recent online, nonlinear, non-Gaussian multidimensional statistical filtering methods which integrates tracking of sound-source direction and spectro-temporal dynamics of two mixed voices. The framework used is in agreement with the notion of evaluating competing hypotheses. To limit the number of hypotheses which need to be evaluated, the approach developed here uses a detailed statistical description of the high-dimensional spectro-temporal dynamics of speech, which is measured from a large speech database. The results show that the algorithm tracks sound source directions very precisely, separates the voice envelopes with algorithmic convergence times down to 50 ms, and enhances the signal-to-noise ratio in adverse conditions, requiring high computational effort. The approach has a high potential for improvements of efficiency and could be applied for voice separation and reduction of nonstationary noises

A main task for computational auditory scene analysis (CASA) is to separate several concurrent speech sources. From psychoacoustics it is known that common onsets, common amplitude modulation and sound source direction are among the important cues which allow the separation for the human auditory system. A new algorithm is presented here, that performs statistical estimation of different sources by a state-space approach which integrates temporal and frequency-specific features of speech. It is based on a Sequential Monte Carlo (SMC) scheme and tracks magnitude spectra and direction on a frame-by-frame basis. First results for estimating sound source direction and separating the envelopes of two voices are shown. The results indicate that the algorithm is able to localize two superimposed sound sources in a time scale of 50 ms. This is achieved by integrating measured high-dimensional statistics of speech. Also, the algorithm is able to track the short-time envelope and the short-time magnitude spectra of both voices on a time scale of 10-40ms. The algorithm presented in this paper is developed for but not restricted to use in binaural hearing aid applications, as it is based on two head-mounted microphone signals as input. It is conceptionally able to separate more than two voices and integrate additional cues. 1.

Noise reduction algorithms have been proposed that try to imitate some aspects of the binaural information processing in the human hearing system by using two microphones at both ears and a central processor. Based on a two channel input, it is theoretically possible to cancel one interfering directional noise source at a time. Due to the relatively large distance between microphones, it should additionally be possible to detect and cancel reverberation based on coherence measures. In practice however, no significant increase in speech intelligibility has been shown so far with these schemes in acoustically complex environments characterized by diffuse noise and reverberation. This paper summarizes these results for a review. Furthermore, a localization algorithm is introduced, which uses a statistical description of the binaural signal differences in order to increase the robustness in estimating sound source incident directions in noisy environments. Possible noise reduction schemes based on this localization algorithm are discussed.