Sound Source localization (SSL) using microphone arrays is an active research topic with many applications, but noise and reverberation make the direction-of-arrival (DOA) and distance estimation a challenging problem. In this work, we propose a novel method to jointly estimate the DOA and distance in noisy and reverberant environments. Our method exploits the linear phase structure across frequencies in a steering vector (SV). We convert the joint estimation issue into an optimization problem, which can be solved by Newton’s method augmented by a gradient ascent method. Our method does not depend on certain microphone array geometry, and it can also be extended to estimate the elevation angle. We conducted experimental evaluations in simulated noisy and reverberant acoustic conditions, which verified the superiority of our proposed method to several established methods in estimation accuracy and computation efficiency.

Sound source localization is important in many applications, but noise and reverberation make the time difference of arrival (TDOA) estimation a challenging problem. In this work, we propose two novel methods to effectively estimate TDOA in noisy and reverberant environments. Our methods exploit the linear phase structure across frequencies in a steering vector (SV). To reduce potential noise and mathematical issues, we utilize absolute phases of SVs. We convert TDOA estimation into an optimization problem that is solvable by Newton's method. We conducted experimental evaluations in simulated acoustic conditions. In conditions with moderate-to-high input SNR and low reverberation, our fast-search method is superior in TDOA accuracy and is very efficient in computation. In conditions with low input SNR and high reverberation, our detailed-search method shows strong robustness and maintains advantageous performance in TDOA estimation.

Time-frequency (TF) masks are widely used in speech enhancement (SE). However, accurately estimating TF masks from noisy speech remains a challenge to both statistical or neural network (NN) approaches. Statistical model based mask estimation usually depends on a good parameter initialization, while NN-based method relies on setting proper and stable learning targets. To address these issues, we propose to extract TF speech structure from clean speech and partition noisy speech spectrogram into mutually exclusive regions. We investigate modeling clean speech by utterance-specific narrowband complex Gaussian mixture models to derive the regions, and using the region targets to supervise the training of UNet++, a high-performance NN, for predicting regions from noisy speech. For multichannel SE, we consider two scenarios of using speech regions: 1) integrating the regions with TF masks by constraining the mask values or the model parameter updates, and 2) using the predicted regions in place of TF masks. For single-channel SE, we consider using the region targets to improve TF mask targets. Furthermore, we propose to use UNet++ for TF mask estimation. Our experiment results on speech recognition (CHiME-3) and SE (CHiME-3 and LibriSpeech) have demonstrated the effectiveness of our proposed approach of modeling speech region structure to improve TF masks for speech recognition and enhancement.

Vector Taylor Series (VTS) based model compensation approach has been successfully applied to various robust speech recognition tasks. In this paper, we propose a novel method of variable transformation to calculate the static statistics. In addition, we provide a detailed explanation of VTS and random variable transformations adopted in some recent papers. Experiments on Aurora 4 showed that the proposed approach obtained 22.8% relative WER reduction over the traditional first-order VTS methods.

Vector Taylor Series (VTS) model based compensation approach has been successfully applied to various robust speech recognition tasks. In this paper, a novel method to derive the formula to calculate the static and dynamic statistics based on second-order VTS (sVTS) is presented, which provides a new insight on the VTS approximation. Lengthy derivation could therefore be avoided when high order VTS is used and the proposed approach is more compact and easier to implement compared to previous high order VTS approaches. Experiments on Aurora 4 showed that the proposed sVTS based model compensation approach obtained 16.7% relative WER reduction over traditional first-order VTS (fVTS) approach.