The spatial information about a sound source is carried by acoustic waves to a microphone array and can be observed through estimation of phase and amplitude differences between microphones. Time difference of arrival (TDoA) captures the propagation delay of the wavefront between microphones and can be used to steer a beamformer or to localize the source. However, reverberation and interference can deteriorate the TDoA estimate. Deep neural networks (DNNs) through supervised learning can extract speech related TDoAs in more adverse conditions than traditional correlation -based methods.Acoustic simulations provide large amounts of data with annotations, while real recordings require manual annotations or the use of reference sensors with proper calibration procedures. The distributions of these two data sources can differ. When a DNN model that is trained using simulated data is presented with real data from a different distribution, its performance decreases if not properly addressed.For the reduction of DNN -based TDoA estimation error, this work investigates the role of different input normalization techniques, mixing of simulated and real data for training, and applying an adversarial domain adaptation technique. Results quantify the reduction in TDoA error for real data using the different approaches. It is evident that the use of normalization methods, domain-adaptation, and real data during training can reduce the TDoA error.

The Time Difference of Arrival (TDoA) of a sound wavefront impinging on a microphone pair carries spatial information about the source. However, captured speech typically contains dynamic non-speech interference sources and noise. Therefore, the TDoA estimates fluctuate between speech and interference. Deep Neural Networks (DNNs) have been applied for Time-Frequency (TF) masking for Acoustic Source Localization (ASL) to filter out non-speech components from a speaker location likelihood function. However, the type of TF mask for this task is not obvious. Secondly, the DNN should estimate the TDoA values, but existing solutions estimate the TF mask instead. To overcome these issues, a direct formulation of the TF masking as a part of a DNN-based ASL structure is proposed. Furthermore, the proposed network operates in an online manner, i.e., producing estimates frame-by-frame. Combined with the use of recurrent layers it exploits the sequential progression of speaker related TDoAs. Training with different microphone spacings allows model re-use for different microphone pair geometries in inference. Real-data experiments with smartphone recordings of speech in interference demonstrate the network's generalization capability.

Applying a predicted time-frequency mask to the noisy speech spectrogram and directly predicting the clean speech magnitude spectrogram are two common deep learning-based speech enhancement approaches. Ensemble techniques such as averaging and neural network-based fusion of magnitude spectra obtained with these two approaches have been shown to improve the objective perceptual quality of speech using synthetic mixtures of data. This work generalizes the averaging ensemble approach by proposing neural network layers to predict time-frequency varying weights for the combination of the two magnitude spectra obtained by time-frequency masking and by direct prediction. In order to combine the best individual magnitude spectrum estimates, the proposed weight prediction layers are trained after the time-frequency mask and magnitude spectrum networks layers have been separately trained for their corresponding objectives and their weights have been fixed. Using the publicly available CHiME3-challenge data, which consists of both simulated and real speech recordings in everyday environments with noise and interference, the proposed approach leads to significantly higher noise suppression in terms of segmental source-to-distortion ratio over the alternative approaches. In addition, the approach achieves similar improvements in the average objective instrumentally measured intelligibility scores with respect to the best achieved scores.

This paper proposes to use low-level spatial features extracted from multichannel audio for sound event detection. We extend the convolutional recurrent neural network to handle more than one type of these multichannel features by learning from each of them separately in the initial stages. We show that instead of concatenating the features of each channel into a single feature vector the network learns sound events in multichannel audio better when they are presented as separate layers of a volume. Using the proposed spatial features over monaural features on the same network gives an absolute F-score improvement of 6.1% on the publicly available TUT-SED 2016 dataset and 2.7% on the TUT-SED 2009 dataset that is fifteen times larger.

The steered response power (SRP) methods can be used to build a map of sound direction likelihood. In the presence of interference and reverberation, the map will exhibit multiple peaks with heights related to the corresponding sound's spectral content. Often in realistic use cases, the target of interest (such as speech) can exhibit a lower peak compared to an interference source. This will corrupt any direction dependent method, such as beamforming. Regression has been used to predict time-frequency (TF) regions corrupted by reverberation, and static broadband noise can be efficiently estimated for TF points. TF regions dominated by noise or reverberation can then be de-emphasized to obtain more reliable source direction estimates. In this work, we propose the use of convolutional neural networks (CNNs) for the prediction of a TF mask for emphasizing the direct path speech signal in time-varying interference. SRP with phase transform (SRP-PHAT) combined with the CNN-based masking is shown to be capable of reducing the impact of time-varying interference for speaker direction estimation using real speech sources in reverberation.

High level of noise reduces the perceptual quality and intelligibility of speech. Therefore, enhancing the captured speech signal is important in everyday applications such as telephony and teleconferencing. Microphone arrays are typically placed at a distance from a speaker and require processing to enhance the captured signal. Beamforming provides directional gai nt owards the source of interest and attenuation of interference. It is often followed by a single channel post-filter to further enhance the signal. Non-linear spatial post-filters are capable of providing high noise suppression but can produce unwanted musical noise that lowers the perceptual quality of the output. This work proposes an artificial neural network (ANN) to learn the structure of naturally occurring post-filters to enhance speech from interfering noise. The ANN uses phase-based features obtained from a multichannel array as an input. Simulations are used to train the ANN in a supervised manner. The performance is measured with objective scores from speech recorded in an office environment. The post-filters predicted by the ANN are found to improve the perceptual quality over delay-and-sum beamforming while maintaining high suppression of noise characteristic to spatial post-filters. Index Terms:Speech enhancement, Microphone arrays, Array signal processing, Artificial neural networks, Psychoacoustics.

In recent years ad-hoc microphone arrays have become ubiquitous, and the capture hardware and quality is increasingly more sophisticated. Ad-hoc arrays hold a vast potential for audio applications, but they are inherently asynchronous, i.e., temporal offset exists in each channel, and furthermore the device locations are generally unknown. Therefore, the data is not directly suitable for traditional microphone array applications such as source localization and beamforming. This work presents a least squares method for temporal offset estimation of a static ad-hoc microphone array. The method utilizes the captured audio content without the need to emit calibration signals, provided that during the recording a sufficient amount of sound sources surround the array. The Cramer-Rao lower bound of the estimator is given and the effect of limited number of surrounding sources on the solution accuracy is investigated. A practical implementation is then presented using non-linear filtering with automatic parameter adjustment. Simulations over a range of reverberation and noise levels demonstrate the algorithm's robustness. Using smartphones an average RMS error of 3.5 samples (at 48 kHz) was reached when the algorithm's assumptions were met.

This paper describes a novel multiple acoustic source tracking method based on track before detect paradigm. Multiple particle filters are used to represent the state of all sources. Sources are detected and removed using a likelihood ratio obtained from particle weights. The weights are obtained by evaluating the likelihood of microphone pair phase difference. Tracking performance from recorded data with rich sequences of speech is presented using multiple object tracking metrics. Results show that the proposed method can detect and track multiple temporally overlapping speech sources as well as switching talkers even in weak signal-to-noise ratios.

Speech signal is perturbed in real environments due to room acoustics and noise sources. The speech wavefront is received by a microphone array, which is used to determine the wavefront direction of arrival (DOA). DOA estimates from several spatially separated arrays are used to locate the speaker. Array channels may sometimes be corrupted by noise. In these cases, a DOA estimate may differ from the actual direction of the speaker. Using such estimates in localization could deteriorate the position estimate. A criterion is presented for DOA exclusion. A subset of DOA estimates is chosen that minimizes this criterion. The method is numerically shown to improve localization robustness. Recordings from a real meeting room are studied.

Increasing demand for automatic large area surveillance has made methods estimating direction of arrival (DOA), especially acoustic methods, an interesting topic. In these systems false alarms and faults are a disturbing factor. This paper presents a reliability criterion and a new method to diminish these errors. In very low signal-to-noise ratio conditions there are seldom any means to improve the performance of a DOA estimator. In these cases it is important to have some information about the estimation reliability. Our focus is on wideband signals propagating as planar waves in three dimensional space. If the estimation method makes no assumptions on the signal propagation speed, it is possible to compute a reliability measure for obtained estimates. With this reliability measure we are able to decide whether the data produced by a time delay based DOA estimation system is usable or not. Indeed, with this criterion an estimation system can state its own reliability.