In Gaussian model-based multichannel audio source separation, the likelihood of observed mixtures of source signals is parametrized by source spectral variances and by associated spatial covariance matrices. These parameters are estimated by maximizing the likelihood through an expectation-maximization algorithm and used to separate the signals by means of multichannel Wiener filtering. We propose to estimate these parameters by applying nonnegative factorization based on prior information on source variances. In the nonnegative factorization, spectral basis matrices can be defined as the prior information. The matrices can be either extracted or indirectly made available through a redundant library that is trained in advance. In a separate step, applying nonnegative tensor factorization, two algorithms are proposed in order to either extract or detect the basis matrices that best represent the power spectra of the source signals in the observed mixtures. The factorization is achieved by minimizing the β-divergence through multiplicative update rules. The sparsity of factorization can be controlled by tuning the value of β. Experiments show that sparsity, rather than the value assigned to β in the training, is crucial in order to increase the separation performance. The proposed method was evaluated in several mixing conditions. It provides better separation quality with respect to other comparable algorithms.

The availability of realistic simulated corpora is of key importance for the future progress of distant speech recognition technology. The reliability, flexibility and low computational cost of a data simulation process may ultimately allow researchers to train, tune and test different techniques in a variety of acoustic scenarios, avoiding the laborious effort of directly recording real data from the targeted environment. In the last decade, several simulated corpora have been released to the research community, including the data-sets distributed in the context of projects and international challenges, such as CHiME and REVERB. These efforts were extremely useful to derive baselines and common evaluation frameworks for comparison purposes. At the same time, in many cases they highlighted the need of a better coherence between real and simulated conditions. In this paper, we examine this issue and we describe our approach to the generation of realistic corpora in a domestic context. Experimental validation, conducted in a multi-microphone scenario, shows that a comparable performance trend can be observed with both real and simulated data across different recognition frameworks, acoustic models, as well as multi-microphone processing techniques.

Speech recognition in a realistic noisy environment using multiple microphones is the focal point of the third CHiME challenge. Over the baseline ASR system provided for this challenge, we apply state of the art algorithms for boosting acoustic model learning and hypothesis rescoring to improve the final output. To this aim, we first use the automatic transcription of each channel to re-train the acoustic model for that channel and then we apply linear language model rescoring to find a better solution in the n-best list. LM rescoring is performed using an efficient set of N-gram and Recurrent Neural Network LM (RNNLM) trained on a wisely-selected text set. In the experiments, we show that the proposed approach improves not only the individual channel transcription, but also the enhanced channels produced by MVDR and delay-and-sum beamforming.

This work proposes a solution to the problem of under-determined audio source separation using pre-trained redundant source-based prior information. In local Gaussian modeling of a mixing process, an observed mixture is modeled by a Gaussian distribution parameterized by source variances and spatial covariance matrices. The separation is performed by estimating the parameters, and applying Wiener filtering on the observed mixture. We propose, in a training phase, to build a redundant library of spectral basis matrices of all probable source power spectra, applying non-negative tensor factorization (NTF). In the testing phase, the matrices that match the observed mixture are detected using NTF. With the help of the detected matrices, a maximum likelihood algorithm is proposed in order to iteratively estimate the parameters of the model, exploiting the spatial redundancy of the observed mixture and using NTF. The proposed algorithm proves more flexibility and efficiency with respect to a baseline algorithm used as a reference.

This paper describes a new corpus of multi-channel audio data designed to study and develop distant-speech recognition systems able to cope with known interfering sounds propagating in an environment. The corpus consists of both real and simulated signals and of a corresponding detailed annotation. An extensive set of speech recognition experiments was conducted using three different Acoustic Echo Cancellation (AEC) techniques to establish baseline results for future reference. The AEC techniques were applied both to single distant microphone input signals and beamformed signals generated using two state-of-the-art beamforming techniques. We show that the speech recognition performance using the different techniques is comparable for both the simulated and real data, demonstrating the usefulness of this corpus for speech research. We also show that a significant improvement in speech recognition performance can be obtained by combining state-of-the-art AEC and beamforming techniques, compared to using a single distant microphone input.

This paper proposes a new method of frequency-domain blind source separation (FD-BSS), able to separate acoustic sources in challenging conditions. In frequency-domain BSS, the time-domain signals are transformed into time-frequency series and the separation is generally performed by applying independent component analysis (ICA) at each frequency envelope. When short signals are observed and long demixing filters are required, the number of time observations for each frequency is limited and the variance of the ICA estimator increases due to the intrinsic statistical bias. Furthermore, common methods used to solve the permutation problem fail, especially with sources recorded under highly reverberant conditions. We propose a recursively regularized implementation of the ICA (RR-ICA) that overcomes the mentioned problem by exploiting two types of deterministic knowledge: 1) continuity of the demixing matrix across frequencies; 2) continuity of the time-activity of the sources. The recursive regularization propagates the statistics of the sources across frequencies reducing the effect of statistical bias and the occurrence of permutations. Experimental results on real-data show that the algorithm can successfully perform a fast separation of short signals (e.g., 0.5-1s), by estimating long demixing filters to deal with highly reverberant environments (e.g., ms).

This paper describes a surveillance system for intrusion detection which is based only on information derived from the processing of audio signals acquired by a distributed microphone network (DMN). In particular the system exploits different acoustic features and estimates of acoustic event positions in order to detect intrusion and reject possible false alarms that may be generated by sound sources inside and outside the monitored room. An evaluation has been conducted in order to measure the performance in terms of false alarms and missed alarms in presence of acoustic events produced inside and outside a test room.The obtained results are very promising and encouraging for future works aimed at improving the actual system accuracy.

A novel method is presented for a robust two channel multiple time difference of arrival (TDOA) estimation for multispeaker localization which can provide satisfactory performance even in highly reverberant environment. The method is based on a recursive frequency-domain independent component analysis (ICA) and on a novel state coherence transform (SCT). Exploiting the phase coherence of the demixing matrices obtained in the ICA stage the SCT is able to generate envelopes with clear peaks in the corresponding maximum-likelihood TDOAs. The SCT envelopes are computed independently in each time-block and accurate multiple TDOAs are estimated by means of a time-frequency sparse representation of the sources. The method has been applied to real data obtained by recording many sources in a room with a reverberation time of 700 ms. Experimental results show that an accurate localization of 7 closely-spaced sources is possible given only few seconds of data even in the case of low SNR. Experiments also show the advantage of using the proposed solution rather than the well-known GCC-PHAT.

An interface for distant-talking control of home devices requires the possibility of identifying the positions of multiple users. Acoustic maps, based either on global coherence field (GCF) or oriented global coherence field (OGCF), have already been exploited successfully to determine position and head orientation of a single speaker. This paper proposes a new method using acoustic maps to deal with the case of two simultaneous speakers. The method is based on a two step analysis of a coherence map: first the dominant speaker is localized; then the map is modified by compensating for the effects due to the first speaker and the position of the second speaker is detected. Simulations were carried out to show how an appropriate analysis of OGCF and GCF maps allows one to localize both speakers. Experiments proved the effectiveness of the proposed solution in a linear microphone array set up.

Acoustic maps created on the basis of the signals acquired by distributed networks of microphones allow to identify position and orientation of an active talker in an enclosure. In adverse situations of high background noise, high reverberation or unavailability of direct paths to the microphones, localization may fail. This paper proposes a novel approach to talker localization and estimation of head orientation based on the classification of global coherence field (GCF) or oriented GCF maps. Preliminary experiments with data obtained by simulated propagation as well as with data acquired in a real room show that the match with precalculated map models provides a robust behavior in adverse conditions.

Abstract This paper proposes a new speaker localization method that isbased on a preliminary estimation of the head orientation. The ba-sic information on which the estimation is accomplished is calledOriented Global Coherence Field (OGCF).The new algorithm is shown to be significantly more robustthan the traditional ones so far explored. Its robustness is also dueto an effective speech activity detection, implicitly performed bya thresholding technique applied to OGCF information. To showthe performance of the proposed system, experiments were con-ducted on the NIST RT-05 Spring Evaluation source localizationtask, which is based on real recordings of lectures in noisy andreverberant environments. Index Terms : speaker localization, head orientation, microphonearrays, global coherence field. 1. Introduction Since 1990, several Speaker LOCalization (SLOC) techniqueshave been proposed as reported in [1, 2]. Most of the traditionalSLOC techniques are based on the estimation of time differencesof wavefront arrival at each sensor and on a consequent applica-tion of geometrical information to infer the acoustic source posi-tions. One of the most common techniques for Time Delay Es-timation (TDE) is based on Generalized Cross-Correlation PhaseTransform (GCC-PHAT) [3, 4]. Other effective SLOC techniquesare based on a preliminary computation of an acoustic map, asfor instance the Global Coherence Field (GCF) [5] representation,fromwhichthemostlikelysourcepositionisderivedthroughmax-imization in space.This paper aims at describing a new SLOC method that wasconceived starting from the effectiveness of the Oriented GlobalCoherence Field(OGCF), introduced in [6], which allows to char-acterize the orientation of an active speaker’s head with a satis-factory accuracy (in terms of angle error) even under reverberantconditions. By exploiting OGCF information, one can also derivemore robust speaker position estimates, since they are mostly re-lated to the propagation of a direct wavefront from a given point.On the other hand, previous SLOC techniques did not deal withthe way the sound is being radiated from a hypothesized positionin space.Theproposed method requires touseadistributed microphonenetwork similar to those available in the laboratories involved inthe EC CHIL

To realize the long-term goal of ubiquitous computing, technological advances in multi-channel acoustic analysis are needed in order to solve several basic problems, including speaker localization and tracking, speech activity detection (SAD) and distant-talking automatic speech recognition (ASR). The European Commission integrated project CHIL, “ Computers in the Human Interaction Loop”, aims to make significant advances in these three technologies. In this work, we report the results of our initial automatic source localization, speech activity detection, and speech recognition experiments on the CHIL seminar corpus, which is comprised of spontaneous speech collected by both near- and far-field microphones. In addition to the audio sensors, the seminars were also recorded by calibrated video cameras. This simultaneous audio-visual data capture enables the realistic evaluation of component technologies as was never possible with earlier data bases.

This paper refers to an activity under way at the speech recognition technology level for the development of a hands-free dialogue interaction system in the car environment. The use of a set of HMM recognizers, running in parallel, is being investigated in order to ensure low complexity, modularity, fast response, and to allow a real-time reconfiguration of the language models and grammars according to the policy indicated by natural language understanding and dialogue manager modules. A corpus of spontaneous speech interactions was collected using the Wizard-of-Oz method in a real driving situation with a microphone placed far from the driver. The use of parallel recognition units, each specialized on a given geographical domain, was explored using the resulting real corpus. Experiments show the advantage of selecting the recognized sentence according to the maximum likelihood among the active units when compared to the use of a single language model based on a very large vocabulary.

This paper refers to an activity under way at the speech recognition technology level for the development of a hands-free dialogue interaction system in the car environment. The work here presented concerns the use of two noise reduction techniques, as well as of MLLR adaptation, for recognition error reduction in low and medium complexity tasks, namely connected digits and spelling with or without bigram/trigram statistical constraints. Experiments are based on the use of SpeechDat Car database, a corpus collected under real noisy conditions. Results show the additive improvements in performance, obtained by adopting noise reduction techniques and MLLR adaptation.

A database of in-car speech for the Italian language was collected under the European projects SpeechDatCar and VODIS II. It consists of 600 sessions recorded under various noise and driving conditions and includes close-talk signals and far microphone signals for hands-free interaction. This paper describes some recognition experiments on two tasks conceived on a portion of this database: connected digit sequences and isolated command words. Recognition rate achieved by means of HMMs trained on real in-car speech is compared with that accomplished by a speech contamination approach, which aims at simulating in-car data starting from a clean speech corpus. Recognition performance is also analyzed as a function of the different noise conditions and of the consequent SNR at the far microphones. Finally, the effect of HMM adaptation is investigated in order to tune the recognizer on the conditions of the various sessions.

This paper addresses the problem of hands-free speech recognition in a noisy office environment. An array of six omnidirectional microphones and a corresponding time delay compensation module are used to provide a beamformed signal as input to a HMM-based recognizer. Training of HMMs is performed either using a clean speech database or using a filtered version of the same database. Filtering consists in a convolution with the acoustic impulse response between the speaker and microphone, to reproduce the reverberation effect. Background noise is summed to provide the desired SNR. The paper shows that the new models trained on these data perform better than the baseline ones. Furthermore, the paper investigates on maximum likelihood linear regression (MLLR) adaptation of the new models. It is shown that a further performance improvement is obtained, allowing to reach a 98.7% WRR in a connected digit recognition task, when the talker is at 1.5 m distance from the array.

A scenario concerning hands-free connected digit recognition in a noisy office environment is investigated. An array of six omnidirectional microphones and a corresponding time delay compensation module are used to provide a beamformed signal as input to a hidden Markov model (HMM) based recognizer. Two different techniques of phone HMM adaptation have been considered, to reduce the mismatch between training and test conditions. Adaptation material and test material were collected in two different sessions. Results show that a digit accuracy close to 98% can be achieved when the talker is at 1.5 m distance from the array. This result has to be compared with 99.5% accuracy obtained by using a close-talk microphone.

The use of a microphone array for hands-free continuous speech recognition in noisy and reverberant environment is investigated. An array of eight omnidirectional microphones was placed at different angles and distances from the talker. A time delay compensation module was used to provide a beamformed signal as input to a hidden Markov model (HMM) based recognizer. A phone HMM adaptation, based on a small amount of phonetically rich sentences, further improved the recognition rate obtained by applying only beamforming. These results were confirmed both by experiments conducted in a noisy and reverberant environment and by simulations. In the latter case, different conditions were recreated by using the image method to reproduce synthetic versions of the array microphone signals.

A microphone array can be used to locate a dominant acoustic source in a given environment. This capability is successfully employed to locate an active talker in teleconferencing or other multi-speaker applications. In this work the source location is obtained in two steps: (1) a time difference of arrival (TDOA) computation between the signals of the array; (2) an "optimal" source location based on the interchannel delay estimates and on a geometrical description of the sensor arrangement. The crosspower spectrum phase technique was used for TDOA estimation, while a maximum likelihood approach was followed to derive the source coordinates. Source location experiments in a three-dimensional space were performed by means of an array of 8 microphones. For this purpose both a loudspeaker and a real talker were used to collect data in a large noisy and reverberant room.

In this work hands-free continuous speech recognition based on microphone arrays is investigated. A set of experiments was carried out using arrays having different numbers of omnidirectional microphones as well as different configurations. Both real and simulated array signals, generated by means of the image method, were used. An enhanced input to a recognizer based on Hidden Markov Models was obtained by a time delay compensation module providing a beamformed signal. HMM adaptation was used to realign the recognizer acoustic modeling to the given acoustic condition.

The article reports on the use of crosspower-spectrum phase (CSP) analysis as an accurate time delay estimation (TDE) technique. It is used in a microphone array system for the location of acoustic events in noisy and reverberant environments. A corresponding coherence measure (CM) and its graphical representation are introduced to show the TDE accuracy. Using a two-microphone pair array, real experiments show less than a 10 cm average location error in a 6 m/spl times/6 m area.

A linear four microphone array can be employed for acoustic event location in a real environment using an accurate time delay estimation. This paper refers to the use of a specific technique, based on crosspower spectrum phase (CSP) analysis, that yielded accurate location performance. The behavior of this technique is investigated under different noise and reverberation conditions. Real experiments as well as simulations were conducted to analyze a wide variety of situations. Results show system robustness at quite critical environmental conditions.

This paper describes advances in the use of HMM based technology for speaker independent continuous speech recognition, in noisy environment, under hands free interaction mode. For this purpose an array of four omnidirectional microphones is employed as the acquisition system. The processing of phase information in the cross-power spectrum provides the capability both of locating the talker position and of reconstructing an enhanced speech spectrum. Two enhancement techniques are described, that provide recognition improvement in the case of clean input speech as well as under different adverse conditions. The results refer to the use of a new multichannel corpus, collected in a real environment by a microphone array as well as a close-talk microphone.

An extensive speech database was collected in two experimental enclosures for research on microphone arrays. The first enclosure is a digitally controlled variable acoustics facility where the reverberation time can be adjusted from 1.7 s to 0.1 s, and the second is a regular hard-walled laboratory. Two different sets of data were collected to address source location and spatial volume selectivity issues. Preliminary results on Matched-filter processing and on Source location are reported for the new database. I. Introduction Performance of microphone sound pick up is degraded by deleterious properties of the acoustic environment, such as multipath distortion (reverberation) and ambient noise. The degradation becomes more prominent in a teleconferencing environment in which the microphone is positioned far away from the speaker. Besides, the ideal teleconference should feel as easy and natural as face-to-face communication with another person. This suggests hands-free sound capture w...

Performance of a delay-and-sum beamformer is typically degraded in a reverberant enclosure because the beam captures not only the desired source signal (direct path) but also all images along the beam axis. Matched-filter processing of microphone arrays is shown to improve the quality of sound capture in reverberant environments. This paper also reports the performance of matched-filter array processing in a real room. Significant improvements are shown over single microphones and the traditional delay-and-sum beamformer.

Linear microphone arrays can be employed for acoustic event localization in a noisy environment using time delay estimation. Three techniques are investigated that allow delay estimation, namely normalized cross correlation, LMS adaptive filters, crosspower-spectrum phase: they are combined with a bidimensional representation, the coherence measure, in order to emphasize information that can be exploited for estimating position of both non-moving and moving acoustic sources. To compare the given techniques, different acoustic sources were considered, that generated events in different positions in space. Expressing performance in terms of accuracy of the wavefront direction angle, experiments showed that the crosspower-spectrum phase based technique outperforms the other two. This technique provided very promising preliminary results also in terms of source position estimation.>

Mismatch in training and testing conditions reduces considerably the performance of a speaker-independent HMM-based continuous speech recognizer. Compensation of this mismatch can avoid the complex and time-consuming retraining of the recognizer. This paper describes an acquisition system based on a four omnidirectional microphone array that was employed to reproduce a "beamformed" version of the original acoustic messages acquired in a noisy and reverberant environment, with a talker-microphone distance of one meter. In this preliminary activity, some simple noise compensation techniques (i.e. a Mean Spectrum based Enhancement and a Cepstrum Mean Subtraction) were incorporated in this preprocessing stage to obtain an enhanced version of the given utterance. Feeding a clean-condition trained continuous speech recognizer with enhanced signals led to a significant improvement of performance, if compared to the use of unprocessed single-microphone signals as input. I. INTRODUCTION Perfo...

This paper deals with the use of linear microphone arrays for detection, localization and enhancement of a generic acoustic message produced in a noisy environment. A CrosspowerSpectrum Phase based analysis and a Coherence Measure representation are presented, that allow an accurate time delay estimation employed for the acoustic source position hypothesis. Preliminary results in terms of source localization accuracy are given. Once source position is estimated, an enhanced version of the original acoustic message is derived, that can represent the input for a speech recognition system. Keywords: Microphone Arrays, Talker Localization, Speech Enhancement. 1. INTRODUCTION Automatic speech recognizer performance often degrades drastically when employed in conditions that are different with respect to those for which they were designed. One reason for this is environmental noise. Another reason can be inconsistent use of the acoustic transducers during message acquisition. For example,...