This paper proposes a robust voice activity detector (VAD) based on the observation that the distribution of speech captured with far-field microphones is highly varying, depending on the noise and reverberation conditions. The proposed VAD employs a convex combination scheme comprising three statistical distributions - a Gaussian, a Laplacian, and a two-sided Gamma - to effectively model captured speech. This scheme shows increased ability to adapt to dynamic acoustic environments. The contribution of each distribution to this convex combination is automatically adjusted based on the statistical characteristics of the instantaneous audio input. To further improve the performance of the system, an adaptive threshold is introduced, while a decision-smoothing scheme caters to the intra-frame correlation of speech signals. Extensive experiments under realistic scenarios support the proposed approach of combining several models for increased adaptation and performance.

The problem of detecting the location of an active acoustic source in an enclosure remains subject to a series of difficulties. Algorithms operate repeatedly on small frames of data from microphone recordings and provide estimates of the current source location. In a typical room, the quality of these recordings is affected by noise and reverberation. Additionally, the presence of silence gaps in speech signals and possible competing speakers can reduce tracking accuracy further. We discuss a novel localization and tracking framework that is based on particle filtering. This is driven by detection methods based on information theory that remain robust under reverberant and noisy environments. Integrating a second particle filter allows the system to track interchanging acoustic sources that reside far apart. A further extension involves the integration of a voice activity detection scheme that uses the same detection measures and deals with human-speech gaps. Performance is first examined using simulations that parameterize results according to environmental variables like reverberation and system geometry. The system is then used in a real-world scenario with data from multi-person meetings. Results indicate that the proposed framework outperforms all systems used for comparison in this work while remaining adequately robust in the examined environments.

Voice activity detection systems attempt to discriminate between voice and other ambient sounds. Most systems use a single microphone approach and rely on training prior to employment. The performance of these systems relies heavily on reverberation and noise levels. In this paper we present an unsupervised voice activity detection system that uses pairs of microphones to discern between a coherent acoustic source and spatially diffuse noise of low coherence. Measurement of coherency is performed using an information theoretic metric that integrates means to filter out more effectively the effect of reverberation and noise. Using extensive experiments, the performance of the system is investigated. Based on the conditions imposed by the experimental environments it is shown that the proposed system remains more robust than its counterparts in all cases.

We propose a system for detecting the active speaker in cluttered and reverberant environments where more than one person speaks and moves. Rather than using only audio information, the system utilizes audiovisual information from multiple acoustic and video sensors that feed separate audio and video tracking modules. The audio module operates using a particle filter (PF) and an information-theoretic framework to provide accurate acoustic source location under reverberant conditions. The video subsystem combines in 3-D a number of 2-D trackers based on a variation of Stauffer's adaptive background algorithm with spatiotemporal adaptation of the learning parameters and a Kalman tracker in a feedback configuration. Extensive experiments show that gains are to be expected when fusion of the separate modalities is performed to detect the active speaker.

We propose a system for detecting the active speaker in cluttered and reverberant environments where more than one person speaks and moves. Rather than using only audio information, the system utilizes audiovisual information from multiple acoustic and video sensors that feed separate audio and video tracking modules. The audio module operates using a particle filter (PF) and an information-theoretic framework to provide accurate acoustic source location under reverberant conditions. The video subsystem combines in 3-D a number of 2-D trackers based on a variation of Stauffer's adaptive background algorithm with spatiotemporal adaptation of the learning parameters and a Kalman tracker in a feedback configuration. Extensive experiments show that gains are to be expected when fusion of the separate modalities is performed to detect the active speaker.

This paper proposes a system for tracking people in three dimensions, utilizing audiovisual information from multiple acoustic and video sensors. The proposed system comprises a video and an audio subsystem combined using a Kalman filter. The video subsystem combines in 3D a number of 2D trackers based on a variation of Stauffer's adaptive background algorithm with spatio-temporal adaptation of the learning parameters and a Kalman tracker in a feedback configuration. The audio subsystem uses an information theoretic metric upon a pair of microphones to estimate the direction from which sound is arriving from. Combining measurements from a series of pairs the actual coordinate of the speaker in space is derived. Experiments show that gains are to be expected when fusion of the separate tracking systems is performed

We study the effect of reverberation and source movement on the performance of blind source separation and deconvolution (BSSD) algorithms. Using the model of statistical room acoustics we derive theoretical performance measures for a class of unmixing algorithms when these are used in a reverberant room. We specifically investigate the cases: 1) where separation of only direct paths is performed and 2) the case where unmixing of the full reverberant paths is attempted. We develop closed-form performance measures that are dependent on the geometry used and the chosen unmixing system. Using these measures allows us to draw general conclusions on the robustness to source movement of typical BSSD algorithms. Results indicate that performance of systems that show very good separation in static reverberant environments is significantly reduced when sources move, with performance degrading to that of simple direct-path separation

Estimating the direction of arrival (DOA) of an acoustic source relies on the successful estimation of the relative delay between pairs of microphone signals. Processing is performed at the current time by operating on blocks of recorded data. When these recordings are performed in environments of strong multipath reflections, algorithms often fail to distinguish between the true DOA and that of a dominant reflection. In this letter, we assume Gaussianity of the source signal and use an information-theoretical measure, often met in blind source separation algorithms, to derive a robust DOA estimator, even under significant reverberant conditions. We discuss the most popular algorithm for time delay estimation, namely, the generalized cross-correlation method, and demonstrate under certain conditions its connection to the proposed one. Performance is demonstrated for both algorithms with sets of simulated results as a function of different reverberation times, microphone spacing, and data block size. The results indicate that the examined framework can accurately track the DOA of a typical acoustic source.

Using statistical room acoustics, we investigate the performance of blind source separation and deconvolution (BSSD) algorithms when used in a reverberant room. We focus on the case where one of the sources moves, and examine the relative impact of source movement and room reverberation on the expected performance. We derive theoretical expressions, and verify these through image model simulations.