In this paper we present a solution to the problem of tracking and separation of a mixture of concurrent sources in a reverberant environment where the number of sources is unknown and varies with time: new sources can appear and existing sources can disappear or undergo silence periods. In order to deal with this challenging problem, we synergistically combine two key ideas, one in the front end and the other at the back end. In the front end we employ independent component analysis (ICA) to demix the mixtures and the state coherence transform (SCT) to represent the signals in a direction of arrival (DOA) detection framework. By exploiting the frequency sparsity of the sources, ICA/SCT is even effective when the number of simultaneous sources is greater than the number of sensors therefore allowing for minimal number of sensors to be used. At the back end, the probability hypothesis density (PHD) filter is incorporated in order to track the multiple DOAs and determine the number of sources. The PHD filter is based on random finite sets (RFS) where the multi-target states and the number of targets are integrated to form a set-valued variable with uncertainty in the number of sources. A Gaussian mixture implementation of the PHD filter (GM-PHD) is utilized that solves the data association problem intrinsically, hence providing distinct DOA tracks. The distinct tracks also make the separation task possible by going back and rearranging the outputs of the ICA stage. The tracking and separation capabilities of the proposed method is demonstrated using simulations of multiple sources in reverberant environments.

In this paper, we explore the problem of separating and localizing multiple maneuvering speakers in a reverberant environment. Moreover, the speakers can become silent intermittently, making the problem more challenging especially when the speaker is moving while silent. A two stage method is proposed. In the first stage a multiple model particle filter (MMPF) is used to track the mixing matrices by “glimpsing” or listening in the silence gaps as the set of active speakers change with time. In the second stage, a secondary microphone array is utilized to localize and track the speakers by intersecting their directions of arrival (DOA) obtained from estimates of the tracked mixing matrices. Computer simulations using the proposed method are conducted showing improved performance when compared with an online independent component analysis (ICA)-based algorithm.

In this paper, we explore the problem of separating convolutedly mixed signals in the overcomplete (degenerate) case of having more sources than sensors. We exploit a common form of nonstationarity, especially present in speech, wherein the signals have silence periods intermittently, hence varying the set of active sources with time. A novel approach is proposed that takes advantage of different combinations of silence gaps in the source signals at each time period. This enables the algorithm to “glimpse” or listen in the gaps, hence compensating for the global degeneracy by allowing it to learn the mixing matrices at periods where it is locally less degenerate. Experiments using simulated and real room recordings were carried out yielding good separation results.

Independent vector analysis (IVA) is a method for separating convolutedly mixed signals that significantly reduces the occurrence of the well-known permutation problem in frequency domain blind source separation (BSS). In this paper, we develop a novel IVA-based unifying framework for overcomplete/complete/undercomplete convolutive noisy BSS. We show that in order for the sources to be separable in the frequency domain, they must have a temporal dynamic structure. We exploit a common form of dynamics, especially present in speech, wherein the signals have silence periods intermittently, hence varying the set of active sources with time. This feature is extremely useful in dealing with overcomplete situations. An approach using hidden Markov models (HMMs) is proposed that takes advantage of different combinations of silence gaps of the source signals at each time period. This enables the algorithm to “glimpse” or listen in the gaps, hence compensating for the global degeneracy by allowing it to learn the mixing matrices at periods where it is locally less degenerate. The same glimpsing strategy can be employed to the complete/undercomplete case as well. Moreover, additive noise is considered in our model. Real and simulated experiments were carried out for overcomplete convoluted mixtures of speech signals yielding improved separation results compared to a sparsity-based robust time-frequency masking method. Signal-to-disturbance ratio (SDR) and machine intelligibility of a speech recognizer was used to evaluate their performances. Experiments were also conducted for the classical complete setting using the proposed algorithm and compared with standard IVA showing that the results compare favorably.

Independent vector analysis (IVA) is a method for separating convolutedly mixed signals that avoids the well-known permutation problem in frequency domain blind source separation (BSS). In this paper, we exploit the nonstationarity of signals, a common feature, for BSS. One common type of nonstationarity, especially in speech, is that the signal can have silence periods intermittently, hence varying the set of active sources with time. To deal with such situations, we propose a novel state-based IVA algorithm. Moreover, we consider additive noise in our model. Computer simulations are conducted to compare the proposed method with the standard IVA and the results compare favorably.