Joint factor analysis (JFA) is a recently developed method to model speaker and session variability in Gaussian Mixture Models (GMMs). In this paper, both batch and sequential Bayesian analysis of JFA models are evaluated for robust speaker recognition. Various sources of uncertainties in JFA models, from latent speaker and channel factors to Gaussian mixture indicator variables, are examined from a Bayesian perspective. By integrating over all these latent factors, we could better account for the sources of variability in speaker enrollment and verification processes than considering only point estimates; through this study, we could also analyze and identify the contribution of these various underlying model uncertainties to the final speaker verification performance. However, as all latent variables in JFA GMM become correlated with each other given observed data, it becomes practically intractable to do Bayesian analysis in closed analytic form. Hence, an alternative approach based on variational Bayes is developed in this paper to explore Bayesian JFA models in an approximate yet efficient way. In this method, fully correlated a posteriori distribution is approximated by a variational distribution of factored form to facilitate inference; and a lower bound on model likelihood is also derived to construct detection scores. Experimental results on the 2008 NIST Speaker Recognition Evaluation (NIST SRE) show that these variational Bayesian JFA models could obtain significant performance improvements over JFA using point estimates, especially for the cases with limited enrollment and test data. For the 10-s task in the 2008 NIST SRE, the variational Bayesian JFA systems obtained relatively 9.4% EER and 11.5% DCF reductions compared to the baseline JFA system. This paper also shows the importance of taking into account the uncertainties in both speaker and channel factors, which is more effective than considering uncertainties in channel factors alone.

From the results of the NIST speaker recognition evaluation in resent years, speaker recognition systems which are mainly developed based on English training data suffer the language gap problem, namely, the performance of non-English trails is much worse than that of English trails. This problem is addressed in this paper. Based on the conventional joint factor analysis model, we enrolled in the language factors which are mean to capture the language character of each testing and training speech utterance, and compensation was carried out by removing the language factors in order to shrink the difference between languages. Experiments on 2006 NIST SRE data show that, the language factor compensation alone can reduce the gap between the performance of English and non-English trails, and the score level combination with eigenchannels can further improve the performance of non-English trails, e.g., for female part, we observed about 19% relatively reduction in EER, when compared with eigenchannels session variability compensation alone.

Joint factor analysis (JFA) has been successfully applied to speaker verification tasks to tackle speaker and session variability. In the sense of Bayesian statistics, it is beneficial to take account of the uncertainties in JFA to better characterize its speaker enrollment and verification processes, e.g. representing target speaker model by posteriori distribution of latent speaker factors and evaluating model likelihood by integrating over all latent factors. However, in a JFA model which has a large number of latent factors, it is computationally demanding to carry out these things in their exact form. In this paper, an alternative approach based on variational Bayesian is developed to explore uncertainties in JFA in an approximate yet efficient way. In this method, fully correlated posteriori distribution is approximated by a variational distribution of factorial form to facilitate inference; and a tight lower bound on model likelihood is derived. Experimental results on the 10sec4w-10sec4w task of the 2006 NIST SRE show that variational Bayesian JFA could obtain better performance than JFA using point estimate.

Nuisance attribute projection (NAP) was successfully applied in SVM-based speaker verification systems to improve performance by doing projection to remove dimensions from the SVM feature space that cause unwanted variability in the kernel. Previous studies of NAP were focused mainly on linear and generalized linear kernel SVMs. In this paper, NAP in nonlinear kernel SVMs, e.g. polynomial or Gaussian kernels, are investigated. Instead of doing explicit feature expansion and projection in high-dimension feature space, kernel principal component analysis is employed to find nuisance dimensions; and, NAP is carried out implicitly by incorporating it into some compensated kernel functions. Experimental results on the 2006 NIST SRE corpus indicate the effectiveness of such nonlinear kernel NAP. Compared with linear NAP, nonlinear NAP with Gaussian kernel obtained about 11% relative improvement in equal error rate (EER).

Nuisance attribute projection (NAP) and within-class covariance normalization (WCCN) are two effective techniques for intersession variability compensation in SVM based speaker verification systems. However, by normalizing or removing the nuisance subspace containing the session variability can not guarantee to enlarge the distance between speakers. In this paper, we investigated the probability of using linear discriminant analysis (LDA) for discriminative training. To cope with the small sample size problem which prevents us from using LDA directly, we adapted the subspace LDA approach, which first projects the whole feature space into a relatively low dimensional subspace by PCA, and then performs LDA in the subspace. By some modification, the subspace LDA can be degenerated into a kind of WCCN approach, which we called subspace WCCN. Experiments on NIST SRE tasks showed that, the subspace WCCN outperformed the conventional direct WCCN, especially in low dimensional feature space.

Nuisance attribute projection (NAP) was an effective method to reduce session variability in SVM-based speaker verification systems. As the expanded feature space of nonlinear kernels is usually high or infinite dimensional, it is difficult to find nuisance directions via conventional eigenvalue analysis and to do projection directly in the feature space. In this paper, two different approaches to nonlinear kernel NAP are investigated and compared. In one way, NAP projection is formulated in the expanded feature space and kernel PCA is employed to do kernel eigenvalue analysis. In the second approach, a gradient descent algorithm is proposed to find out projection over input variables. Experimental results on the 2006 NIST SRE corpus show that both kinds of NAP can reduce unwanted variability in nonlinear kernels to improve verification performance; and NAP performed in expanded feature space using kernel PCA obtains slightly better performance than NAP over input variables. Index Terms: speaker verification, session variability modeling, support vector machines, nuisance attribute projection 1.

In this paper, we investigate SVM-based speaker verification by location in the space of reference speakers. Speaker location is represented by a vector of log-likelihoods of utterance data given reference speaker models. Channel or session variability in speaker locations due to microphone, acoustic environments etc. would impair verification performance. To reduce such variability, within-class covariance normalization (WCCN), nuisance attribute projection (NAP) and their combination are applied, and significant performance improvements are obtained. Experimental results on a NIST SRE 2006 task show that this location SVM system achieves comparable performance to a state-of-art cepstral GMM-UBM verification system, and their fusion can give additional performance gains.

In this paper, we propose the use of cluster adaptive training (CAT) weights as features in support vector machine (SVM) based text-independent verification task. The speaker utterance is characterized by a vector of cluster weights, which are extracted during the cluster adaptive training process. The effects of the number of classes, which are obtained by partitioning the components of the model, and the number of clusters on the verification performance are investigated. To remove session variability due to influences of microphone, environment, etc, Nuisance Attribute Projection (NAP) is also evaluated. Experimental results in a NIST SRE 2006 task show that this CAT weights SVM system achieves comparable performance to a state-of-the-art cepstral GMM-UBM verification system, and their fusion can give further performance gains.

In anchor modeling, each speaker utterance is represented as a fixed-length location vector in the space of reference speakers by scoring against a set of anchor models. SVMbased speaker verification systems using the anchor location representation have been studied in previously reported work with promising results. In this paper, linear combination weights in reference speaker weighting (RSW) adaptation are explored as an alternative kind of speaker location representation. And this kind of RSW location representation is compared with the anchor location representation in various speaker verification tasks on the 2006 NIST Speaker Recognition Evaluation corpus. Experimental results indicate that with long utterances for reliable maximum likelihood estimation in RSW, the RSW location representation leads to better speaker verification performance than the anchor location; while the latter is more effective for verification of short utterances in high-dimensional representation space. Index Terms: speaker verification, speaker location, anchor modeling, reference speaker weighting, support vector machines 1.

In conventional microphone array speech recognition, the array processor and the speech recognizer are loosely coupled. The only connection between the two modules is the enhanced target signal output from the array processor, which then gets treated as a single input to the recognizer. In this approach, useful environmental information, which can be provided by the array processor and also needs to be exploited by the recognizer, is ignored. Inherently, the array processor can generate multiple outputs of spatially filtered signals, as a multi-input-multi-output (MIMO) module. In this paper, a closely coupled approach is proposed, in which a recognizer with model-based noise compensation exploits the reference noise outputs from a MIMO array processor. Specifically, a multichannel model-based noise compensation is presented, including the compensation procedure using the vector Taylor series (VTS) expansion and parameter estimation using the expectation-maximization (EM) algorithm. It is also shown how to construct MIMO array processors from conventional beamformers. A number of practical implementations of the conventional loosely coupled approach and the proposed closely coupled approach were tested on a publicly available database, the Multichannel Overlapping Number Corpus (MONC). Experimental results showed that the proposed closely coupled approach significantly improved the speech recognition performance in the overlapping speech situations

When traditional maximum a posteriori (MAP) adaptation is used to adapt a universal background model (UBM), some model components with little or no enrollment data would remain unchanged in the derived speaker model. These model components would have weak discriminative capability over the background model, and would impair subsequent verification performance. In this paper, we present a new speaker adaptation method which combines MAP and reference speaker weighting (RSW) adaptation in a hierarchical, multigrained mode. It enables all model components to be updated in a way that strikes a good balance between model complexity and available data. The experimental results of NIST speaker recognition evaluation confirmed the effective performance increase with this new method compared with using MAP or RSW adaptation techniques alone

In this paper, a new microphone array speech recognition system in which the array processor and the speech recognizer are closely coupled is studied. The system includes a generalized sidelobe canceller (GSC) beamformer followed by a recognizer with vector Taylor series (VTS) compensation. The GSC beamformer provides two outputs, allowing more information to be used in the recognizer. One is the enhanced target speech output, the other is the reference noise output. VTS is used to compensate the effect of the residual noise in the GSC speech output, utilizing the GSC reference noise output. The compensation is done in a minimum mean square error (MMSE) sense. Moreover, an iteration procedure using an expectation-maximization (EM) algorithm is developed to refine the compensation parameters. Experimental results on the MONC database showed that the new system significantly improved the speech recognition performance in overlapping speech situations.