In this paper, we propose a cross-lingual voice conversion (VC) scheme leveraging non-parallel and parallel methodologies. The goal of cross-lingual VC is to transform the voice of one speaker from a language dataset into the voice of another speaker from a different language dataset. First, two non-parallel methods are separately investigated, they are CycleGAN-VC2 and phonetic posteriorGrams (PPG) VC. Second, two different parallel VC systems are developed to enhance the quality of the converted speech spectrogram, where the output speech from the non-parallel VC is used to form the parallel pair with the corresponding original speech. Focusing on Mandarin-English bilingual databases, the proposed VC scheme improves speech naturalness and speaker similarity as compared to the baseline non-parallel methods.

Replay speech poses a growing threat to speaker verification systems, thus the detection of replay speech becomes increasingly important. A critical factor differentiating replay speech and genuine speech is the representation of device information. Replay speech carries physical device information that originates from recording device, playback device, and environmental noise. In this work, a device-related linear transformation strategy is proposed to disentangle non-device information from replay speech. First, we conduct factor analysis by introducing a common vector for both replay utterance and the corresponding genuine speech utterance on parallel training data; then, we derive an expectation maximization formula to obtain the parameters of the device-related linear transformation; subsequently, three device feature extraction methods are developed based on the device-related linear transformation. The developed device features are evaluated on ASVspoof 2017 version 2.0 and ASVspoof 2021 physical access corpora. The experimental results demonstrate that our proposed linear transformation strategy is effective for replay spoofing detection, and the resultant device features outperform many typical features. Moreover, our spoofing detection systems display superior performance over several competitive state-of-the-art systems.

A device feature, which contains information from both the recording channel and playback channel, is of paramount importance in replay spoofing detection. Presently, no technical reports, pertaining to the usage of device information for spoofing detection to achieve speaker verification, have been published. In this article, we propose to build a replay device feature (RDF) extractor on the basis of the genuine-replay-pair parallel neural network training database. The target of the neural network for such an RDF extractor is constrained in a spectrum domain such as discrete Fourier transform or constant-Q transform. Different types of neural networks are investigated for the parallel training framework. Finally the RDF extractor is formed by applying discrete cosine transform to the output vector of neural network. The bidirectional long short term memory gives the best performance among the investigated types of neural networks. The experimental results on ASVspoof 2017 corpus version 2.0 indicate that the equal error rate (EER) of a replay detection system with the proposed RDF has a value of 15.08%. Furthermore, by combining the RDF with constant-Q cepstral coefficients plus the log energy in score level, the EER of the detection system can be further reduced to 8.99%. In addition, the experimental results also reveal that the RDF exhibits great complementarity with many handcrafted features.

Speech recognition performance deteriorates in face of unknown noise. Speech enhancement offers a solution by reducing the noise in speech at runtime. However, it also introduces artificial distortions to the speech signals. In this paper, we aim at reducing the artifacts that has adverse effects on speech recognition. With this motivation, we propose a modification scheme including smoothing adaptation to frame SNR and reestimation of a priori SNR for spectral-domain log-spectral-amplitude (LSA) speech enhancement. The experiments show that the proposed scheme of enhancement significantly improves the performance of the state-of-the-art speech recognition over the baseline speech enhancement.

Recently, joint factor analysis (JFA) and identity-vector (i-vector) represent the dominant techniques used for speaker recognition due to their superior performance. Developed relatively earlier, the Gaussian mixture model - support vector machine (GMM-SVM) with nuisance attribute projection (NAP) has gradually become less popular. However, when developing the relevance factor in maximum a posteriori (MAP) estimation of GMM to be adapted by application data in place of the conventional fixed value, it is noted that GMM-SVM demonstrates some advantages. In this paper, we conduct a comparative study between GMM-SVM with adaptive relevance factor and JFA/i-vector under the framework of Speaker Recognition Evaluation (SRE) formulated by the National Institute of Standards and Technology (NIST).

International audience

I4U is a joint entry of nine research Institutes and Universities across 4 continents to NIST SRE 2012. It started with a brief discussion during the Odyssey 2012 workshop in Singapore. An online discussion group was soon set up, providing a discussion platform for different issues surrounding NIST SRE’12. Noisy test segments, uneven multi-session training, variable enrollment duration, and the issue of open-set identification were actively discussed leading to various solutions integrated to the I4U submission. The joint submission and several of its 17 sub-systems were among top-performing systems. We summarize the lessons learnt from this large-scale effort.

In this paper, we study speech enhancement in eigen-domain. In our previous work on audible noise reduction, we use masking properties of the human auditory system to define the audible noise quantity in the eigen-domain. We then derived the speech enhancement algorithm using white noise model. Without loss of generality, in this paper, we introduce the noise reduction method for colored noise. Through many simulations, we show that colored noise modeling is superior over other existing eigen-decomposition methods in terms of objective and subjective evaluations.

International audience

In this paper, the relevance factor in maximum a posteriori (MAP) adaptation of Gaussian mixture model (GMM) from universal background model (UBM) is studied for language recognition. In conventional MAP, relevance factor is typically set as a constant empirically. Knowing that relevance factor determines how much the observed training data influence the model adaptation, thus the resulting GMM models, we believe that the relevance factor should be dependent to the data for more effective modeling. We formulate the estimation of relevance factor in a systematic manner and study its role in characterizing spoken languages with supervectors. We use a Bhattacharyya-based language recognition system on National Institute of Standards and Technology (NIST) language recognition evaluation (LRE) 2009 task to investigate the validate of the data-dependent relevance factor. Experimental results show that we achieve improved performance by using the proposed relevance factor. Index Terms: maximum a posteriori, supervector, Gaussian mixture model, support vector machine

Support vector machines (SVMs), and kernel classifiers in general, rely on the kernel functions to measure the pairwise similarity between inputs. This paper advocates the use of discrete representation of speech signals in terms of the probabilities of discrete events as feature for speaker verification and proposes the use of Bhattacharyya coefficient as the similarity measure for this type of inputs to SVM. We analyze the effectiveness of the Bhattacharyya measure from the perspective of feature normalization and distribution warping in the SVM feature space. Experiments conducted on the NIST 2006 speaker verification task indicate that the Bhattacharyya measure outperforms the Fisher kernel, term frequency log-likelihood ratio (TFLLR) scaling, and rank normalization reported earlier in literature. Moreover, the Bhattacharyya measure is computed using a data-independent square-root operation instead of data-driven normalization, which simplifies the implementation. The effectiveness of the Bhattacharyya measure becomes more apparent when channel compensation is applied at the model and score levels. The performance of the proposed method is close to that of the popular GMM supervector with a small margin.

Among conventional methods for text-independent speaker recognition, Gaussian mixture model (GMM) is known for its effectiveness and scalability in modeling the spectral distribution of speech. A GMM-supervector characterizes a speaker's voice by the GMM parameters such as the mean vectors, covariance matrices and mixture weights. Besides the first-order statistics, it is generally believed that speaker's cues are partly conveyed by the second-order statistics. In this paper, we introduce a Bhattacharyya-based GMM-distance to measure the distance between two GMM distributions. Subsequently, the GMM-UBM mean interval (GUMI) concept is introduced to derive a GUMI kernel which can be used in conjunction with support vector machine (SVM) for speaker recognition. The GUMI kernel allows us to exploit the speaker's information not only from the mean vectors of GMM but also from the covariance matrices. Moreover, by analyzing the Bhattacharyya-based GMM-distance measure, we extend the Bhattacharyya-based kernel by involving both the mean and covariance statistical dissimilarities. We demonstrate the effectiveness of the new kernel on the National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) 2006 dataset.

This paper describes the performance of the I4U speaker recognition system in the NIST 2008 Speaker Recognition Evaluation. The system consists of seven subsystems, each with different cepstral features and classifiers. We describe the I4U Primary system and report on its core test results as they were submitted, which were among the best-performing submissions. The I4U effort was led by the Institute for Infocomm Research, Singapore (IIR), with contributions from the University of Science and Technology of China (USTC), the University of New South Wales, Australia (UNSW), Nanyang Technological University, Singapore (NTU) and Carnegie Mellon University, USA (CMU).

Gaussian mixture model (GMM) supervector is one of the effective techniques in text independent speaker recognition. In our previous work, we introduce the GMM-UBM mean interval (GUMI) concept based on the Bhattacharyya distance. Subsequently GUMI kernel was successfully used in conjunction with support vector machine (SVM) for speaker recognition. Besides the first order statistics, it is generally believed that speaker cues are also partly conveyed by second order statistics. In this paper, we extend the Bhattacharyya-based SVM kernel by constructing the supervector with the mean statistical vector and the covariance statistical vector. Comparing with the Kullback-Leibler (KL) kernel, we demonstrate the effectiveness of the new kernel on the 2006 National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) dataset.

Gaussian mixture model (GMM) and support vector machine (SVM) have become popular classifiers in text-independent speaker recognition. A GMM-supervector characterizes a speaker's voice with the parameters of GMM, which include mean vectors, covariance matrices, and mixture weights. GMM-supervector SVM benefits from both GMM and SVM frameworks to achieve the state-of-the-art performance. Conventional Kullback-Leibler (KL) kernel in GMM-supervector SVM classifier limits the adaptation of GMM to mean value and leaves covariance unchanged. In this letter, we introduce the GMM-UBM mean interval (GUMI) concept based on the Bhattacharyya distance. This leads to a new kernel for SVM classifier. Comparing with the KL kernel, the new kernel allows us to exploit the information not only from the mean but also from the covariance. We demonstrate the effectiveness of the new kernel on the 2006 National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) dataset.

This paper introduces a spoken language recognition system with a generative front-end and a discriminative backend. The generative front-end is built upon an ensemble of Gaussian densities. These Gaussian densities are trained to represent elementary speech sound units characterizing a wide variety of languages. We formulate the generative front-end in a form of sequence kernel. This sequence kernel transforms a spoken utterance into a feature vector with its attributes representing the occurrence statistics of the speech sound units. A discriminative support vector machine (SVM) then operates on the feature vectors to make classification decision. The proposed language recognition system demonstrates competitive performance on NIST 1996, 2003 and 2005 LRE corpora.

In this paper, we investigate the contribution of the speech enhancement to telephony name speech recognition system that has been of the feature enhancement in noisy environment. Since the masking based subband Kalman filtering (MSKF) method has been shown to have good performance over many existing speech enhancement methods for human listening, we especially examine the MSKF speech enhancement method for the speech recognition system. We evaluate the performance of the speech enhancement methods based on the recorded telephony name speech database. Experimental results show that the recognition error rate of the speech recognition system can be reduced much by using MSKF.

Support vector machine (SVM) equipped with sequence kernel has been proven to be a powerful technique for speaker verification. A number of sequence kernels have been recently proposed, each being motivated from different perspectives with diverse mathematical derivations. Analytical comparison of kernels becomes difficult. To facilitate such comparisons, we propose a generic structure showing how different levels of cues conveyed by speech utterances, ranging from low-level acoustic features to highlevel speaker cues, are being characterized within a sequence kernel. We then identify the similarities and differences between the popular generalized linear discriminant sequence (GLDS) and GMM supervector kernels, as well as our own probabilistic sequence kernel (PSK). Furthermore, we enhance the PSK in terms of accuracy and computational complexity. The enhanced PSK gives competitive accuracy with the other two kernels. Fusing all the three kernels yields an EER of 4.83 % on the 2006 NIST SRE core test. Index Terms: speaker verification, characteristic vector, support vector machine, sequence kernel

In this paper, autoregressive parameter estimation for Kalman filtering speech enhancement is studied. In conventional Kalman filtering speech enhancement, spectral subtraction is usually used for speech autoregressive (AR) parameter estimation. We propose log spectral amplitude (LSA) minimum mean-square error (MMSE) instead of spectral subtraction for the estimation of speech AR parameters. Based on an observation that full-band Kalman filtering speech enhancement often causes an unbalanced noise reduction between speech and non-speech segments, a spectral solution is proposed to overcome the unbalanced reduction of noise. This is done by shaping the spectral envelopes of the noise through likelihood ratio. Our simulation results show the effectiveness of the proposed method.

This paper describes the derivation of a sequence kernel that transforms speech utterances into probabilistic vectors for classification in an expanded feature space. The sequence kernel is built upon a set of Gaussian basis functions, where half of the basis functions contain speaker specific information while the other half implicates the common characteristics of the competing background speakers. The idea is similar to that in the Gaussian mixture model – universal background model (GMM-UBM) system, except that the Gaussian densities are treated individually in our proposed sequence kernel, as opposed to two mixtures of Gaussian densities in the GMM-UBM system. The motivation is to exploit the individual Gaussian components for better speaker discrimination. Experiments on NIST 2001 SRE corpus show convincing results for the probabilistic sequence kernel approach.

A signal subspace scheme based on masking properties is proposed for enhancement of speech degraded by additive noise. Since the masking properties are related to the critical frequency band that is derived from the characteristics of human cochlea, the incorporation of masking threshold into a subspace technique requires the transformation between the frequency and eigen domains. We present and apply an invertible transformation between the frequency and eigen domains. In this paper, we use masking properties of the human auditory system to define the audible noise quantity in the eigendomain. We derive the eigen-decomposition of the estimated speech autocorrelation matrix with the assumption of white noise. Subsequently, an audible noise reduction scheme is developed based on a signal subspace technique, and the implementation of our proposed scheme is outlined. We further extend the scheme to the colored noise case. Simulation results show the superiority of our proposed scheme over other existing subspace methods in terms of segmental signal-to-noise ratio (SNR), perceptual evaluation of speech quality (PESQ), modified Bark spectral distortion (MBSD), spectrogram and informal listening tests.

To enhance the noisy speech signal, a perceptual weighting based Kalman filtering is investigated. Our study is to seek a high perceptual quality of speech enhancement system which optimizes the trade-off between the speech distortion and noise reduction. Using perceptual weighting to replace the masking threshold avoids the frequency domain complexity, it is suitable for time domain Kalman filtering to estimate the state-space vector in only time domain. Through many simulations, it is demonstrated that the proposed perceptual Kalman filtering outperforms the conventional Kalman filtering.

A novel subspace-based speech enhancement scheme, based on a criterion of audible noise reduction, is considered. The masking properties of the human auditory system are used to define the audible noise quantity in the eigen-domain. Subsequently, an audible noise reduction scheme is developed based on a signal subspace technique. We derive the eigendecomposition of the estimated speech autocorrelation matrix with the assumption of white noise and outline the implementation of our proposed scheme. We further extend the scheme to the colored noise case. Simulation results show that our proposed scheme outperforms many existing subspace methods in terms of segmental signal-to-noise ratio (SNR), perceptual evaluation of speech quality (PESQ) and informal listening tests.

Masking properties have been widely exploited in speech enhancement techniques, especially those implemented in the spectral domain. The incorporation of auditory masking in a subspace technique invariably requires a transformation linking the frequency and eigendomains. In this letter, an invertible transformation between the frequency and eigendomains is derived. The proposed transformation is verified through a conventional masking-based subspace speech-enhancement method. Simulation results show that our proposed transformation for speech enhancement outperforms the conventional transformation in terms of segmental signal-to-noise ratio (SNR), perceptual evaluation of speech quality (PESQ), and listening tests.

This paper proposes /spl beta/-order minimum mean-square error (MMSE) speech enhancement approach for estimating the short time spectral amplitude (STSA) of a speech signal. We analyze the characteristics of the /spl beta/-order STSA MMSE estimator and the relation between the value of /spl beta/ and the spectral amplitude gain function of the MMSE method. We further investigate the effectiveness of a range of fixed-/spl beta/ values in estimating STSA based on the MMSE criterion, and discuss how the /spl beta/ value could be adapted using the frame signal-to-noise ratio (SNR). The performance of the proposed speech enhancement approach is then evaluated through spectrogram inspection, objective speech distortion measures and subjective listening tests using several types of noise sources from the NOISEX-92 database. Evaluation results show that our approach can achieve a more significant noise reduction and a better spectral estimation of weak speech spectral components from a noisy signal as compared to many existing speech enhancement algorithms.

The paper describes a new speech enhancement approach which employs adaptive /spl beta/-order minimum mean square error (MMSE) spectral estimation of the short time spectral amplitude (STSA) of a speech signal. In the proposed approach, the human perceptual auditory masking effect is incorporated into the speech enhancement algorithm. The relationship between the value of /spl beta/ and the noise masking threshold is considered. The algorithm is based on a criterion by which the audible noise may be masked rather than being attenuated, thereby reducing the chance of speech distortion. Performance assessment is given to show that our proposal can achieve a more significant noise reduction and a better spectral estimation of weak speech spectral components from a noisy signal as compared to many existing speech enhancement algorithms.

A single channel speech enhancement system based on Kalman filtering and masking properties of the human auditory system is studied. The main objective of our study is to develop a high quality speech enhancement system for car mobile communication, which optimizes the tradeoff between speech distortion and noise reduction. To develop a robust Kalman filtering speech enhancement system, we propose that the noise variance estimate be modified as a function of masking threshold in each subband. By using such a Kalman filtering with masking method, we can achieve better results as compared to full band Kalman filtering, especially in the case of weak speech spectral components in noise. Through a great deal of simulations, we show the advantages of our proposed system in terms of objective and subjective measurements as compared to many existing speech enhancement methods.

This paper introduces an adaptive /spl beta/-order minimum mean square error (MMSE) spectral estimator for the short time spectral amplitude (STSA) of speech. The characteristic of /spl beta/-order MMSE attenuation function is introduced and analyzed. The performance of the proposed adaptive /spl beta/-order MMSE has been thoroughly examined by a large number of computer simulations. The new proposed scheme has been found to outperform the conventional a priori SNR Wiener filtering, the Ephraim and Malah (1984) STSA-MMSE and log spectral amplitude (LSA) schemes. It can achieve a more significant noise reduction and a better spectral estimation for weak speech components from a noisy speech signal as compared to the conventional schemes.