Many feature enhancement methods make use of probabilistic models of speech and noise in order to improve performance of speech recognizers in the presence of background noise. The traditional approach for training such models is maximum likelihood estimation. This paper investigates the novel application of variational Bayesian learning for front-end models under the Algonquin denoising framework. Compared to maximum likelihood training, it is shown that variational Bayesian learning has advantages both in terms of increased robustness with respect to choice of model complexity, as well as increased performance.

This paper briefly reviews state of the art related to the topic of speech variability sources in automatic speech recognition systems. It focuses on some variations within the speech signal that make the ASR task difficult. The variations detailed in the paper are intrinsic to the speech and affect the different levels of the ASR processing chain. For different sources of speech variation, the paper summarizes the current knowledge and highlights specific feature extraction or modeling weaknesses and current trends

Amplitude modulation (AM) and frequency modulation (FM) have been well defined and studied in the context of communications systems(S. Haykin, 1994). Borrowing upon these ideas, several researchers have applied AM-FM (V. Tyagi et al., 2003, M. Athineos et al., 2004, Q. Zhu and A. Alwan, 2000, B.E.D. Kingsbury et al., 1998) modeling for speech signals with mixed results. These techniques have varied in their definition and consequently the demodulation methods used therein. In this paper, we carefully define AM and FM signals in the context of ASR. We show that for a theoretically meaningful estimation of the AM signal, it is necessary to decompose the speech signal into several narrow spectral bands as opposed to the previous use of the speech modulation spectrum (V. Tyagi et al., 2003, M. Athineos et al., 2004, Q. Zhu and A. Alwan, 2000, B.E.D. Kingsbury et al., 1998), which was derived by decomposing the speech signal into increasingly wider spectral bands (such as critical, Bark or Mel). Due to the Hilbert relationships, the AM signal induces a component in the FM signal which is fully determinable from the AM signal (R. Kumaresan and A. Rao, 1999, V. Tyagi and C. Wellekens, 2005). We present a novel homomorphic filtering technique to extract the leftover FM signal after suppressing the redundant part of the FM signal. The estimated AM message signals are downsampled and their lower DCT coefficients are retained as speech features. These features carry information that is complementary to the MFCCs. A Tandem (H. Hermansky, 2003) combination of these two features is shown to improve recognition accuracy

This paper addresses distant-talking speech recognition by means of remote sensors in a reverberant room. Recognition performances is investigated for different ways of initializing, steering, and optimizing the related beamformer. Results show how much critical that front-end processing may be in such a challenging setup, according to the different positions and orientations of the speaker

It is well known that the peaks in the spectrum of a log Mel-filter bank are important cues in characterizing speech sounds. However, low energy perturbations in the power spectrum may become numerically significant after the log compression. We show that even if the spectral peaks are kept constant, the low energy perturbations in the power spectrum can create huge variations in the cepstral coefficients. We show, both analytically and experimentally, that exponentiating the log Mel-filter bank spectrum before the cepstrum computation can significantly reduce the sensitivity of the cepstra to spurious low energy perturbations. The Mel-cepstrum modulation spectrum (Tyagi, V. et al., Proc. IEEE ASRU, 2003) is computed from the processed cepstra which results in further noise robustness of the composite feature vector. In experiments with speech signals, it is shown that the proposed technique based features yield a significant increase in speech recognition performance in non-stationary noise conditions when compared directly to the MFCC and RASTA-PLP features.

An approach based on variational Bayesian feature saliency (VBFS) for robust audio type classification is proposed. VBFS aims at finding the most discriminative features in Gaussian mixture model based recognition systems. VBFS is applied to capture inter-type and intra-type feature saliency for different audio types (music, background noise, wide band speech, narrow band speech, etc.) in order to increase a model generality that is always poor in non-speech models. We show that inferring saliency for different audio types improves classifications. Experiments are run on broadcast news 1996-Hub4.

In this paper we explore the use of variational Bayesian (VB) learning for adaptation in a speaker clustering framework. Variational learning offers the interesting property of making model learning and model selection at the same time. We compare VB learning with a classical MAP/BIC (MAP for training, BIC for model selection) approach. Results on the NIST BN-96 HUB4 database show that VB learning can outperform the classical MAP-BIC method.

It is often acknowledged that speech signals contain short-term and long-term temporal properties that are difficult to capture and model by using the usual fixed scale (typically 20ms) short time spectral analysis used in hidden Markov models (HMMs), based on piecewise stationarity and state conditional independence assumptions of acoustic vectors. For example, vowels are typically quasi-stationary over 40-80ms segments, while plosives typically require analysis below 20ms segments. Thus, fixed scale analysis is clearly sub-optimal for ``optimal'' time-frequency resolution and modeling of different stationary phones found in the speech signal. In the present paper, we investigate the potential advantages of using variable size analysis windows towards improving state-of-the-art speech recognition systems. Based on the usual assumption that the speech signal can be modeled by a varying autoregressive (AR) Gaussian process, we estimate the largest piecewise quasi-stationary speech segments, based on the likelihood that a segment was generated by the same AR process. This likelihood is estimated from the Linear Prediction (LP) residual error. Each of these quasi-stationary segments is then used as an analysis window from which spectral features are extracted. Such an approach thus results in variable scale time spectral analysis, adaptively estimating the largest possible analysis window size such that the signal remains quasi-stationary, thus the best temporal/frequency resolution tradeoff. Speech recognition experiments on the OGI Numbers95 database show that the proposed multi-scale piecewise stationary spectral analysis based features indeed yield improved recognition accuracy in clean conditions, compared to features based on minimum cross entropy spectrum as well as those based on fixed scale spectral analysis.

In this paper we show that feature selection problem can be formulated as a model selection problem. A Bayesian framework for feature selection in unsupervised learning based on Gaussian mixture models is applied to speech recognition. In the original formulation (Figueiredo (2002)) a minimum message length criterion is used for model selection; we propose a new model selection technique based on variational Bayesian learning that shows a higher robustness to the amount of training data. Results on speech data from the TIMIT database show a high efficiency in determining feature saliency.

This paper aims at comparing the Bayesian Information Criterion and the Variational Bayesian approach for scoring unknown multi-ple speakerclustering. Variational Bayesian learning is a very effec-tive method that allows parameter learning and model selection at the same time. The application we consider here consists in finding the optimal clustering in a conversation where the speaker number is not a priori known. Experiments are run on synthetic data and on the evaluation data set NIST-1996 HUB-4. VB learning achieves higher score in terms of average cluster purity and average speaker purity compared to ML/BIC. 1.

This paper further develops a previously proposed adaptation method for speech recognition called symbolic speaker adaptation (SSA). The basic idea of SSA is to model a speaker's pronunciation as a blend of speech varieties (SVs) - regional dialects and foreign accents - for which the system has existing pronunciation models. The system determines during an adaptation process the relative applicability of those models, yielding a speech variety profile (SVP) for each speaker. Speaker-dependent lexica for recognition are determined from a speaker's SVP. In this paper, we discuss a series of experiments designed to analyze how the SSA method is affected by SV-balanced training, expanded phone inventories, reduced amounts of adaptation data, and speech from SVs not modeled by the system. The most dramatic improvements were obtained by using expanded ("SV-inclusive") phone inventories. SSA was also shown to be effective with a very small number of adaptation sentences. And, SSA's SV blending scheme yields higher accuracy than using a SV classification scheme for speakers of novel (unseen) SVs.

This paper describes a new training approach based on two different techniques (minimum classification error and eigenvoices), in order to achieve a better robustness when only poor training data is provided. In the first two sections of this paper we describe the MCE training and the eigenvoice approach. Then a unified MCE/eigenvoice training algorithm is proposed describing theoretical advantages. We compare the proposed method with classical ML/eigenvoice methods for a speaker identification task. The identification rate improvement is huge for sparse training data (up to 50% in the best case).

In this paper, we explore the potentialities of Variational Bayesian (VB) learning for speech recognition problems. VB methods deal in a more rigorous way with model selection and are a generalization of MAP learning. VB training for Gaussian Mixture Models is less affected than EM-ML training by overfitting and singular solutions. We compare two types of Variational Bayesian Gaussian Mixture Models (VBGMM) with classical EM-ML GMM in a phoneme recognition task on the TIMIT database. VB learning performs better than EM-ML learning and is less affected by the initial model guess.

Setting linear constraints on HMM model space appears to be very effective for speaker adaptation. In doing so, we assume that model parameters are jointly Gaussian. While this approach has proven reasonably successful, we question it accuracy in the case of very high dimensionality parameter spaces. To address this problem, we employ a hierarchical piecewise linear model. Gross speaker variations are modeled with a linear eigenspace, subsuming the joint Gaussian model, and finer residues are modeled using another eigenspace chosen depending on the location of the first values. We perform experiments on Wall Street Journal (WSJ) dictation task, and we observe a cumulative 1.3% WER improvement (11 % relative) when using self-adaptation.

In order to better model pronunciation variations, we present in this paper a method to build a lexicon whose content changes dynamically with the input speech. To achieve this goal, we proceeded in two steps. In the first step, a static augmented lexicon is created by adding new phone transcriptions to a basic lexicon. These new variants are derived from phonetic features that are automatically extracted from some training speech. Then in the second step, phonetic features are extracted again during recognition and help to select entries in the augmented lexicon that best match the phonetic characteristics of a given speech. These selected transcriptions constitute the dynamic lexicon, which is specific to each input utterance. Experiments showed a 16.0% relative reduction in WER compared to the baseline and 16.7% compared to when a static augmented lexicon is used.

A speaker tracking system (STS) is built by using successively a speaker change detector and a speaker verification system. The aim of the STS is to find in a conversation between several persons (some of them having already enrolled and other being totally unknown) target speakers chosen in a set of enrolled users. In a first step, speech is segmented into homogeneous segments containing only one speaker, without any use of a priori knowledge about speakers. Then, the resulting segments are checked to belong to one of the target speakers. The system has been used in a NIST evaluation test with satisfactory results.

Phonetic decision-tree based acoustic modeling has been widely used in speech recognition systems. However, the assumption that all states clustered in the same leaf node share both their Gaussians and mixture weights restricts the improvement of the acoustic models. In this paper, we propose a new structure called a two-level decisiontree. With this structure we can make better use of training data and improve the model accuracy and robustness. Two-level decision trees provide more flexibility to control the number of parameters. By tuning the balance of the first and second level tree nodes, we can get better performance with even fewer parameters than the traditional decision-tree based approach. Experiments on the Wall Street Journal tasks show that our approach can achieve about a 10% word error rate reduction over the conventional approach.

Among the different attempts to improve recognition scores and robustness to noise, the recognition of parallel streams of data each one representing partial information on the test signal and the fusion of the decisions have received a great deal of interest. The problem of training such models taking recombination constraints at the level of speech-subunits has not yet been rigorously addressed. This paper shows how equivalence with an extended meta-HMM solves the problem and how reestimation formulas have to be applied to guarantee equivalence between the multistream model and the meta-HMM. 1. INTRODUCTION Since speech is a non-stationary signal, it is analyzed over small time windows where local stationarity is assumed. A typical width for these windows is 10 ms and different kinds of analysis can be conducted on them. The first standard approach is a harmonic analysis that could be obtained using either filter banks or FFT transforms. After elimination of the fundamental period ...

Indexing of video soundtracks is an important issue for the navigation in multimedia databases. Based on wordspotting techniques, it should meet very constraining specifications; namely fast response to queries, concise processed speech information for limiting the storage memory, speaker independant mode, easy characterization of any word by its phonemic spelling. A solution based on phonemic lattices and on a division of the indexing process into an off-line and an online part is proposed. Previous works based on frame labelling and maximum likelihood criterion are now modified to take into account this new approach based on a maximum a posteriori (MAP) criterion. The REMAP algorithm implements this MAP criterion for training. It has several advantages such as maximizing the global discriminant criterion, avoiding the difficult problem of phoneme transition detection during the training process and being well suited for a hybrid hidden Markov model (HMM) and neural network (NN) approach.

The amount of available video information is dramatically increasing due to the development of multimedia applications. As a consequence, content based retrieval tools are urgently needed for fast and easy access to multimedia database but also to movies and recorded video news. In particular, queries may rely on off-line indexing. Keyword spotting on video soundtracks could be of great help in this indexation process and in the future associated with pattern or event recognition out of the strictly visual information. Specific constraints for this application are identified and a solution based on phonemic lattices is proposed. The word spotter achieves indexing on open vocabularies uttered by any speaker. It is fast enough for practical applications and does not require much additional stored information.

In speech recognition, speech units are usually modeled as hidden Markov processes. The use of Gaussian mixture densities for computing the local emission probabilities has led to a significant performance improvement. Analytical estimators for updating the parameters in the iterative training are well known if the Baum Welch algorithm is used, but, unfortunately, analytical formulas in the Viterbi case cannot be derived by a limiting process from the Baum Welch formulas. Approximate techniques are used with fairly good success. The authors propose analytical estimators in the Viterbi case as a natural extension of similar formulas used for the plain Gaussian density.>

The statistical use of a particular classic form of a connectionist system, the multilayer perceptron (MLP), is described in the context of the recognition of continuous speech. A discriminant hidden Markov model (HMM) is defined, and it is shown how a particular MLP with contextual and extra feedback input units can be considered as a general form of such a Markov model. A link between these discriminant HMMs, trained along the Viterbi algorithm, and any other approach based on least mean square minimization of an error function (LMSE) is established. It is shown theoretically and experimentally that the outputs of the MLP (when trained along the LMSE or the entropy criterion) approximate the probability distribution over output classes conditioned on the input, i.e. the maximum a posteriori probabilities. Results of a series of speech recognition experiments are reported. The possibility of embedding MLP into HMM is described. Relations with other recurrent networks are also explained.>

Recently, connectionist models have been recognized as an interesting alternative tool to hidden Markov models for speech recognition. Their main property lies in their combination of good discriminating power and the ability to capture input-output relations. They have also been proved useful in dealing with statistical data. However, the serial aspect remains difficult to handle in that kind of model, and several authors have proposed original architectures to deal with this problem. This study establishes links among them and compares their respective advantages. Relations with hidden Markov models are explained.>

The Hidden Markov models are generalized by defining a new emission probability which takes the correlation between successive feature vectors into account. Estimation formulas for the iterative learning both along Viterbi and Maximum likelihood criteria are presented.

A connected speech recognition method based on the Baum forward backward algorithm is presented. The segmentation of the test sentence uses the probability that an acoustic vector lays at the separation of two speech subunit models (Hidden Markov models). The labelling rests on the highest probability that a vector has been emitted on the last state of a subunit model. Results are presented for word- and phoneme-recognition.

In this paper, a method for speaker dependent connected speech recognition based on phonemic units is described. In this recognition system, each phoneme is characterized by a very simple 3-state Hidden Markov Model (HMM) which is trained on connected speech by a Viterbi algorithm. Each state has associated with it a continuous (Gaussian) or discrete probability density function (pdf). With the phonemic models so obtained, the recognition is then performed either directly at word level (by the reconstruction of reference words from the models of the constituting phonemes) or via a phonemic labelling. Good results are obtained as well with a German ten digit vocabulary (20 phonemes) as with a French 80 word vocabulary (36 phonemes).

The principles of classification applied to the representation of the words in a vocabulary lead to the clustering of the acoustic vectors into prototype vectors. For a small number of prototypes, recognition scores comparable to those observed with unclustered vocabularies are obtained with a highly reduced computation time. Two different forms (deterministic and stochastic) of the single-level recognition method for concatenated words are described and the improvements obtained by vector quantization are put into evidence. The use of prototypes in the training phase of the finite stochastic automata representing a vocabulary word is also described.

The paper presents a new method for the optimal design of minimum phase FIR filters. First, a constrained approximation procedure is used to obtain the magnitude function and the transmission zeros in the stopband(s). Next, the zeros of the transfer function inside the unit circle are calculated via a low-degree polynomial factorization. For lowpass filters, a straightforward exchange algorithm is presented which achieves the constrained approximation step and which can be implemented via a simple modification of the Parks-McClellan program. The efficacy of the method is illustrated by a numerical example. Attention is drawn on the fact that bandpass filters may in principle require more sophisticated means.