This paper proposes a two-stage deep feed-forward neural network (DNN) to tackle the acoustic-to-articulatory inversion (AAI) problem. DNNs are a viable solution for the AAI task, but the temporal continuity of the estimated articulatory values has not been exploited properly when a DNN is employed. In this work, we propose to address the lack of any temporal constraints while enforcing a parameter-parsimonious solution by deploying a two-stage solution based only on DNNs: (i) Articulatory trajectories are estimated in a first stage using DNN, and (ii) a temporal window of the estimated trajectories is used in a follow-up DNN stage as a refinement. The first stage estimation could be thought of as an auxiliary additional information that poses some constraints on the inversion process. Experimental evidence demonstrates an average error reduction of 7.51% in terms of RMSE compared to the baseline, and an improvement of 2.39% with respect to Pearson correlation is also attained. Finally, we should point out that AAI is still a highly challenging problem, mainly due to the non-linearity of the acoustic-to-articulatory and one-to-many mapping. It is thus promising that a significant improvement was attained with our simple yet elegant solution.

This paper presents methods and results for optimizing subword detectors in continuous speech. Speech detectors are useful within areas like detection-based ASR, pronunciation training, phonetic analysis, word spotting, etc. We propose a new discriminative training criterion for subword unit detectors that is based on the Minimum Phone Error framework. The criterion can optimize the F-score or any other detection performance metric. The method is applied to the optimization of HMMs and MFCC filterbanks in phone detectors. The resulting filterbanks differ from each other and reflect acoustic properties of the corresponding detection classes. For the experiments in TIMIT, the best optimized detectors had a relative accuracy improvement of 31.3% over baseline and 18.2% over our previous MCE-based method.

Hidden Markov models (HMMs) are powerful generative models for sequential data that have been used in automatic speech recognition for more than two decades. Despite their popularity, HMMs make inaccurate assumptions about speech signals, thereby limiting the achievable performance of the conventional speech recognizer. Penalized logistic regression (PLR) is a well-founded discriminative classifier with long roots in the history of statistics. Its classification performance is often compared with that of the popular support vector machine (SVM). However, for speech classification, only limited success with PLR has been reported, partially due to the difficulty with sequential data. In this paper, we present an elegant way of incorporating HMMs in the PLR framework. This leads to a powerful discriminative classifier that naturally handles sequential data. In this approach, speech classification is done using affine combinations of HMM log-likelihoods. We believe that such combinations of HMMs lead to a more accurate classifier than the conventional HMM-based classifier. Unlike similar approaches, we jointly estimate the HMM parameters and the PLR parameters using a single training criterion. The extension to continuous speech recognition is done via rescoring of N-best lists or lattices.

Many feature enhancement methods make use of probabilistic models of speech and noise in order to improve performance of speech recognizers in the presence of background noise. The traditional approach for training such models is maximum likelihood estimation. This paper investigates the novel application of variational Bayesian learning for front-end models under the Algonquin denoising framework. Compared to maximum likelihood training, it is shown that variational Bayesian learning has advantages both in terms of increased robustness with respect to choice of model complexity, as well as increased performance.

In this paper we present an approach that makes use of both Bayesian predictive classification (BPC) and parallel model combination (PMC) to achieve increased robustness towards noise. PMC provides a method for finding parameter estimates for speech corrupted by noise, while BPC is a method that com-pensates for uncertainty of parameter estimates. Thus, these methods can be combined in order to obtain knowledge about the mismatch situation and simultaneously account for uncer-tainty in this knowledge. We apply this technique in an unsu-pervised approach on the Aurora2 database and show that good performance is obtained. 1.

The work presented is centered around a speech production model called the chained dynamical system model (CDSM) which is motivated by the fundamental limitations of the mainstream ASR approaches. The CDSM is essentially a smoothly time varying continuous state nonlinear dynamical system, consisting of two sub dynamical systems coupled as a chain so that one system controls the parameters of the next system. The speech recognition problem is posed as inverting the CDSM, for which we propose a solution based on the theory of embedding. The resulting architecture, which we call inverted CDSM (ICDSM) is evaluated in a set of experiments involving a speaker independent, continuous speech recognition task on the TIMIT database. Results of these experiments which can be compared with the corresponding results in the literature, confirm the feasibility and advantages of the approach.

A system for on-line generation of closed captions (subtitles) for broadcast of live TV-programs is described. During broadcast, a commentator formulates a possibly condensed, but semantically correct version of the original speech. These compressed phrases are recognized by a continuous speech recognizer, and the resulting captions are fed into the teletext system. This application will provide the hearing impaired with an option to read captions for live broadcast programs, i.e., when off-line captioning is not feasible. The main advantage in using a speech recognizer rather than a stenography-based system (e.g., Velotype) is the relaxed requirements for commentator training. Also, the amount of text generated by a system based on stenography tends to be large, thus making it harder to read. 1.

Estimating the nonlinear dynamical system which has generated a given time series is considered. Traditionally, this is achieved via two separate steps: projecting the signal frames onto the state space, and then estimating the predictive relationship between successive state vectors. In the method presented here, these two steps are performed simultaneously, where both the projection and prediction functions are represented by MLPs. In addition, a measurement function, again represented by an MLP, is used. This MLP maps each state vector to a sample of the given signal. Original estimation problem is then interpreted as a training (optimization) problem, and a backpropagation type training algorithm is proposed for solving it. The method proposed is tested with an artificial time series produced by Henon map and a speech signal. Performances are compared with those of popular traditional methods, which indicates the clear superiority of this method.

This paper deals with speaker-independent continuous speech recognition. Our approach is based on continuous density hidden Markov models with a non-linear input feature transformation performed by a multilayer perceptron. We discuss various optimisation criteria and provide results on a TIMIT phoneme recognition task, using single frame (mutual information or relative entropy) MMI embedded in Viterbi training, and a global MMI criterion. As expected, global MMI is found superior to the frame-based criterion for continuous recognition. We further observe that optimal sentence decoding is essential to achieve maximum recognition rate for models trained by global MMI. Finally, we find that the simple MLP input transformation, with five frames of context information, can increase the recognition rate significantly compared to just using delta parameters.>

An isolated-word speech recognizer based on acoustically defined subwords is described. In the training stage the speech is decomposed by an automatic two-pass segmentation algorithm. In the first pass the subword boundaries are estimated. The second pass divides each subword into three subsegments, thus matching the states in the three-state HMMs used to model the subwords. On the basis of both the segmental and subsegmental information, the acoustic lexicon and the HMMs are created. Used in a speaker-independent mode the recognizer is slightly inferior to a corresponding whole-word-based recognizer.>