Conversational speech recognition is a challenging problem primarily because speakers rarely fully articulate sounds. A successful speech recognition approach must infer intended spectral targets from the speech data, or develop a method of dealing with large variances in the data. Hidden dynamic models (HDMs) attempt to automatically learn such targets in a hidden feature space using models that integrate linguistic information with constrained temporal trajectory models. HDMs are a radical departure from conventional hidden Markov models (HMMs), which simply account for variation in the observed data. We present an initial evaluation of such models on a conversational speech recognition task involving a subset of the SWITCHBOARD corpus. We show that in an N-best rescoring paradigm, HDMs are capable of delivering performance competitive with HMMs.

This paper introduces a new approach to acoustic-phonetic modelling, the hidden dynamic model (HDM), which explicitly accounts for the coarticulation and transitions between neighbouring phones. Inspired by the fact that speech is really produced by an underlying dynamic system, the HDM consists of a single vector target per phone in a hidden dynamic space in which speech trajectories are produced by a simple dynamic system. The hidden space is mapped to the surface acoustic representation via a non-linear mapping in the form of a multilayer perceptron (MLP). Algorithms are presented for training of all the parameters (target vectors and MLP weights) from segmented and labelled acoustic observations alone, with no special initialisation. The model captures the dynamic structure of speech, and appears to aid a speech recognition task based on the SwitchBoard corpus.

The objective of this work is a computationally efficient method for inferring vocal tract shape trajectories from acoustic speech signals. We use an multilayer perceptron (MLP) to model the vocal tract shape-to-acoustics mapping, then in an analysis-by-synthesis approach, optimise an objective function that includes both the accuracy of the spectrum approximation and the credibility of the vocal tract dynamics. This optimisation carries out gradient descent using backpropagation of derivatives through the MLP. Employing a series of MLPs of increasing order avoids getting trapped in local optima caused by the many-to-one mapping between vocal tract shapes and acoustics. We obtain two orders of magnitude speed increase compared with our previous methods using codebooks and direct optimisation of a synthesiser.

This paper describes a cross-validation method to determine the appropriate weight with which dynamic constraints should be applied when estimating vocal tract shapes from speech. This data-dependent method can estimate the weighting without the need for separate prior knowledge of the source and noise statistics. The principles are first demonstrated on a simple onedimensional system analogous to speech production. As the data here is synthetic, the statistics are known, and so the success of the method can be objectively assessed. Next, the same principles are extended to real speech to improve the estimation of vocal tract shape trajectories. 1 INTRODUCTION For speech coding, recognition, and synthesis, it is believed that an articulatory description may be preferable to conventional acoustic features because physical limitations of the vocal tract imply slowly changing parameters. Also, the close relationship between the articulatory and phonetic domains suggests that an articul...

Compared with the usual acoustic representations, articulatory models offer potential benefits in giving a compact, slowly changing representation, having a closer relationship with the phonetic domain and allowing a straightforward treatment of coarticulation and transitional effects. A long-standing difficulty preventing the realisation of these benefits is the estimation of appropriate parameters from the speech signal. This paper reviews recent progress in this area and describes some initial experiments that attempt to estimate the parameters of the recently proposed Distinctive Regions Model (DRM) [1], using a dynamic programming search of an articulatory codebook. 1 INTRODUCTION An articulatory description 1 may be preferable to an acoustic feature set because physical dynamic limitations of the vocal tract imply slowly changing parameters. Also, the close relationship between the articulatory and phonetic domains suggests that an articulatory parameter set could be a more ...

The authors extend to continuous speech recognition (CSR) the Alphanet approach to integrating backprop networks and HMM (hidden Markov model)-based isolated word recognition. They present the theory of a method for discriminative training of components of a CSR system, using training data in the form of complete sentences. The derivatives of the discriminative score with respect to the parameters are expressed in terms of the posterior probabilities of state occupancies (gammas) under two conditions called 'clamped' and 'free' because they correspond to the two conditions in Boltzmann machine training. The authors compute these clamped and free gammas using the forward-backward algorithm twice, and use the differences to drive the adaptation of a preprocessing data transformation, which can be thought of as replacing the linear transformation which yields MFCCs, or which normalizes a grand covariance matrix.>

Speaker verification is performed by comparing the output probabilities of two Markov models of the same phonetic unit. One of these Markov models is speaker-specific, being built from utterances from the speaker whose identity is to be verified. The second model is built from utterances from a large population of speakers. The performance of the system is improved by treating the pair of models as a connectionist network, an alpha-net, which then allows discriminative training to be carried out. Experimental results show that adapting the spectral observation probabilities of each state of the model by the back propagation of errors can correct misclassification errors. The real-time implementation of the system produced an average digit error rate of 4.5% and only one misclassification in 600 trials using a five-digit sequence.>

A particular form of neural network is described which has terminals for acoustic patterns, class labels, and speaker parameters. A method of training this network to tune in the speaker parameters to a new speaker is outlined. This process can also be viewed from a Bayesian perspective as maximizing the likelihood of the speaker's data by optimizing the model and speaker parameters. A method for doing this when the data are labeled is described. Results of using this technique with whole-word hidden Markov models (HMMs) indicate an improvement over speaker-independent performance and, for unlabeled data, a performance close to that achieved on labeled data.>

Research on continuous-speech recognition using phoneme-level hidden Markov models (HMMs) is described. The aim of the project is automatic recognition of spoken airborne reconnaissance mission (ARM) reports. The evolution of the ARM system from a simple baseline system to its current configuration is described, and a considerable number of experimental results are included. Work on alternative approaches to modeling contextual effects and on improved duration modeling is described.>

A general approach to speaker adaptation in speech recognition is described, in which speaker differences are treated as arising from a parameterized transformation. Given some unlabeled data from a particular speaker, a process is described which maximizes the likelihood of this data by estimating the transformation parameters at the same time as refining estimates of the labels. The technique is illustrated using isolated vowel spectra and phonetically motivated linear spectrum transformations and is shown to give significantly better performance than nonadaptive classification.>

A preliminary theoretical and experimental examination is made of three noise compensation techniques. The three techniques are those due to: Klatt (1976); Bridle et al (1984); and Holmes & Sedgwick (1986). The first two of these techniques have been re-interpreted for use within a hidden Markov model based recogniser. A description is given of how this was done, together with a discussion on some implementation considerations. Experimental results are given for the performance of the algorithms at various signal-to-noise ratios. The principles of recognition in noise are discussed from an implementation point of view and it is shown how the three techniques can be viewed as variations on a single theme.>

Measuring the performance of speech recognition systems is integral to advancing the technology as well as applying it. Appropriately designed controlled experiments are the necessary tools by which both research and commercial systems must assess the value of different algorithms and approaches. Furthermore, they form the foundation upon which potential consumers of this technology may base sound price- performance decisions for commercial applications. The objectives of this presentation are: 1) to review the major performance assessment activities presently underway, especially those of the National Bureau of Standards (U.S. Dept. of Commerce) and the NATO Research Study Group on Speech Recognition, 2) to present a set of recommendations and evaluation guidelines recently compiled from speech researchers based in academia, industry, and government, 3) to reference a number of specific evaluation proposals generated within the speech community, and 4) to summarize and briefly describe a compilation of speech data bases which under certain conditions have been or may become available.

In automatic speech recognition (ASR) using whole-word templates, dynamic programming (DP) is frequently used to determine the similarity of two patterns (derived from spoken words) using the optimal way of aligning their timescales. In ASR the actual timescale alignment is of secondary interest to the degree of similarity and is not normally computed. We present ZIP, a modified DP algorithm designed to compute the time alignment of two utterances of the same text of any length. By using a window and partial traceback the amount of computation and storage is kept to a modest level, although the optimality of the final path is no longer absolutely guaranteed. Uses of ZIP are given.

The principles of an efficient one-pass dynamic programming whole-word pattern matching algorithm for the recognition of spoken sequences of connected words are described. Particular attention is given to the technique for keeping track of word-sequence decisions, which may be constrained by a finite-state syntax. Some extensions of the technique are discussed.

This paper is concerned with inverse filtering of the speech waveform in order to study the details of vocal tract excitation. An interactive digital inverse filtering system is described in which the advantages of previously described analogue and digital methods are combined to provide a facility with much greater convenience and power than either. Experience with this system is compared with that obtained using closed-period covariance-method linear prediction analysis. Evidence is presented to support the contention that since there is continued activity in the closed-glottis period there are definite advantages in having an interactive capability.