This paper describes the application of deep neural networks (DNNs), trained to discriminate among speakers, to improving performance in text-independent speaker verification. Activations from the bottleneck layer of these DNNs are used as features in an i-vector based speaker verification system. The features derived from this network are thought to be more robust with respect to phonetic variability, which is generally considered to have a negative impact on speaker verification performance. The verification performance using these features is evaluated on the 2012 NIST SRE core-core condition with models trained from a subset of the Fisher and Switchboard conversational speech corpora. It is found that improved performance, as measured by the minimum detection cost function (minDCF), can be obtained by appending speaker discriminative features to the more widely used mel-frequency cepstrum coefficients.

A group of junior and senior researchers gathered as a part of the 2014 Frederick Jelinek Memorial Workshop in Prague to address the problem of predicting the accuracy of a nonlinear Deep Neural Network probability estimator for unknown data in a different application domain from the domain in which the estimator was trained. The paper describes the problem and summarizes approaches that were taken by the group1.

Multi-task learning (MTL) for deep neural network (DNN) multilingual acoustic models has been shown to be effective for learning parameters that are common or shared between multiple languages[1, 2]. In the MTL paradigm, the number of parameters in the output layer is large and scales with the number of languages used in training. This output layer becomes a computational bottleneck. For mono-lingual DNNs, low-rank matrix factorization (LRMF) of weight matrices have yielded large computational savings[3, 4]. The LRMF proposed in this work for MTL, is for the original languagespecific block matrices to “share” a common matrix, with resulting low-rank language specific block matrices. The impact of LRMF is presented in two scenarios, namely : (a) improving performance in a target language when auxiliary languages are included during multi-lingual training; and (b) cross-language transfer to an unseen language with only 1 hour of transcribed training data. A 44% parameter reduction in the final layer, manifests itself in providing a lower memory footprint and faster training times. An experimental study shows that the LRMF multi-lingual DNN provides competitive performance compared to a full-rank multi-lingual DNN in both scenarios.

The use of a graph embedding framework is investigated as a regularization technique in the expectation-maximization (EM) algorithm applied to automatic speech recognition (ASR). The technique is motivated by the fact that graph em-beddings of feature vectors have been shown to provide useful characterizations of the underlying manifolds on which these features lie. Incorporating intrinsic graphs that describe these manifolds in the optimization criteria for the EM algorithm has the effect of constraining the solution space in a way that preserves the local structure of the data. Graph embedding based regularization is applied here to estimating parameters in constrained maximum likelihood linear regression (CMLLR) speaker adaptation in continuous density hidden Markov model (CDHMM) based ASR. CMLLR adaptation has been widely used as a maximum likelihood procedure for reducing mismatch between a given HMM model and utterances from an unknown speaker through a linear feature space transformation. However, there is no guarantee that CMLLR transformations will preserve the relationships of the feature vectors along this manifold. It is argued here that graph embedding based regularization will preserve this structure. The impact of this approach on ASR performance is evaluated for unsupervised speaker adaptation on two large vocabulary speech corpora.

A two-stage speaker adaptation approach is proposed for the subspace Gaussian mixture model (SGMM) [1] in large vocabulary automatic speech recognition (ASR). The SGMM differs from the more well known continuous density hidden Markov model (CDHMM) in that a large portion of the SGMM parameters are dedicated to shared full covariance Gaussian subspace parameters and a relatively small number of parameters are used for state dependent projection vectors. Both model space and feature space adaptation are investigated. First, an efficient regression based approach for subspace vector adaptation (SVA) is presented. Second, an efficient approach is presented for feature space adaptation using constrained maximum likelihood linear regression (CMLLR) in the SGMM. While both of these adaptation scenarios have previously been investigated in the context of the SGMM [2, 3], a more efficient and numerically stable procedure is presented here for estimating the parameters of the regression based transformations. Both transformation matrices are obtained using an optimization technique that iteratively updates the rows of the regression matrices. It is shown that using these feature space and model space approaches for unsupervised speaker adaptation provides complementary improvements in SGMM based ASR word accuracy.

A method for speaker normalization in deep neural network (DNN) based discriminative feature estimation for automatic speech recognition (ASR) is presented. This method is applied in the context of a DNN configured for auto-encoder based low dimensional bottleneck (AE-BN) feature extraction where the derived features are used as input to a continuous Gaussian density hidden Markov model (HMM/GMM) based ASR decoder. While AE-BN features are known to provide significant reduction in ASR word error rate (WER) with respect to more conventional spectral magnitude based features, there is no general agreement on how these networks can reduce the impact of speaker variability by incorporating prior knowledge of the speaker. An approach is presented in this paper where spectrum based DNN inputs are augmented with speaker inputs that are derived from separate regression based speaker transformations. It is shown the proposed method could reduce the WER by 3% relative to the best speaker adapted AE-BN CDHMM system.

Deep neural networks (DNNs) have been successfully applied to a variety of automatic speech recognition (ASR) tasks, both in discriminative feature extraction and hybrid acoustic modeling scenarios. The development of improved loss functions and regularization approaches have resulted in consistent reductions in ASR word error rates (WERs). This paper presents a manifold learning based regularization framework for DNN training. The associated techniques attempt to preserve the underlying low dimensional manifold based relationships amongst speech feature vectors as part of the optimization procedure for estimating network parameters. This is achieved by imposing manifold based locality preserving constraints on the outputs of the network. The techniques are presented in the context of a bottleneck DNN architecture for feature extraction in a tandem configuration. The ASR WER obtained using these networks is evaluated on a speech-in-noise task and compared to that obtained using DNN-bottleneck networks trained without manifold constraints. Index Terms: manifold learning, deep neural networks, speech recognition, tandem feature extraction

This paper presents a study of two acoustic speaker adaptation techniques applied in the context of the subspace Gaussian mixture model (SGMM) for automatic speech recognition (ASR). First, a model space linear regression based approach is presented for adaptation of SGMM state projection vectors and is referred to as subspace vector adaptation (SVA). Second, an easy to implement realization of constrained maximum likelihood linear regression (CMLLR) is presented for feature space adaptation in the SGMM. Numerically stable procedures for row-by-row estimation of the regression based transformation matrices are presented for both SVA and CMLLR adaptation. These approaches are applied to SGMM models that are estimated using speaker adaptive training (SAT), a technique for estimating more compact speaker independent acoustic models. Unsupervised speaker adaptation performance is evaluated on conversational and read speech task domains and compared to unsupervised adaptation performance obtained using the hidden Markov model-Gaussian mixture model (HMM-GMM) in ASR. It is shown that the feature space and model space adaptation approaches applied to the SGMM provide complementary reductions in word error rate (WER) and provide lower WERs than that obtained using CMLLR adaptation for the HMM-GMM.

This paper presents a family of discriminative manifold learning approaches to feature space dimensionality reduction in noise robust automatic speech recognition (ASR). The specific goal of these techniques is to preserve local manifold structure in feature space while at the same time maximizing the separability between classes of feature vectors. In the manifold space, the relationships among the feature vectors are defined using nonlinear kernels. Two separate distance measures are used to characterize the kernels, namely the conventional Euclidean distance and a cosine-correlation based distance. The performance of the proposed techniques is evaluated on two task domains involving noise corrupted utterances of connected digits and read newspaper text. Performance is compared to existing approaches used for feature space transformations, including linear discriminant analysis (LDA) and locality preserving linear projections (LPP). The proposed approaches are found to provide a significant reduction in word error rate (WER) with respect to the more well-known techniques for a variety of noise conditions. Another contribution of the paper is to quantify the interaction between acoustic noise conditions and the shape and size of local neighborhoods which are used in manifold learning to define local relationships among feature vectors. Based on this analysis, a procedure for reducing the impact of varying acoustic conditions on manifold learning is proposed .

This paper is concerned with the problem of building acoustic models for automatic speech recognition (ASR) using speech data from multiple languages. Techniques for multi-lingual ASR are developed in the context of the subspace Gaussian mixture model (SGMM)[2, 3]. Multi-lingual SGMM based ASR systems have been configured with shared subspace parameters trained from multiple languages but with distinct language dependent phonetic contexts and states[11, 12]. First, an approach for sharing state-level target language and foreign language SGMM parameters is described. Second, semi-tied covariance transformations are applied as an alternative to full-covariance Gaussians to make acoustic model training less sensitive to issues of insufficient training data. These techniques are applied to Hindi and Marathi language data obtained for an agricultural commodities dialog task in multiple Indian languages.

An approach is proposed for adapting subspace projection vectors in the subspace Gaussian mixturemodel (SGMM) [1]. Subword models in the SGMM are composed of states, each of which are parametrized using a small number of subspace projection vectors. It is shown here that these projection vectors provide a compact and well-behaved characterization of phonetic information in speech. A regression based subspace vector adaptation approach is proposed for adapting these parameters. The performance of this approach is evaluated for unsupervised speaker adaptation on two large vocabulary speech corpora.

We summarize the accomplishments of a multi-disciplinary workshop exploring the computational and scientific issues surrounding zero resource (unsupervised) speech technologies and related models of early language acquisition. Centered around the tasks of phonetic and lexical discovery, we consider unified evaluation metrics, present two new approaches for improving speaker independence in the absence of supervision, and evaluate the application of Bayesian word segmentation algorithms to automatic subword unit tokenizations. Finally, we present two strategies for integrating zero resource techniques into supervised settings, demonstrating the potential of unsupervised methods to improve mainstream technologies.

In this paper the use of acoustic similarity of speech intervals for generating improved confidence scores for spoken term detection (STD) is investigated. A procedure based on acoustic dotplots which requires no training data is deployed for discovering similar speech intervals. A graph based random walk algorithm incorporates acoustic similarity of hypothesized term occurrences for improving the corresponding confidence scores. The proposed approach is evaluated in an open vocabulary STD task defined on a lecture domain corpus. It is shown that updating the confidence scores in this fashion results in a significant increase in term detection performance of out of vocabulary search terms. A relative improvement of 12.9% in figure of merit was gained relative to that obtained from a baseline lattice based STD system.

This paper considers the application of a random projections based hashing scheme, known as locality sensitive hashing (LSH), for fast computation of neighborhood graphs in manifold learning based feature space transformations in automatic speech recognition (ASR). Discriminative manifold learning based feature transformations have already been found to provide significant improvements in ASR performance. The motivation of this work is the fact that the high computational complexity of these techniques has prevented their application to very large speech corpora. The performance of this integrated system is evaluated both in terms of computational complexity and ASR word recognition accuracy. Further comparisons of ASR performance with the well-known linear discriminant analysis are provided. These results demonstrate that LSH provides the much needed speed boost to manifold learning techniques with minimal impact on their ASR performance, thus enabling the application of these algorithms to large speech databases.

This paper considers the application of discriminative manifold learning approaches in feature analysis for automatic speech recognition (ASR). The issue of manifold learning is addressed for feature space dimensionality reduction in domains involving noise corrupted speech. The locality preserving discriminant analysis (LPDA) approach to manifold learning is investigated. This class of techniques exploits the assumption that there is a structural relationship among data vectors which can be maintained by preserving the local relationships among the transformed data vectors. The paper presents a procedure for reducing the impact of varying acoustic conditions on manifold learning. Noise aware manifold learning (NAML) is described as an approach for exploiting estimated background characteristics to define the size of the local neighborhoods used for LPDA feature space transformations. It is shown that NAML significantly reduces the speech recognition WER in a noisy speech recognition task over LPDA, particularly at low signal-to-noise ratios.

In this paper, the application of semi-supervised manifold learning techniques to the task of verifying hypothesized occurrences of spoken terms is investigated. These techniques are applied in a two stage spoken term detection framework where ASR lattices are first generated using a large vocabulary ASR system and hypothesized occurrences of spoken query terms in the lattices are verified in a second stage. The verification process is performed using a fixed dimensional feature representation derived from each hypothesized term occurrence. Two semi-supervised approaches namely, manifold regularized least squares (RLS) classification and spectral clustering, are investigated for distinguishing correct hypotheses from false alarms. It is shown that, exploiting unlabeled data in addition to labeled data using semi-supervised approaches, significantly improves the verification performance compared to the case where only the labeled data is used. This improvement in performance increases as the ratio of unlabeled to labeled data augments. It is also shown that, when training data is very limited, a comparable verification performance can be gained by exploiting only the acoustic similarity between the test samples using the spectral clustering approach.

Manifold learning based techniques have been found to be useful for feature space transformations and semi-supervised learning in speech processing. However, the immense computational requirements in building neighborhood graphs have hindered the application of these techniques to large speech corpora. This paper presents an approach for fast computation of neighborhood graphs in the context of manifold learning. The approach, known as locality sensitive hashing (LSH), has been applied to a discriminative manifold learning based feature space transformation technique that utilizes a cosine-correlation based distance measure. Performance is evaluated first in terms computational savings at a given level of ASR performance. The results demonstrate that LSH provides a factor of 9 reduction in the computational complexity with minimal impact on speech recognition performance. A study is also performed comparing the efficiency of the LSH algorithm presented here and other LSH approaches in identifying nearest neighbors.

The subspace Gaussian mixture model (SGMM) has been recently proposed as an acoustic modeling technique suitable for configuring multilingual speech recognition systems. It is attractive for this purpose since its parametrization allows its “shared” model parameters to be trained with data from multiple languages [1]. In this work, we report on the results of an experimental study carried out with the goal of improving native Spanish language speech recognition performance using an existing telephone speech corpus of English spoken by speakers of Spanish origin. Compensation for sources of acoustic variability between Spanish and English language data sets was found to be important in obtaining good multilingual ASR performance. We conclude with a discussion about the notion of acoustic similarity between the state dependent parameters of the SGMM, and its possible use in effectively modelling pronunciation variation.

This paper presents an efficient approach to spoken term detection (STD) from unstructured audio recordings using word lattices generated off-line from an automatic speech recognition (ASR) system. The approach facilitates open vocabulary STD and focuses specifically on reducing the difference between detection performance obtained for within-vocabulary (IV) and out-of-vocabulary (OOV) search terms. Improved OOV detection performance is obtained by using a two-pass search procedure. Candidate audio segments are retrieved from an index of word lattice paths in the first pass. Locations of OOV search terms are detected in the second pass from a constrained alignment of phonemic expansions of the query terms with phoneme sequences obtained from acoustic segments using an unconstrained neural network based phone decoder. It is found that the combination of first pass segment retrieval and second pass term verification significantly increases STD performance for OOV query terms with no increase in search time for utterances taken from a lecture speech domain.

This paper is concerned with combining models for decoding an optimum translation for a dictation based machine aided human translation (MAHT) task. Statistical language model (SLM) probabilities in automatic speech recognition (ASR) are updated using statistical machine translation (SMT) model probabilities. The effect of this procedure is evaluated for utterances from human translators dictating translations of source language documents. It is shown that computational complexity is significantly reduced while at the same time word error rate is reduced by 30%.

State-of-the-art spoken term detection (STD) systems are built on top of large vocabulary speech recognition engines, which generate lattices that encode candidate occurrences of each invocabulary query. These lattices specifiy start and stop times of hypothesized term occurrences, providing a clear opportunity to return to the acoustics to incorporate novel confidence measures for verification. In this paper, we introduce a novel exemplar distance metric to the recently proposed discriminative point process modeling (DPPM) framework and use the resulting whole word models to generate STD confidence scores. In doing so, we introduce STD to a completely distinct acoustic modeling pipeline, trading Gaussian mixture models (GMM) for multi-layer perceptrons and replacing dictionary-derived hidden Markov models (HMM) with exemplar-based point process models. We find that whole word DPPM scores both perform comparably and are complementary to lattice posterior scores produced by a state-of-the-art speech recognition engine. Index Terms: spoken term detection, point process model, discriminative training, whole word model

A nonlinear discriminant analysis based approach to feature space dimensionality reduction in noise robust automatic speech recognition (ASR) is proposed. It utilizes a correlation based distance measure instead of the conventional Euclidean distance. The use of this ‘correlation preserving discriminant analysis’ (CPDA) procedure is motivated by evidence suggesting that correlation based cepstrum distance measures can be more robust than Euclidean based distances when speech is corrupted by noise. The performance of CPDA is evaluated in terms of the word error rate obtained by using CPDA derived features on a speech in noise task, and is compared to a number of Euclidean distance based approaches to feature space transformations, namely linear discriminant analysis (LDA), locality preserving projections (LPP), and locality preserving discriminant analysis (LPDA).

Data imputation approaches for robust automatic speech recognition reconstruct noise corrupted spectral information by exploiting prior knowledge of the relationship between tar get speech and background characterized by spectrographic masks. Most of these approaches operate without considering the temporal or spectral trajectories of the spectral components. Discrete wavelet transform (DWT) based filter banks are investigated here for spectrogram reconstruction to address the well known importance of preserving spectro temporal modulation characteristics in the speech spectrum. A novel approach is presented for propagating prior spectro graphic mask probabilities to serve as oracle information for thresholding coefficients in a wavelet de-noising scenario. The results of an experimental study are presented to demonstrate the performance of DWT based data imputation relative to a well known MMSE based approach on the Aurora 2 noisy speech recognition task.

This paper investigates the impact of sub space based techniques for acoustic modeling in automatic speech recognition (ASR). There are many well known approaches to subspace based speaker adaptation which represent sources of variability as a projection within a low dimensional subspace. A new approach to acoustic modeling in ASR, referred to as the subspace based Gaussian mixture model (SGMM), represents phonetic variability as a set of projections applied at the state level in a hidden Markov model (HMM) based acoustic model. The impact of the SGMM in modeling these intrinsic sources of variability is evaluated for a continuous speech recognition (CSR) task. The SGMM is shown to provide an 18% reduction in word error rate (WER) for speaker independent (SI) ASR relative to the continuous density HMM (CDHMM) in the resource management CSR domain. The SI performance obtained from SGMM also represents a 5% reduction in WER relative to subspace based speaker adaption in an unsupervised speaker adaptation scenario.

Although research has previously been done on multilingual speech recognition, it has been found to be very difficult to improve over separately trained systems. The usual approach has been to use some kind of “universal phone set” that covers multiple languages. We report experiments on a different approach to multilingual speech recognition, in which the phone sets are entirely distinct but the model has parameters not tied to specific states that are shared across languages. We use a model called a “Subspace Gaussian Mixture Model” where states' distributions are Gaussian Mixture Models with a common structure, constrained to lie in a subspace of the total parameter space. The parameters that define this subspace can be shared across languages. We obtain substantial WER improvements with this approach, especially with very small amounts of in-language training data.

In this paper we present a novel approach for estimating feature-space maximum likelihood linear regression (fMLLR) transforms for full-covariance Gaussian models by directly maximizing the likelihood function by repeated line search in the direction of the gradient. We do this in a pre-transformed parameter space such that an approximation to the expected Hessian is proportional to the unit matrix. The proposed algorithm is as efficient or more efficient than standard approaches, and is more flexible because it can naturally be combined with sets of basis transforms and with full covariance and subspace precision and mean (SPAM) models.

Preparation of a lexicon for speech recognition systems can be a significant effort in languages where the written form is not exactly phonetic. On the other hand, in languages where the written form is quite phonetic, some common words are often mispronounced. In this paper, we use a combination of lexicon learning techniques to explore whether a lexicon can be learned when only a small lexicon is available for boot-strapping. We discover that for a phonetic language such as Spanish, it is possible to do that better than what is possible from generic rules or hand-crafted pronunciations. For a more complex language such as English, we find that it is still possible but with some loss of accuracy.

We describe an acoustic modeling approach in which all phonetic states share a common Gaussian Mixture Model structure, and the means and mixture weights vary in a subspace of the total parameter space. We call this a Subspace Gaussian Mixture Model (SGMM). Globally shared parameters define the subspace. This style of acoustic model allows for a much more compact representation and gives better results than a conventional modeling approach, particularly with smaller amounts of training data.

This paper investigates spoken term detection (STD) from audio recordings of course lectures obtained from an existing media repository. STD is performed from word lattices generated offline using an automatic speech recognition (ASR) system configured from a meetings domain. An efficient STD approach is presented where lattice paths which are likely to contain search terms are identified and an efficient phone based distance is used to detect the occurrence of search terms in phonetic expansions of promising lattice paths. STD and ASR results are reported for both in-vocabulary (IV) and out-of-vocabulary (OOV) search terms in this lecture speech domain.

This paper describes a set of techniques for improving the performance of automated voice search services intended for mobile users accessing these services over a range of portable devices. Voice search is implemented as a two stage search procedure where string candidates generated by an automatic speech recognition (ASR) system are re-scored in order to identify the best matching entry from a potentially very large application specific database. The work in this paper deals specifically with user utterances that contain spoken letter sequences corresponding to spelled instances of search terms. Methods are investigated for identifying the most likely database entry associated with the decoded utterance. An experimental study is presented describing the characteristics of actual user utterances obtained from a prototype voice search service. The impact of these methods on word error rate is presented.

This paper investigates indexing strategies for open vocabulary spoken term detection (STD) in a lecture speech domain. STD is performed from word lattices generated offline using an automatic speech recognition (ASR) system configured from a meetings task domain. Indexing of lattice paths is performed to avoid exhaustive search of audio segments which can be impractical for extremely large media repositories. The method is based on constructing a word-based index from these lattices and using an approximate subword-based algorithm for accessing index entries from subword expansions of query terms. Results are presented for an experimental study demonstrating both STD performance and the potential for scaling the indexing strategy to very large collections of audio segments.

This paper presents a model for machine-aided human translation (MAHT) that integrates source language text and target language acoustic information to produce the text translation of source language document. It is evaluated on a scenario where a human translator dictates a first draft target language translation of a source language document. Information obtained from the source language document, including translation probabilities derived from statistical machine translation (SMT) and named entity tags derived from named entity recognition (NER), is incorporated with acoustic phonetic information obtained from an automatic speech recognition (ASR) system. One advantage of the system combination used here is that words that are not included in the ASR vocabulary can be correctly decoded by the combined system. The MAHT model and system implementation is presented. It is shown that a relative decrease in word error rate of 29% can be obtained by this combined system relative to the baseline ASR performance on a French to English document translation task in the Hansard domain. In addition, it is shown that transcriptions obtained by using the combined system show a relative increase in NIST score of 34% compared to transcriptions obtained from the baseline ASR system.

Techniques are presented for detecting phoneme level mispronunciations in utterances obtained from a population of impaired children speakers. The intended application of these approaches is to use the resulting confidence measures to provide feedback to patients concerning the quality of pronunciations in utterances arising within interactive speech therapy sessions. The pronunciation verification scenario involves presenting utterances of known words to a phonetic decoder and generating confusion networks from the resulting phone lattices. Confidence measures are derived from the posterior probabilities obtained from the confusion networks. Phoneme level mispronunciation detection performance was significantly improved with respect to a baseline system by optimizing acoustic models and pronunciation models in the phonetic decoder and applying a nonlinear mapping to the confusion network posteriors.

This paper presents an efficient method for combining models of speech and noise for robust speech recognition applications in noisy environments. This method decomposes the cepstrum domain representation of noise-corrupted speech into clean speech cepstrum and background noise cepstrum components using a minimum mean squared error-log spectral amplitude (MMSE-LSA) criterion. Speech recognition is then performed on noisy cepstrum domain observations using a model that is formed by parallel combination of cepstrum domain clean speech distributions and background noise distributions estimated using this MMSE-LSA based noise decomposition. This method is far more efficient than other parallel model combination (PMC) procedures because model combination is performed directly in the cepstrum domain rather than in the linear spectral domain. Whereas background noise model estimation is addressed as a separate issue in existing PMC procedures, this method explicitly incorporates a mechanism to continually update background noise models and signal-to-noise ratio (SNR) estimates over time. The performance of the proposed cepstrum-domain model combination method is compared with a well known implementation of PMC which uses a log-normal approximation when combining speech and background noise model means and variances on a connected digit string recognition task which is subjected to mismatched channel and environment conditions. As a result, it is shown that the proposed model combination technique gives a word error rate that is comparable to PMC when background noise information and SNR are known prior to estimation. The paper will also present the results of experiments where a combination of cepstrum-domain feature compensation and model combination are applied to this task.

This paper presents a new approach to feature analysis in automatic speech recognition (ASR) based on locality preserving projections (LPP). LPP is a manifold based dimensionality reduction algorithm which can be trained and applied as a linear projection to ASR features. Conventional manifold based dimensionality reduction algorithms are generally restricted to batch mode implementation and it is difficult in practice to apply them to unseen data. It is argued that LPP can model feature vectors that are assumed to lie on a nonlinear embedding subspace by preserving local relations among input features, so it has a potential advantage over conventional linear dimensionality reduction algorithms like principal components analysis (PCA) and linear discriminant analysis (LDA). Experimental results obtained on the Resource Management (RM) data set showed that when LPP based dimensionality reduction was applied in the context of mel frequency cepstrum coefficient (MFCC) based feature analysis, a significant reduction of word error rate (WER) was obtained with respect to standard MFCC features.

This paper deals with the interaction between progressive model adaptation and score normalization strategies which are used for reducing the variation in likelihood ratio scores in making speaker verification decisions. This issue is important in establishing robust decision thresholds for practical speaker verification systems. An adaptive score normalization method is proposed that is designed to reduce the drift in likelihood ratio scores that occurs when speaker models are adapted. This method is investigated and compared with other more well know score normalization methods in the context of a joint factor analysis speaker verification approach. All approaches are evaluated on the progressive adaptation track in the NIST 2005 text independent speaker verification evaluation plan.

This paper presents an approach for integrating statistical ma-chine translation and automatic speech recognition for machine aided human translation (MAHT). It is applied to the problem of improving ASR performance for a human translator dictating translations in a target language while reading from a source language document. The approach addresses the issues asso-ciated with task independent ASR including out of vocabulary words and mismatched language models. We show in this paper that by obtaining domain information from the document in the form of labelled named entities from the source language text the accuracy of the ASR system can be improved by 34.5%. 1.

This paper presents a study of confidence measure based techniques for detecting phoneme level mispronunciations in utterances from impaired children with neuromuscular disorders. Several different adaptation scenarios are investigated to determine the effects of mismatched speaker characteristics and mismatched task domain on the ability to verify the phoneme level pronunciations. These techniques are evaluated in the context of a speech corpus where utterances were elicited from children in interactive speech therapy sessions involving a multimodal game-like environment. Results are presented in terms of phone detection charateristics where, for example, equal error rates of as low as 16.2 % were obtained for detecting instances where phonemes were deleted by impaired speakers. 1.

This letter addresses the issue of frequency warping-based speaker normalization in noisy acoustic environments. Techniques are developed for improving the robustness of localized estimates of frequency warping transformations that are applied to individual observation vectors. It is shown that automatic speech recognition (ASR) performance can be improved by using speaker class-dependent distributions characterizing frequency warping transformations associated with individual hidden Markov model states. The effect of these techniques is demonstrated over a range of noise conditions on the Aurora 2 speech corpus.

The new model reduces the impact of local spectral and temporal variability by estimating a finite set of spectral and temporal warping factors which are applied to speech at the frame level. Optimum warping factors are obtained while decoding in a locally constrained search. The model involves augmenting the states of a standard hidden Markov model (HMM), providing an additional degree of freedom. It is argued in this paper that this represents an efficient and effective method for compensating local variability in speech which may have potential application to a broader array of speech transformations. The technique is presented in the context of existing methods for frequency warping-based speaker normalization for ASR. The new model is evaluated in clean and noisy task domains using subsets of the Aurora 2, the Spanish Speech-Dat-Car, and the TIDIGITS corpora. In addition, some experiments are performed on a Spanish language corpus collected from a population of speakers with a range of speech disorders. It has been found that, under clean or not severely degraded conditions, the new model provides improvements over the standard HMM baseline. It is argued that the framework of local warping is an effective general approach to providing more flexible models of speaker variability.

Speaker space-based adaptation methods for automatic speech recognition have been shown to provide significant performance improvements for tasks where only a few seconds of adaptation speech is available. However, these techniques are not widely used in practical applications because they require large amounts of speaker-dependent training data and large amounts of computer memory. The authors propose a robust, low-complexity technique within this general class that has been shown to reduce word error rate, reduce the large storage requirements associated with speaker space approaches, and eliminate the need for large numbers of utterances per speaker in training. The technique is based on representing speakers as a linear combination of clustered linear basis vectors and a procedure is presented for maximum-likelihood estimation of these vectors from training data. Significant word error rate reduction was obtained using these methods relative to speaker independent performance for the Resource Management and Wall Street Journal task domains.

This paper is concerned with the problem of machine aided human language translation. It addresses a translation scenario where a human translator dictates the spoken language translation of a source language text into an automatic speech dictation system. The source language text in this scenario is also presented to a statistical machine translation system (SMT). The techniques presented in the paper assume that the optimum target language word string which is produced by the dictation system is modeled using the combined SMT and ASR statistical models. These techniques were evaluated on a speech corpus involving human translators dictating English language translations of French language text obtained from transcriptions of the proceedings of the Canadian House of Commons. It will be shown in the paper that the combined ASR/SMT modeling techniques described in the paper were able to reduce ASR WER by 26.6 percent relative to the WER of an ASR system that did not incorporate SMT knowledge. 1.

Speaker space based adaptation methods for automatic speech recognition have been shown to provide signicant performance improvements for tasks where only a few seconds of adaptation speech is available. This paper proposes a robust, low complexity technique within this general class that has been shown to reduce word error rate, reduce the large storage requirements associated with speaker space approaches, and eliminate the need for large numbers of utterances per speaker in training. The technique is based on representing speakers as a linear combination of clustered linear basis vectors and a procedure is presented for ML estimation these vectors from training data. Signicant word error rate reduction was obtained relative to speaker independent performance for the Resource Management and Wall Street Journal task domains. Index Terms: speech recognition, speaker adaptation

This paper addresses the issue of speaker variability and session variability in text-independent Gaussian mixture model (GMM)-based speaker verification. A speaker model adaptation procedure is proposed which is based on a joint factor analysis approach to speaker verification. It is shown in this paper that this approach facilitates the implementation of a progressive unsupervised adaptation strategy which is able to produce an improved model of speaker identity while minimizing the influence of channel variability. The paper also deals with the interaction between this model adaptation approach and score normalization strategies which act to reduce the variation in likelihood ratio scores. This issue is particularly important in establishing decision thresholds in practical speaker verification systems since the variability of likelihood ratio scores can increase as a result of progressive model adaptation. These adaptation methods have been evaluated under the adaptation paradigm defined under the NIST 2005 Speaker Recognition Evaluation Plan, which is based on conversation sides derived from telephone speech utterances. It was found that when target speaker models were trained from a single conversation, an equal error rate (EER) of 4.5% was obtained under the NIST unsupervised speaker adaptation scenario.

This paper presents a study of the interaction between frequency warping based speaker normalization algorithms, environment compensation algorithms, and discriminant feature space transformations (DFT) in providing consistent reductions in ASR word error rate (WER) over a range of acoustic degradations. Performance improvements obtained using speaker normalization algorithms, including vocal tract length normalization (VTLN) and a newly proposed augmented state space acoustic decoder, are shown to improve substantially when applied in a discriminant feature space where acoustic environment compensation has been applied. Furthermore, the effects on ASR performance of the DFT are also shown to be enhanced by reducing within class variability by applying the DFT on a speaker and an environment normalized feature space

This paper presents a decoding method for automatic speech recognition (ASR) that reduces the impact of local spectral and temporal variabilities on ASR performance. The procedure involves augmenting the standard Viterbi search for an optimum state sequence with a locally constrained search for optimum degrees of spectral warping or temporal warping applied to individual analysis frames. It is argued in the paper that this represents an efficient and effective method for compensating for local variability in speech which may have potential application to a broader array of speech transformations. The techniques are presented in the context of existing methods for frequency warping based speaker normalization and existing methods for computation of dynamic features for ASR. The modified decoding algorithms were evaluated in both clean and noisy task domains using

The paper presents methods for improving the efficiency of automatic speech recognition (ASR) decoders in multiuser applications. The methods involve allocating ASR resources to service human-machine dialogs in deployments that make use of many low-cost, commodity servers. It is shown that even very simple strategies for efficient allocation of ASR servers to incoming utterances has the potential to double the capacity of a multiuser deployment. This is important because, while there has been a great deal of work applied to increasing the efficiency of individual ASR engines, there has been little effort applied to increasing overall efficiency at peak loads in multiuser scenarios.

This paper addresses issues that are specific to the implementation of automatic speech recognition (ASR) applications and services in client-server scenarios. It is assumed in all of these scenarios that functionality in a human-machine dialog system is distributed between mobile client devices and network based multi-user media and application servers. It is argued that, while there has already been a great deal of research addressing issues relating to the communications channels associated with these scenarios, there are many additional problems that have received relatively little attention. These include issues of how environmental and speaker robustness algorithms are implemented in mobile domains and how multiple ASR channels can be implemented more efficiently in multi-user deployments. Preliminary results are summarized showing the effect of user specific unsupervised adaptation and normalization algorithms on ASR performance in mobile domains. Results are also presented demonstrating the efficiencies that are obtainable from using intelligent algorithms for assigning ASR decoders to computation servers in multi-user deployments.

A distributed framework for implementing automatic speech recognition (ASR) services on wireless mobile devices is presented. The framework is shown to scale easily to support a large number of mobile users connected over a wireless network and degrade gracefully under peak loads. The importance of using robust acoustic modeling techniques is demonstrated for situations when the use of specialized acoustic transducers on the mobile devices is not practical. It is shown that unsupervised acoustic normalization and adaptation techniques can reduce speech recognition word error rate (WER) by 30 percent. It is also shown that an unsupervised paradigm for updating and applying these robust modeling algorithms can be efficiently implemented within the distributed framework.

This paper presents a set of acoustic feature pre-processing techniques that are applied to improving automatic speech recognition (ASR) performance on noisy speech recognition tasks. The principal contribution of this paper is an approach for cepstrum-domain feature compensation in ASR which is motivated by techniques for decomposing speech and noise that were originally developed for noisy speech enhancement. This approach is applied in combination with other feature compensation algorithms to compensating ASR features obtained from a mel-filterbank cepstrum coefficient front-end. Performance comparisons are made with respect to the application of the minimum mean squared error log spectral amplitude (MMSE-LSA) estimator based speech enhancement algorithm prior to feature analysis. An experimental study is presented where the feature compensation approaches described in the paper are found to greatly reduce ASR word error rate compared to uncompensated features under environmental and channel mismatched conditions.

We propose a cepstrum-domain model combination method for automatic speech recognition in noisy environments. The distinguishing aspect of the method is that noise-corrupted speech is decomposed into clean speech and noise components directly in the cepstrum domain without having to transform to the linear spectrum domain as is necessary for many existing model combination approaches. This is accomplished by exploiting the properties of the minimum mean squared error-log spectral amplitude (MMSE-LSA) based speech enhancement algorithm. As a result, a clean speech hidden Markov model (HMM) is easily compensated for a noise-corrupted domain by adding the means and covariance matrices of the clean speech HMM and those of an estimated noise model. The complexity of the proposed model combination procedure is significantly reduced with respect to conventional parallel model combination. The procedure was applied to a noisy connected digit recognition task. A 40% reduction in word error rate was achieved when it was combined with acoustic feature compensation techniques under mismatched environmental and channel conditions.

We propose a feature enhancement algorithm for wireless speech recognition in adverse acoustic environments. A speech recognition system is realized at the network side of a wireless communications system and feature parameters are extracted directly from the bitstream of the speech coder employed in the system, where the feature parameters are composed of spectral envelope information and coder-specific information. The coder-specific information is apt to be affected by environmental noise because the speech coder fails to generate high quality speech in noisy environments. We first found that enhancing noisy speech prior to speech coding improves the recognizer's performance. However, our aim was to develop a robust front-end operating at the network side of a wireless communications system without regard to whether speech enhancement was applied at the sender side. We investigated the effect of a speech enhancement algorithm on the bitstream-based feature parameters. Consequently, a feature enhancement algorithm is proposed which incorporates feature parameters obtained from the decoded speech and a noise suppressed version of the decoded speech. The coder-specific information can also be improved by re-estimating the codebook gains and residual energy from the enhanced residual signal. HMM-based connected digit recognition experiments show that the proposed feature enhancement algorithm significantly improves recognition performance at low signal-to-noise ratio (SNR) without causing poorer performance at high SNR. From large vocabulary speech recognition experiments with far-field microphone speech signals recorded in an office environment, we show that the feature enhancement algorithm greatly improves word recognition accuracy.

The paper is concerned with the implementation of automatic speech recognition (ASR) based services on wireless mobile devices. Techniques are investigated for improving the performance of ASR systems in the context of the devices themselves, the environments that they are used in, and the networks they are connected to. A set of ASR tasks and ASR system architectures that are applicable to a wide range of simple mobile devices is presented. A prototype ASR based service is defined and the implementation of the service on a wireless mobile device is described. A database of speech utterances was collected from a population of fifty users interacting with this prototype service in multiple environments. An experimental study was performed where model compensation procedures for improving acoustic robustness and lattice rescoring procedures for reducing task perplexity were evaluated on this speech corpus.

This paper presents a set of acoustic feature pre–processing techniques that are applied to improving automatic speech recognition (ASR) performance on the Aurora 2 noisy speech recognition task. The principal contribution of this paper is an approach for cepstrum domain feature compensation in ASR which is motivated by techniques for decomposing speech and noise that were originally developed for noisy speech enhancement. This approach is applied in combination with other feature compensation algorithms to compensating ASR features obtained from a mel–filterbank cepstrum coefficient (MFCC) front–end. Performance comparisons are made with respect to the application of the minimum mean squared error log spectral amplitude estimator (MMSE–LSA) based speech enhancement algorithm prior to feature analysis. An experimental study is presented where the feature compensation approaches described in the paper are found to reduce ASR word error rate by as much as 31% relative to uncompensated features under simulated environmental and channel mismatched conditions.

This paper describes the design and evaluation of an automatic speech recognition (ASR) system on the Naval Research Laboratory Speech In Noise (SPINE) speech corpus. This corpus represents a task which involves human-human interaction on a constrained problem solving scenario under six di erent simulated noisy environments. Acoustic and language modeling were performed using a small dataset taken entirely from a subset of the acoustic environments. Speech recognition was performed on continuous conversations by detecting speech utterances, performing acoustic feature analysis and normalization, and adapting HMMmodels in multiple passes over each conversation-side. The ASR word accuracy (WAC) ranged from 77 percent in an o ce environment to 61 percent in conditions that include signi cant levels of background speech and noise.

An experimental study of the application of scale-transform to improve the performance of speaker independent continuous speech recognition, is presented in this paper. Three major results are described. First, a comparison was made between the scale-transform based magnitude cepstrum coefficients (STCC) and mel-scale filter bank cepstrum coefficients (MFCC) on a telephone based connected digit recognition task. It was shown that the STCC can obtain a performance that is close to that of the MFCC. Second, a simple frequency-normalization procedure was applied to the scale-transform representation that improved performance on the connected digit recognition task with respect to the MFCC. Finally, in a more controlled experimental setting using the TIMIT database, it was shown that the application of phone-specific frequency warpings improved phone classification performance over using a single speaker-specific warping. This last result may have general implications for all frequency warping based speaker normalization procedures.

This paper introduces a set of acoustic modeling and decoding techniques for utterance verification (UV) in hidden Markov model (HMM) based continuous speech recognition (CSR). Utterance verification in this work implies the ability to determine when portions of a hypothesized word string correspond to incorrectly decoded vocabulary words or out-of-vocabulary words that may appear in an utterance. This capability is implemented here as a likelihood ratio (LR) based hypothesis testing procedure for, verifying individual words in a decoded string. There are two UV techniques that are presented here. The first is a procedure for estimating the parameters of UV models during training according to an optimization criterion which is directly related to the LR measure used in UV. The second technique is a speech recognition decoding procedure where the "best" decoded path is defined to be that which optimizes a LR criterion. These techniques were evaluated in terms of their ability to improve UV performance on a speech dialog task over the public switched telephone network. The results of an experimental study presented in the paper shows that LR based parameter estimation results in a significant improvement in UV performance for this task. The study also found that the use of the LR based decoding procedure, when used in conjunction with models trained using the LR criterion, can provide as much as an 11% improvement in UV performance when compared to existing UV procedures. Finally, it was also found that the performance of the LR decoder was highly dependent on the use of the LR criterion in training acoustic models. Several observations are made in the paper concerning the formation of confidence measures for UV and the interaction of these techniques with statistical language models used in ASR.

This paper investigates techniques for minimizing the impact of non-speech events on the performance of large vocabulary continuous speech recognition (LVCSR) systems. An experimental study is presented that investigates whether the careful manual labeling of disfluency and background events in conversational speech can be used to provide an additional level of supervision in training HMM acoustic models and statistical language models. First, techniques are investigated for incorporating explicitly labeled disfluency and background events directly into the acoustic HMM. Second, phrase-based statistical language models are trained from utterance transcriptions which include labeled instances of these events. Finally, it is shown that significant word accuracy and run-time performance improvements are obtained for both sets of techniques on a telephone-based spoken language understanding task.

The goal of the work described in this paper is to develop and evaluate procedures for automatic estimation of modulation spectrum filters to compensate for distortions in the modulation spectrum domain. The modulation spectrum (MS) is often used to describe the time sequence of spectral parameters (TSSPs) that are derived from the speech waveform, and is thought to be a good representation of many sources of variability in speech. These procedures will be used in the context of automatic speech recognition (ASR) applications where there is likely to be a significant mismatch in the MS characteristics that exist for system training and evaluation. Results are presented describing application of the algorithm to one task involving an artificially introduced MS distortion and to another task involving differences in speaking styles for training and testing. It is shown in the paper that these techniques are able to compensate for the effects of artificially introduced distortions that appear in testing. It is also shown that a small degree of compensation is obtained for speaking style mismatch, and this result is compared with the measured effects of the speaking style differences in the MS domain. An algorithm is presented for automatic estimation of the An algorithm to estimate automatically filters in the modulation spectrum domain. These are used to compensate for distortions in this domain or to obtain the difference coefficients that are a part of the acoustic vector handed over to the HMM-based speech model. The mathematical properties of the new algorithm are analyzed. Its performance is studied in two different experiments: in the first the goal is to alleviate an artificially simulated distortion while in the second we try to compensate for speaking rate distortions in a database which has two distinct parts differing significantly in speaking rate.

A modi ed decoding algorithm for automatic speech recognition ASR will be described which facilitates a closer coupling between the acoustic and language modeling components of a speech recognition system.This closer coupling is obtained by extracting word level measures of acoustic con dence during decoding, and making coded representations of these con dence measures available to the ASR network during decoding.A simulation of this decoding strategy is implemented using a word lattice rescoring paradigm.A joint acoustic language model will be described where linguistic context is augmented to include the encoded values of acoustic con dence.Finally, the performance of the word lattice based implementation of the decoding algorithm will be evaluated on a large vocabulary natural language understanding task.

Methods for utterance verification (UV) and their integration into statistical language modeling and spoken language understanding formalisms for a large vocabulary spoken understanding system are presented. The paper consists of three parts. First, a set of acoustic likelihood ratio based utterance verification techniques are described and applied to the problem of rejecting portions of a hypothesized word string that may have been incorrectly decoded by a large vocabulary continuous speech recognizer. Second, a procedure for integrating the acoustic level confidence measures with the statistical language model is described. Finally, the effect of integrating acoustic level confidence into the spoken language understanding unit (SLU) in a call-type classification task is discussed. These techniques were evaluated on utterances collected from a highly unconstrained call routing task performed over the telephone network. They have been evaluated in terms of their ability to classify utterances into a set of fifteen semantic actions corresponding to call-types that are accepted by the application.

In an effort to reduce the degradation in speech recognition performance caused by variation in vocal tract shape among speakers, a frequency warping approach to speaker normalization is investigated. A set of low complexity, maximum likelihood based frequency warping procedures have been applied to speaker normalization for a telephone based connected digit recognition task. This paper presents an efficient means for estimating a linear frequency warping factor and a simple mechanism for implementing frequency warping by modifying the filterbank in mel-frequency cepstrum feature analysis. An experimental study comparing these techniques to other well-known techniques for reducing variability is described. The results have shown that frequency warping is consistently able to reduce word error rate by 20% even for very short utterances.

Frequency warping approaches to speaker normalization have been proposed and evaluated on various speech recognition tasks. These techniques have been found to significantly improve performance even for speaker independent recognition from short utterances over the telephone network. In maximum likelihood (ML) based model adaptation a linear transformation is estimated and applied to the model parameters in order to increase the likelihood of the input utterance. The purpose of this paper is to demonstrate that significant advantage can be gained by performing frequency warping and ML speaker adaptation in a unified framework. A procedure is described which compensates utterances by simultaneously scaling the frequency axis and reshaping the spectral energy contour. This procedure is shown to reduce the error rate in a telephone based connected digit recognition task by 30-40%.

This paper investigates procedures for obtaining user-configurable speech recognition vocabularies. These procedures use example utterances of vocabulary words to perform unsupervised automatic acoustic baseform determination in terms of a set of speaker independent subword acoustic units. Several procedures, differing both in the definition of subword acoustic model context and in the phonotactic constraints used in decoding have been investigated. The tendency of input utterances to contain out-of-vocabulary or non-speech information is accounted for using likelihood ratio based utterance verification procedures. Comparisons of different definitions of the likelihood ratio used for utterance verification and of different criteria for estimating parameters used in the likelihood ratio test have been performed. The performance of these techniques has been evaluated on utterances taken from a trial of a voice label recognition service.

There is increasing interest in systems which attempt to automate a task or a transaction using speech input and output. To function effectively with imperfect speech recognition, such systems require an estimate of which words in the output from the recogniser are likely to be correct and which can probably be disregarded as incorrect, i.e. a confidence-measure for each decoded word. We define a measure for evaluating the effectiveness of a post-classifier which estimates confidence-measures, and describe the development of a post-classifier for words decoded from the SWITCHBOARD database, which uses statistics derived from a Viterbi decoder. Without any grouping of the decoded word-classes, the post-classifier increased the probability of deciding whether a decoded word was correct or incorrect by 32%. When grouping was used, longer words showed an improvement of 65%.

In an effort to reduce the degradation in speech recognition performance caused by variation in vocal tract shape among speakers, a frequency warping approach to speaker normalization is investigated. A set of low complexity, maximum likelihood based frequency warping procedures have been applied to speaker normalization for a telephone based connected digit recognition task. This paper presents an efficient means for estimating a linear frequency warping factor and a simple mechanism for implementing frequency warping by modifying the filter-bank in mel-frequency cepstrum feature analysis. An experimental study comparing these techniques to other well-known techniques for reducing variability is described. The results showed that frequency warping was consistently able to reduce word error rate by 20% even for very short utterances.

It is often necessary in speech recognition to include a mechanism for verifying decoded utterances in order to account for incorrectly decoded vocabulary words and utterances corresponding to words or sounds that are not included in a prespecified lexicon. This paper describes an utterance verification procedure for hidden Markov model (HMM) based continuous speech recognition that is based on a likelihood ratio (LR) criterion. There are two important contributions. The first is a search algorithm which directly optimizes a likelihood ratio criterion. This search algorithm is important because it allows decoding to be performed in speech recognition according to the same measure of confidence that is used in hypothesis testing. The second contribution is a corresponding training procedure for estimating model parameters which also directly optimizes the same likelihood ratio criterion. These techniques are applied to spontaneous spoken queries in the context of a "movie locator" dialog system.

A procedure is proposed for verifying the occurrence of string hypotheses produced by a hidden Markov model (HMM) based continuous speech recognizer. Most existing procedures verify word hypotheses through likelihood ratio scoring procedures computed using ad hoc approximations for the density of the alternative hypothesis in the denominator of the likelihood ratio statistic. The discriminative training procedure described in this paper attempts to adjust the parameters of the null hypothesis and the alternate hypothesis models to increase the power of a hypothesis test for utterance verification. The training procedure was evaluated for its ability to detect a twenty word vocabulary in a subset of the Switchboard conversational speech corpus. Experimental results show that the use of this procedure results in significant improvement in the word verification operating characteristic, as well as an improvement in the overall system performance.

This paper introduces and motivates the use of Gaussian mixture models (GMM) for robust text-independent speaker identification. The individual Gaussian components of a GMM are shown to represent some general speaker-dependent spectral shapes that are effective for modeling speaker identity. The focus of this work is on applications which require high identification rates using short utterance from unconstrained conversational speech and robustness to degradations produced by transmission over a telephone channel. A complete experimental evaluation of the Gaussian mixture speaker model is conducted on a 49 speaker, conversational telephone speech database. The experiments examine algorithmic issues (initialization, variance limiting, model order selection), spectral variability robustness techniques, large population performance, and comparisons to other speaker modeling techniques (uni-modal Gaussian, VQ codebook, tied Gaussian mixture, and radial basis functions). The Gaussian mixture speaker model attains 96.8% identification accuracy using 5 second clean speech utterances and 80.8% accuracy using 15 second telephone speech utterances with a 49 speaker population and is shown to outperform the other speaker modeling techniques on an identical 16 speaker telephone speech task.>

Discusses the problem of robust parametric model estimation and classification in noisy acoustic environments. Characterization and modeling of the external noise sources in these environments is in itself an important issue in noise compensation. The techniques described provide a mechanism for integrating parametric models of the acoustic background with the signal model so that noise compensation is tightly coupled with signal model training and classification. Prior information about the acoustic background process is provided using a maximum likelihood parameter estimation procedure that integrates an a priori model of the acoustic background with the signal model. An experimental study is presented on the application of this approach to text-independent speaker identification in noisy acoustic environments. Considerable improvement in speaker classification performance was obtained for classifying unlabeled sections of conversational speech utterances from a 16-speaker population under cross-environment training and testing conditions.>

The Lincoln hidden Markov model (HMM)-based word spotting system has demonstrated good performance in spotting keywords in completely unconstrained continuous speech utterances (see R.C. Rose and D.B. Paul, 1990). The word spotter has been evaluated under a number of scenarios, and has been integrated into a system that performs the higher level of classifying conversational speech messages according to topic. In all of these scenarios, anywhere from 25 to 78 examplars per keyword have been used to train the subword acoustic HMMs that are used in the word spotter. In most word spotting applications it is simply not possible to collect such a large number of spoken utterances for all the keywords in the vocabulary every time the system is to be reconfigured for a given task. Therefore, it is essential that techniques be developed to reduce the amount of task-specific speech data required for training HMM-based work spotters.>

A procedure for text-independent speaker identification in noisy environments where the interfering background signals cannot be characterized using traditional broadband or impulsive noise models is examined. In the procedure, both the speaker and the background processes are modeled using mixtures of Gaussians. Speaker and background models are integrated into a unified statistical framework allowing the decoupling of the underlying speech process from the noise corrupted observations via the expectation-minimization algorithm. Using this formalism, speaker model parameters are estimated in the presence of the background process, and a scoring procedure is implemented for computing the speaker likelihood in the noise corrupted environment. The performance was evaluated using a 16-speaker conversational speech database with both speech babble and white noise background processes.>

Discriminant techniques for training speaker-independent hidden Markov model (HMM) parameters in a wordspotter are proposed for improving the discrimination between keyword utterances and background speech utterances. These techniques are applied in the context of completely unconstrained conversational speech utterances that contain a large number of nonvocabulary words. Hidden Markov keyword model parameters are modified in the procedure so that keyword models are made more probable with respect to a network of background speech models. While there have been many discriminant training techniques proposed for relatively constrained closed vocabulary tasks, the implementation described deals with the more general problem of discriminating keyword utterances from a broad class of acoustic events. Results are presented showing improved wordspotter operating characteristics on a conversational speech utterance.>

The components of a speech message information retrieval system include an acoustic front end which provides an incomplete transcription of a spoken message, and a message classifier that interprets the incomplete transcription and classifies the message according to message category. The techniques and experiments described are concerned with the integration of these components and represent the first demonstration of a complete system that accepts speech messages as input and produces as estimated message class as output. The complete system has been implemented on special-purpose digital signal processing hardware and demonstrated using live speech input. The results obtained on a conversational speech task have demonstrated the feasibility of the technology and also illustrate the need for further work. Even with a perfect acoustic front end, a message classification accuracy of only 78% was obtained with a 126 keyword vocabulary.>

The use of probabilistic mixture densities for text-independent speaker identification in a noisy telephone channel environment is investigated. Two techniques for noise compensation are considered. In the first approach, a background noise model is integrated directly into the model for speech. In the second approach, noise preprocessing techniques are used to compensate noisy observations before passing them along to the speaker identification on conversational utterances collected over long-distance telephone channels from ten speakers. The integrated noise model was noted for having a noise suppression characteristic that arises naturally from the statistical model, which is important if one wishes to avoid the ad hoc fine tuning of thresholds required in the noise processing approach implemented.>

A speaker-independent hidden Markov model (HMM) keyword recognizer (KWR) based on a continuous-speech-recognition model is presented. The baseline keyword recognition system is described, and techniques for dealing with nonkeyword speech and linear channel effects are discussed. The training of acoustic models to provide an explicit representation of nonvocabulary speech is investigated. A likelihood ratio scoring procedure is used to account for sources of variability affecting keyword likelihood scores. An acoustic class-dependent spectral normalization procedure is used to provide explicit compensation for linear channel effects. Keyword recognition results for a standard conversational speech task with a 20-keyword vocabulary reach 82% probability of detection at a false alarm rate of 12 false alarms per keyword per hour.>

An acoustic-class-dependent technique for text-independent speaker identification on very short utterances is described. The technique is based on maximum-likelihood estimation of a Gaussian mixture model representation of speaker identity. Gaussian mixtures are noted for their robustness as a parametric model and their ability to form smooth estimates of rather arbitrary underlying densities. Speaker model parameters are estimated using a special case of the iterative expectation-maximization (EM) algorithm, and a number of techniques are investigated for improving model robustness. The system is evaluated using a 12 reference speaker population from a conversational speech database. It achieves 80% average text-independent speaker identification performance for a 1-s test utterance length.>

This paper presents a formal objective and subjective comparison of a number of LPC vocoders which operate at bit rates around 4800 bps. In this work, particular emphasis is placed on the Self Excited Vocoder (SEV), a new speech coding approach which was introduced by the authors at ICASSP86 [1]. Many members of a class of LPC vocoders of which the SEV, the well known Multiple Pulse Excited Linear Predictive Coder (MPLPC) [2], and Code Excited Linear Predictive Coder (CELPC) [3] are members, are simulated and compared. Through these experiments, the SEV is shown to be a low complexity, simply implemented speech coder that is competitive with the other coders in this class in producing high quality speech at low bit rates.

This paper presents a new speech coding technique for near toll-quality performance at around 4800 bps. Currently, the most promising techniques at these bit rates emphasize the efficient coding of residual information in a Linear Predictive Coding context. The Multipulse Linear Predictive Coder (MPLPC) [1] and the Code Excited Linear Predictive Coder (CELPC) [2] are examples where considerable success has been achieved at medium bit rates. This paper introduces the Self Excited Vocoder (SEV), a new speech coding technique which belongs to the same class of coders as the MPLPC and the CELPC.

Multiple pulse excited linear predictive coding (MPLPC) has recently received a great deal of attention in the literature as an attractive means of speech coding at data rates below 10 Kbits/second. The existing approaches to MPLPC analysis arrive at the parameters for an all-pole model by minimizing the mean squared modeling error before attempting to find a set of pulses to excite the model. The strategy proposed here selects the all-pole parameters to concentrate the model excitation in a finite number of locations. The goal is then to produce a maximally pulse-like residual as a result of the all-pole parameter estimation.