Jointly training a speech enhancement (SE) front-end and an automatic speech recognition (ASR) back-end has been investigated as a way to mitigate the influence of processing distortion generated by single-channel SE on ASR. In this paper, we investigate the effect of such joint training on the signal-level characteristics of the enhanced signals from the viewpoint of the decomposed noise and artifact errors. The experimental analyses provide two novel findings: 1) ASR-level training of the SE front-end reduces the artifact errors while increasing the noise errors, and 2) simply interpolating the enhanced and observed signals, which achieves a similar effect of reducing artifacts and increasing noise, improves ASR performance without jointly modifying the SE and ASR modules, even for a strong ASR back-end using a WavLM feature extractor. Our findings provide a better understanding of the effect of joint training and a novel insight for designing an ASR agnostic SE front-end.

It is challenging to improve automatic speech recognition (ASR) performance in noisy conditions with a single-channel speech enhancement (SE) front-end. This is generally attributed to the processing distortions caused by the nonlinear processing of single-channel SE front-ends. However, the causes of such degraded ASR performance have not been fully investigated. How to design single-channel SE front-ends in a way that significantly improves ASR performance remains an open research question. In this study, we investigate a signal-level numerical metric that can explain the cause of degradation in ASR performance. To this end, we propose a novel analysis scheme based on the orthogonal projection-based decomposition of SE errors. This scheme manually modifies the ratio of the decomposed interference, noise, and artifact errors, and it enables us to directly evaluate the impact of each error type on ASR performance. Our analysis reveals the particularly detrimental effect of artifact errors on ASR performance compared to the other types of errors. This provides us with a more principled definition of processing distortions that cause the ASR performance degradation. Then, we study two practical approaches for reducing the impact of artifact errors. First, we prove that the simple observation adding (OA) post-processing (i.e., interpolating the enhanced and observed signals) can improve the signal-to-artifact ratio. Second, we propose a novel training objective, called artifact-boosted signal-to-distortion ratio (AB-SDR), which forces the model to estimate the enhanced signals with fewer artifact errors. Through experiments, we confirm that both the OA and AB-SDR approaches are effective in decreasing artifact errors caused by single-channel SE front-ends, allowing them to significantly improve ASR performance.

It is challenging to improve automatic speech recognition (ASR) performance in noisy conditions with single-channel speech enhancement (SE).In this paper, we investigate the causes of ASR performance degradation by decomposing the SE errors using orthogonal projection-based decomposition (OPD).OPD decomposes the SE errors into noise and artifact components.The artifact component is defined as the SE error signal that cannot be represented as a linear combination of speech and noise sources.We propose manually scaling the error components to analyze their impact on ASR.We experimentally identify the artifact component as the main cause of performance degradation, and we find that mitigating the artifact can greatly improve ASR performance.Furthermore, we demonstrate that the simple observation adding (OA) technique (i.e., adding a scaled version of the observed signal to the enhanced speech) can monotonically increase the signal-to-artifact ratio under a mild condition.Accordingly, we experimentally confirm that OA improves ASR performance for both simulated and real recordings.The findings of this paper provide a better understanding of the influence of SE errors on ASR and open the door to future research on novel approaches for designing effective single-channel SE front-ends for ASR.

Recent work on multichannel end-to-end automatic speech recognition (ASR) has shown that multichannel speech enhancement and speech recognition functions can be integrated into a deep neural network (DNN)-based system, and promising experimental results have been shown using the CHiME-4 and AMI corpora. In other recent DNN-based hidden Markov model (DNN-HMM) hybrid architectures, the effectiveness of speaker adaptation has been well established. Motivated by these results, we propose a multi-path adaptation scheme for end-to-end multichannel ASR, which combines the unprocessed noisy speech features with a speech-enhanced pathway to improve upon previous end-to-end ASR approaches. Experimental results using CHiME-4 show that (1) our proposed multi-path adaptation scheme improves ASR performance and (2) adapting the encoder network is more effective than adapting the neural beamformer, attention mechanism, or decoder network.

Adapting acoustic models to speakers have shown to greatly improve performance for many tasks. Among the adaptation approaches, exploiting auxiliary features characterizing speakers or environments has received great attention because they allow rapid adaptation, i.e. adaptation with limited amount of speech data such as a single utterance. However, the auxiliary features are usually computed in batch mode, which causes some inevitable latency. In this paper we explore an extension of the auxiliary feature-based adaptation to online processing. We employ auxiliary features obtained from bottleneck speaker vectors and extend their computation to online processing using cumulative moving averaging. We test our proposed approach for deep CNN-based acoustic models, using context adaptive networks to exploit the auxiliary features. Experimental results on the CHiME-3 task demonstrate that the proposed approach can realize online speaker adaptation.

We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of gLasso regularization, one for outgoing weight vectors and another for incoming weight vectors, as well as two sizes of DNNs: 2048 hidden layer nodes and 4096 nodes. Furthermore, we compare gLasso and L2 regularizers. Our experiment results demonstrate that our DNN training, in which the gLasso regularizer was embedded, successfully selected the hidden layer nodes that are necessary and sufficient for achieving high classification power.

There have been many attempts to classify imbalanced data, since this classification is critical in a wide variety of applications related to the detection of anomalies, failures, and risks. Many conventional methods, which can be categorized into sampling, cost-sensitive, or ensemble, include heuristic and task dependent processes. In order to achieve a better classification performance by formulation without heuristics and task dependence, we propose confusion-matrix-based kernel logistic regression (CM-KLOGR). Its objective function is the harmonic mean of various evaluation criteria derived from a confusion matrix, such criteria as sensitivity, positive predictive value, and others for negatives. This objective function and its optimization are consistently formulated on the framework of KLOGR, based on minimum classification error and generalized probabilistic descent (MCE/GPD) learning. Due to the merits of the harmonic mean, KLOGR, and MCE/GPD, CM-KLOGR improves the multifaceted performances in a well-balanced way. This paper presents the formulation of CM-KLOGR and its effectiveness through experiments that comparatively evaluated CM-KLOGR using benchmark imbalanced datasets.

Recently, a Hybrid DNN-HMM recognizer trained with the Speaker Adaptive Training (SAT) concept was successfully modified to a more effective speaker-adaptation-oriented recognizer whose DNN front-end adopted a Linear Transformation Network (LTN) Speaker Dependent (SD) module. However, the size of SD modules is still large, which incurs high storage costs and the risk of over-training. To alleviate this problem, we analyze the characteristics of an LTN module by focusing on the relation between its size and its feature-representation capability. Moreover, we propose a new SAT-based scheme for reducing the LTN size using SVD-based matrix compression. Evaluation experiments on the TED Talks corpus prove that our LTN size-reduction scheme not only maintains the adaptation performance of the original LTN-embedded, SAT-based DNN-HMM recognizer but also further increases it especially in cases where the speech data available for adaptation training are severely limited.

Imbalanced data classification, which is a common and important problem in various fields related to the detection of anomaly, failure, and risk, has been studied intensively. Conventional methods are based on sampling, misclassification costs, or ensemble of classifiers, and many of them are heuristic and task dependent. Aiming at a higher classification performance with the solution of such problems, we propose a confusion-matrix-based kernel logistic regression (CM-KLOGR). After pretraining with a cross-entropy error function, CM-KLOGR retrains its parameters using the harmonic mean of the various evaluation criteria, including sensitivity, positive predictive value, and others, which are derived from a confusion matrix. CM-KLOGR inherits the advantages of kernel logistic regression, and it has the potential to raise the values of all the evaluation criteria in a well-balanced way. This paper presents the concept and formulation of CM-KLOGR, accompanied by the results of an exploratory experiment using an imbalanced biomedical dataset.

Recently, a novel speaker adaptation method was proposed that applied the Speaker Adaptive Training (SAT) concept to a speech recognizer consisting of a Deep Neural Network (DNN) and a Hidden Markov Model (HMM), and its utility was demonstrated. This method implements the SAT scheme by allocating one Speaker Dependent (SD) module for each training speaker to one of the intermediate layers of the front-end DNN. It then jointly optimizes the SD modules and the other part of network, which is shared by all the speakers. In this paper, we propose an improved version of the above SAT-based adaptation scheme for a DNN-HMM recognizer. Our new training adopts a Linear Transformation Network (LTN) for the SD module, and such LTN employment leads to more appropriate regularization in both the SAT and adaptation stages by replacing an empirically selected anchorage of a network for regularization in the preceding SAT-DNN-HMM with a SAT-optimized anchorage. We elaborate the effectiveness of our proposed method over TED Talks corpus data. Our experimental results show that a speaker-adapted recognizer using our method achieves a significant word error rate reduction of 9.2 points from a baseline SI-DNN recognizer and also steadily outperforms speaker-adapted recognizers, each of which originates from the preceding SAT-based DNN-HMM.

Among many speaker adaptation embodiments, Speaker Adaptive Training (SAT) has been successfully applied to a standard Hidden-Markov-Model (HMM) speech recognizer, whose state is associated with Gaussian Mixture Models (GMMs). On the other hand, recent studies on Speaker-Independent (SI) recognizer development have reported that a new type of HMM speech recognizer, which replaces GMMs with Deep Neural Networks (DNNs), outperforms GMM-HMM recognizers. Along these two lines, it is natural to conceive of further improvement to a preset DNN-HMM recognizer by employing SAT. In this paper, we propose a novel training scheme that applies SAT to a SI DNN-HMM recognizer. We then implement the SAT scheme by allocating a Speaker-Dependent (SD) module to one of the intermediate layers of a seven-layer DNN, and elaborate its utility over TED Talks corpus data. Experiment results show that our speaker-adapted SAT-based DNN-HMM recognizer reduces the word error rate by 8.4% more than that of a baseline SI DNN-HMM recognizer, and (regardless of the SD module allocation) outperforms the conventional speaker adaptation scheme. The results also show that the inner layers of DNN are more suitable for the SD module than the outer layers.

We propose a new method for automatically determining the smoothness of smooth classification error count loss for the recent Large Geometric Margin Minimum Classification Error (LGM-MCE) training. The method uses the Parzen-estimation-based formalization of MCE training, and it realizes the determination through the maximum likelihood estimation of error count risk in the one-dimensional geometric-margin-based misclassification measure. In the LGM-MCE framework, increase in the loss smoothness directly leads to an effect of producing virtual samples, which are expected to increase the training robustness to unseen samples. Focusing on this point, we also theoretically clarify the mechanism of this virtual sample generation. Through experiments, the utility of the proposed smoothness determination method is demonstrated, and the mechanism of producing virtual samples and its effect in robustness increase are also clearly illustrated.

Minimum Classification Error (MCE) training, which can be used to achieve minimum error classification of various types of patterns, has attracted a great deal of attention. However, to increase classification robustness, a conventional MCE framework has no practical optimization procedures like geometric margin maximization in Support Vector Machine (SVM). To realize high robustness in a wide range of classification tasks, we derive the geometric margin for a general class of discriminant functions and develop a new MCE training method that increases the geometric margin value. We also experimentally demonstrate the effectiveness of our new method using prototype-based classifiers.

This paper presents a novel unified view of a wide variety of objective functions suitable for discriminative training applied to sequential pattern recognition problems, such as automatic speech recognition. Focusing on a central component of conventional objective functions, the sum of modified joint probabilities of observations and strings, the analysis generalizes these objective functions by weighting each term in the sum by an important function, the negative exponential of difference measure between strings. The interesting and valuable results of this investigation are highlighted in a comprehensive relationship chart that covers all of the common approaches (Maximum Mutual Information, Minimum Classification Error, Minimum Phone/Word Error), as well as corresponding novel generalizations and modifications of those approaches.

Subsequence time-series clustering (STSC), which consists of subsequence cutout with a sliding window and k-means clustering, had been commonly used in time-series data mining. However, a problem was pointed out that STSC always generates moderate sinusoidal patterns independently of the input. To address this problem, we theoretically explain and empirically confirm the similarity between STSC and moving average. The present analysis is consistent with, and simpler than, one of the most important analyses of STSC. We also question the pattern extraction in the time domain and discuss another solution.

The minimum classification error (MCE) framework for discriminative training is a simple and general formalism for directly optimizing recognition accuracy in pattern recognition problems. The framework applies directly to the optimization of hidden Markov models (HMMs) used for speech recognition problems. However, few if any studies have reported results for the application of MCE training to large-vocabulary, continuous-speech recognition tasks. This article reports significant gains in recognition performance and model compactness as a result of discriminative training based on MCE training applied to HMMs, in the context of three challenging large-vocabulary (up to 100 k word) speech recognition tasks: the Corpus of Spontaneous Japanese lecture speech transcription task, a telephone-based name recognition task, and the MIT Jupiter telephone-based conversational weather information task. On these tasks, starting from maximum likelihood (ML) baselines, MCE training yielded relative reductions in word error ranging from 7% to 20%. Furthermore, this paper evaluates the use of different methods for optimizing the MCE criterion function, as well as the use of precomputed recognition lattices to speed up training. An overview of the MCE framework is given, with an emphasis on practical implementation issues

This article describes recent results obtained for two challenging large-vocabulary speech recognition tasks using the minimum classification error (MCE) approach to discriminative training. Weighted finite state transducers (WFSTs) are used throughout to represent correct and competing string candidates. The primary task examined is a 22 K word, real-world, telephone-based name recognition task. Lattice-derived WFSTs were used successfully to speed up the MCE training procedure. The results of this difficult task follow the classic picture of discriminative training: small acoustic models trained with MCE outperform much larger baseline models trained with maximum likelihood. MCE training substantially improves the performance of the larger models as well. We also present preliminary results on the 30 K word corpus of spontaneous Japanese (CSJ) lecture speech transcription task, with a training set of 190 hours of audio.

This paper presents a method for modelling the speech spectral envelope using a mixture of Gaussians (MoG). A novel variational Bayesian (VB) framework for Gaussian mixture modelling of a histogram enables the derivation of an objective function that can be used to simultaneously optimise both model parameter distributions and model structure. A histogram representation of the STRAIGHT spectral envelope, which is free of glottal excitation information, is used for parametrisation using this MoG model. This results in a parameterisation scheme that purely models the vocal tract resonant characteristics. Maximum likelihood (ML) and variational Bayesian (VB) solutions of the mixture model on histogram data are found using an iterative algorithm. A comparison between ML-MoG and VB-MoG spectral modelling is carried out using spectral distortion measures and mean opinion scores (MOS). The main advantages of VB-MoG highlighted in this paper include better modelling using fewer Gaussians in the mixture resulting in better correspondence of Gaussians and formant-like peaks, and an objective measure of the number of Gaussians required to best fit the spectral envelope.

In previous work, we proposed a new speech recognition technique that generates a smooth speech trajectory from hidden Markov models (HMMs) by maximizing likelihood subject to the constraints that exist between static and dynamic speech features. This paper presents a theoretical analysis of this method. We show that the approach used to generate the smoothed trajectory is equivalent to a Kalman filter. This result demonstrates that there is a strong relationship between the dynamics of delta features (and delta-delta features) in HMM-based speech recognition and Kalman filter dynamics.

Recently, open-domain question answering (ODQA) systems that extract an exact answer from large text corpora based on text input are intensively being investigated. However, the information in the first question input by a user is not usually enough to yield the desired answer. Interactions for collecting additional information to accomplish QA is needed. This paper proposes an interactive approach for spoken interactive ODQA systems. When the reliabilities for answer hypotheses obtained by an ODQA system are low, the system automatically derives disambiguous queries (DQ) that draw out additional information. The additional information based on the DQ should contribute to distinguishing effectively an exact answer and to supplementing a lack of information by recognition errors. In our spoken interactive ODQA system, SPIQA, spoken questions are recognized by an ASR system, and DQ are automatically generated to disambiguate the transcribed questions. We confirmed the appropriateness of the derived DQ by comparing them with manually prepared ones.

Ina previous work, we showed that the minimum classification error (MCE) criterion function commonly used for discriminative design of pattern recognition systems is equivalent to a Parzen window based estimate of the theoretical classification risk. In this analysis, each training token is mapped to the center of a Parzen kernel in the domain of a suitably defined random variable; the kernels are then summed and integrated over the domain of incorrect classifications, yielding the risk estimate. Here, we deepen this approach by applying Parzen estimation at an earlier stage of the overall definition of classification risk. Specifically, the new analysis uses all incorrect categories, not just the single best incorrect category, in deriving a "correctness" function that is a simple multiple integral of a Parzen kernel over the region of correct classifications. The width of the Parzen kernel determines how many competing categories to use in optimizing the resulting overall risk estimate. This analysis uses the classic Parzen estimation method to support the notion that using multiple competing categories in discriminative training is a type of smoothing that enhances generalization to unseen data.

We have proposed a new speech recognition technique that generates a speech trajectory from HMMs by maximizing the likelihood of the trajectory, while accounting for the relation between the cepstrum and the dynamic cepstrum coefficients. This method has the major advantage that the relation, which is ignored in conventional speech recognition, is directly used in the speech recognition phase. This paper describes an extension of the method for dealing with HMMs whose distributions are mixture Gaussian distributions. The method chooses the sequence of Gaussian distributions by selecting the best Gaussian distribution in the state during Viterbi decoding. Speaker-independent speech recognition experiments were carried out. The proposed method obtained an 18.2% reduction in error rate for the task, proving that the proposed method is effective even for Gaussian mixture HMMs.

Unsupervised adaptation has evolved as a popular approach for tuning the acoustic models of speaker-independent speech recognition systems to specific speakers, speaker groups or channel conditions while making use of only untranscribed data. This study focuses on procedures for unsupervised adaptation of other probabilistic models that are involved in state-of-the-art speech recognizers and on the joint adaptation of multiple knowledge sources. In particular, we outline and evaluate approaches for adapting both the language model and the pronunciation model (lexicon) without supervision. Initial experiments on off-line lecture speech transcription achieved small but promising word error rate improvements with each approach applied separately. The experimental results on the joint application of acoustic, language and pronunciation model adaptation indicate that the individually achievable performance improvements are additive.

Parametric trajectory models have been proposed to exploit this time-dependency. However, parametric trajectory modeling methods are unable to take advantage of efficient HMM training and recognition methods. We have proposed a new speech recognition technique that generates a speech trajectory using an HMM-based speech synthesis method. This method generates an acoustic trajectory by maximizing the likelihood of the trajectory while taking into account the relation between the cepstrum, delta-cepstrum, and delta-delta cepstrum. In this paper, we extend our method to a general formulation including variance training procedure. Speaker independent speech recognition experiments show that the proposed method is effective for speech recognition.

This paper presents and evaluates our recent efforts on efficient decoding for Large Vocabulary Continuous Speech Recognition in the framework of Weighted Finite State Transducers. We evaluate on-the-fly transducer composition for reduced memory consumption combined with weight smearing for a more time-synchronous language model incorporation. It turns out that in the on-line composition mode weight smoothing within the static part of the network is even more beneficial on run-time to accuracy ratio than in the fully precompiled case. Evaluations are carried out on a state-of-the-art recognition system of 10k words, cross-word triphone acoustic models and trigram language model. In this scenario, the Viterbi-search is carried out fully time-synchronously in only a single pass. The combination of on-the-fly network composition with only the unigram part of the language model smoothly compiled into the network achieves a remarkably good run-time to accuracy ratio with only moderate memory requirements.

A pattern recognizer is usually a modular system which consists of a feature extractor module and a classifier module. Traditionally, these two modules have been designed separately, which may not result in an optimal recognition accuracy. To alleviate this fundamental problem, the authors have developed a design method, named discriminative feature extraction (DFE), that enables one to design the overall recognizer, i.e., both the feature extractor and the classifier, in a manner consistent with the objective of minimizing recognition errors. This paper investigates the application of this method to designing a speech recognizer that consists of a filter-hank feature extractor and a multi-prototype distance classifier. Carefully investigated experiments demonstrate that DFE achieves the design of a better recognizer and provides an innovative recognition-oriented analysis of the filter-bank, as an alternative to conventional analysis based on psychoacoustic expertise or heuristics.

We describe an experimental study for discovering underlying laws of market capitalization using BS (Balance Sheet) items. For this purpose, we apply law discovery methods based on neural networks: RF5 (Rule Finder) discovers a single numeric law from data containing only numeric values, RF6 discovers a set of nominally conditioned polynomials from data containing both nominal and numeric values, and MCV regularizer is used to improve both the generalization performance and the readability. Our preliminary experimental results show that these methods are promising for discovering underlying laws from financial data.

This paper describes progress on a commercial application of the MECS recognition system to the task of recognizing Japanese family names spoken by customers into the answering machines of a large marketing/human resource company. The task is thus speaker-independent, open vocabulary, and is characterized by large variation in caller speaking styles, telephone types and acoustic environments. Our results show that context-independent hidden Markov models trained discriminatively with the minimum classification error criterion are a practical alternative to context-dependent models based on phonetic decision trees, yielding better performance with a much smaller number of parameters. On this difficult task we have obtained 59% correct family name recognition. A phoneme-based confidence measure enables us to obtain 85% correct name recognition for accepted utterances, at an overall utterance acceptance rate of 15%.

Today’s automated telephone services generally use recorded speech from one speaker for all output. In applications with large and varying output vocabularies, such as place names, it may be necessary to employ a second speaker to provide new vocabulary items if the original speaker is not available, or to use text-tospeech (TTS) synthesis for the whole or parts of the output. This paper reports a comparison of 10 schemes for the generation of spoken output in a travel information service, ranging from natural speech from a single speaker, through combinations of different voices and of natural and synthetic speech, to TTS synthesis throughout. The results show strong preferences for concatenated speech over TTS and for professionalquality recordings over amateur ones, and a weaker preference for a single speaker over two speakers.

In this paper we propose a novel approach to the detection of acoustic irregular signals using minimum detection error (MDE) training. The MDE training is based on the generalized probabilistic descent method, which was originally developed as a general concept for a discriminative pattern recognizer design. We demonstrate its fundamental utility by experiments in which several acoustic events are detected in a noisy environment.

This paper provides a comprehensive introduction to a novel approach to pattern recognition which is based on the generalized probabilistic descent method (GPD) and its related design algorithms. The paper contains a survey of recent recognizer design techniques, the formulation of GPD, the concept of minimum classification error learning that is closely related to the GPD formalization, a relational analysis between GPD and other important design methods, and various embodiments of GPD-based design, including segmental-GPD, minimum spotting error training, discriminative utterance verification, and discriminative feature extraction. GPD development has its origins in basic pattern recognition and Bayes decision theory. It represents a simple but careful re-investigation of the classical theory and successfully leads to an innovative framework. For clarity of presentation, detailed discussions about its embodiments are provided for examples of speech pattern recognition tasks that use a distance-based classifier. Experimental results in speech pattern recognition tasks clearly demonstrate the remarkable utility of the family of GPD-based design algorithms.

This paper investigates the realization of optimal filter bank-based cepstral parameters. The framework is the discriminative feature extraction method (DFE) which iteratively estimates the filter-bank parameters according to the errors that the system makes. Various parameters of the filter-bank, such as center frequency, bandwidth, and gain are optimized using a string-level optimization and a frame-level optimization scheme. Application to vowel and noisy telephone speech recognition tasks shows that the DFE method realizes a more robust classifier by appropriate feature extraction.

We apply discriminative metric design (DMD), the general methodology of discriminative class-feature design, to a speech recognizer using a hidden Markov model (HMM) classification. This implementation enables one to represent the salient feature of each acoustic unit that is essential for recognition decision, and accordingly enhances robustness against irrelevant pattern variations. We demonstrate its high utility by experiments of speaker-dependent Japanese word recognition using linear feature extractors and mixture Gaussian HMMs. Furthermore, we summarize several other proposed design methods related to our DMD and show that they are special implementations of the DMD concept.

We propose a simple but powerful method for normalizing various sources of mismatch between training and testing conditions in speech recognizers, based on a training methodology called the generalized probabilistic descent method (GPD). In this new framework, a gradient based method is used to adapt the parameters of the feature extraction process in order to minimize the distortion between new speech data and existing classifier models, while most conventional normalization/adaptation methods attempt to adapt classification parameters. The GPD was proposed as a general discriminative training method for pattern recognizers such as neural networks. Up until now this has been used only for classifier design, sometimes in combination with the design of a non adaptive feature extractor. This paper, in contrast, studies the adaptive training benefits of GPD in the framework of normalizing the feature extractor to a new pattern environment. Experiments which use this technique to improve Japanese vowel classification were conducted and demonstrate the ability to reduce error rates by as much as 40%.

In this paper, we present results for the Minimum Classi cation Error (MCE) [1] framework for discriminative training applied to tasks in continuous phoneme recognition. The results obtained using MCE are compared with results for Maximum Likelihood Estimation (MLE). We examine the ability of MCE to attain high recognition performance with a small number of parameters. Phoneme-level and string-level MCE loss functions were used as the optimization criteria for a PrototypeBased Minimum Error Classi er (PBMEC) [2] and an HMM [3]. The former was optimized using Generalized Probabilistic Descent, the latter was optimized using an approximated second order method, the Quickprop algorithm. Two databases were used in this evaluation: 1) the ATR 5240 isolated word datasets for 6 speakers, in both speaker-dependent and multi-speaker mode; 2) the TIMIT database. For both databases, MCE training yielded striking gains in performance and classi er compactness compared to MLE baselines. For instance, through MCE training, performance similar to that of the Maximum Likelihood Successive State Splitting algorithm (ML-SSS) [4] could be obtained with 20 times fewer parameters.

The minimum classification error/generalized probabilistic descent (MCE/GPD) framework has been applied to several recognizer frameworks, such as hidden Markov models, prototype based systems, and systems based on artificial neural networks. However, to our knowledge, the MCE/CPD framework has not yet been applied to a working online speech recognition system in a realistic application environment. We describe the application of MCE/GPD to a telephone-based multi-speaker speech recognition system that accepts spoken Japanese names and forwards calls to any of up to 400 staff members. Points of interest include the automatic collection and labeling of new training data and the use of MCE/GPD training to improve recognizer performance.

This paper proposes a new approach, named discriminative metric design (DMD), to pattern recognition. DMD optimizes discriminant functions with the minimum classification error/generalized probabilistic descent method (MCE/GPD) such that intrinsic features of each pattern class can be represented efficiently. Resulting metrics accordingly lead to robust recognizers. The DMD is quite general. Several existing methods, such as learning vector quantization and the continuous hidden Markov model, are defined as its special cases. The paper specially elaborates the DMD formulation for the quadratic discriminant function, and clearly demonstrates its utility in a speaker-independent Japanese vowel recognition task.

This paper investigates the realization of a filter bank model that achieves minimum classification error. A bank-of-filter feature extractor module is jointly optimized with the classifier 's parameters so as to minimize the errors occurring at the back-end classifier, in the framework of Minimum Classification Error /Generalized Probabilistic Descent Method (MCE/GPD). The method was first applied to readjusting various parameters of filter banks linearly spaced on the Mel-scale for the Japanese vowel recognition task. Analysis of the feature extraction process shows how those parts of the spectrum that are relevant to discrimination are captured. Then the method was applied to a multi-speaker word recognition system, which resulted in an word error rate reduction of more than 20 %. 1. INTRODUCTION It has been argued that the interaction between signal representation and classification strongly influences speech recognition performance [2], [6]. However, the underlying nature of this...

A filter bank model, which achieves minimum error, is investigated in this paper. A bank-of-filter feature extractor module is comprehensively optimized with the classifier's parameters for minimization of the errors occurring at the back-end classifier. The method has been applied to readjusting Mel-scale and Bark-scale based filter banks for the Japanese vowel recognition task, the framework being provided by the minimum classification error (MCE)/generalised probabilistic descent method (GPD). The results show suggestive phenomena underlying the accuracy of the proposed approach.>

A hybrid speech recognition algorithm based on the combination of hidden Markov models (HMMs) and learning vector quantization (LVQ) is presented. The LVQ training algorithms are capable of producing highly discriminative reference vectors for classifying static patterns, i.e., vectors with a fixed dimension. The HMM formulation has also been successfully applied to the recognition of dynamic speech patterns that are of variable duration. It is shown that by combining both LVQ's discriminative power and the HMM's capability of modeling temporal variations of speech in a hybrid algorithm, the performance of the original HMM-based speech recognizer is significantly improved. For a highly confusable vocabulary consisting of the nine American English E-set letters used in a multispeaker, isolated-word test mode, the average word accuracy of the baseline HMM recognizer is 67%. When LVQ is incorporated in the hybrid system, the word accuracy increases to 83%.>

Although several kinds of discriminative training methods based on artificial neural networks have been vigorously tested, the pursuit of highly capable classification of variable-duration speech patterns has been unsatisfactory. In this light, the authors evaluate a generalized probabilistic descent method (GPD) in designing a speech recognizer incorporated with the dynamic time warping methodology. The algorithm can be viewed as generalized learning vector quantization suited to the dynamic programming-based time warping. Experiments were conducted on two tasks: English syllable classification and Japanese phoneme classification. Results clearly demonstrate that GPD can be a viable candidate for a method to realize a high-performance speech recognizer.>

It has since been shown that learning vector quantisation (LVQ) is a special case of a more general method, generalized probabilistic descent (GPD), for gradient descent on a rigorously defined classification loss measure that closely reflects the misclassification rate. The authors to extend LVQ into a prototype-based classifier appropriate for the classification of various long speech units. For word recognition, a dynamic time warping procedure is integrated into the GPD learning procedure. The resulting minimum error classifier (MEC) is no longer a purely LVQ-like method, and it is called the prototype-based minimum error classifier (PBMEC). Results for the difficult Bell Labs E-set task as well as for speaker-dependent isolated word recognition for a vocabulary of 5240 words are presented. They reveal clear gains in performance as a result of using PBMEC.>

An LVQ-HMM (learning vector quantization/hidden Markov model) hybrid algorithm was evaluated. For this evaluation, large vocabulary (5240 word) Japanese word recognition and Japanese phrase recognition were examined. Experiments in both a speaker-dependent and a speaker-independent mode were conducted. Comparison with conventional HMMs showed that LVQ-HMM improved recognition rates for words and phrases as well as for phonemes. In the speaker-dependent mode, LVQ-HMM yielded clear increases in word/phrase recognition accuracy; improvements in the recognition rates ranged between 0.8% and 4.3%. In the speaker-independent mode, however, increases in the word/phrase accuracy were small, but the results suggested some points for further study, e.g., an increase of 8% in recognition rate was achieved for one of the unknown (test) speakers, and LVQ-HMM seemed particularly effective in improving the performance for the test speakers on which conventional HMMs performed poorly.>

The authors present a novel HMM/LVQ (hidden Markov model/learning vector quantization) hybrid algorithm for connected word recognition (CWR). They show that, for combining both the discriminative power of LVQ and the capability of modeling temporal variations, of speech of an HMM into a hybrid algorithm, the performance of the original HMM-based speech recognition algorithm can be improved. The proposed hybrid algorithm is especially effective in cases when the training data are not adequate to characterize the test data. Preliminary results showed that this system gave a word accuracy of 98.5% on the whole TI test set, even when only HMM was used to segment speech utterances into words and states.>

A speech recognition system using the neurally inspired learning vector quantization (LVQ) to train hidden Markov model (HMM) codebooks is described. Both LVQ and HMMs are stochastic algorithms holding considerable promise for speech recognition. In particular, LVQ is a vector quantizer with very powerful classification ability. HMMs, on the other hand, have the advantage that phone models can easily be concatenated to produce long utterance models, such as word or sentence models. The algorithm described combines the advantages inherent in each of these two algorithms. As the result of phoneme recognition experiments using a large vocabulary database of 5240 common Japanese words uttered in isolation by a male speaker, it is confirmed that the high discriminant ability of LVQ could be integrated into an HMM architecture easily extendible to longer utterance models.>

The goal of this algorithm is to reduce learning time in a multireference phoneme-recognition system based on learning vector quantization (LVQ). The algorithm has, within the system, two kinds of vectors: a codebook vector and a reference node vector. An ordering procedure, similar to self-organizing feature maps, is used for the codebook design, and LVQ is used to adapt reference node vectors. The algorithm is divided into four steps: (1) codebook design, (2) mapping from the phoneme vector into the node vector, (3) reduction of the number of reference node vectors, and (4) adaptation of the reference node vectors. In particular, the mapping translates the high-dimensional phoneme vector into a trajectory on a two-dimensional plane; the acoustic feature dynamics can thus be visualized. Geometrical distance on the plane is used in the number reduction and adaptation of the reference node vectors. The learning time is thereby reduced considerably. Experiments using Japanese voiced plosives have shown that the algorithm can considerably speed up learning and still maintain a high, 97% recognition rate.>

A large-scale Japanese speech database is described. It consists of (1) an isolated-word speech database and (2) a continuous-speech database. To facilitate usage for speech research, multiple transcriptions have been made in five different layers from a simple phonemic description to fine acoustic-phonetic transcriptions. A database management system has also been developed to establish an effective link between speech data and label data for easy access to both data.>

The authors describe a shift-tolerant neural network architecture for phoneme recognition. The system is based on LVQ2, an algorithm which pays close attention to approximating the optimal Bayes decision line in a discrimination task. Recognition performances in the 98-99% correct range were obtained for LVQ2 networks aimed at speaker-dependent recognition of phonemes in small but ambiguous Japanese phonemic classes. A correct recognition rate of 97.7% was achieved by a single, larger LVQ2 network covering all Japanese consonants. These recognition results are at least as high as those obtained in the time delay neural network system and suggest that LVQ2 could be the basis for a successful speech recognition system.>