In this paper the performance of different confidence measures used for an address recognition system are evaluated. The recognition system for cursive handwritten German address words is based on hidden Markov models (HMMs). It is essential that the structure of the address (name, street, city, country) is known, so that a specific small but complete dictionary can be selected. Upon choosing a wrong dictionary (OOV: out-of-vocabulary) or misrecognizing a word, the recognition result should be rejected by means of the confidence measure. This paper points out two aspects: the comparison of four confidence measures for single words - based on the likelihood, a garbage-model, a two-best recognition or a character decoding - and the comparison of using complete or wrong dictionaries. It is shown that the best confidence measure - the two-best distance - has a quite different behavior using OOV.

A scheme for handwriting adaptation for post offices is described to improve recognition performance of German addresses. The recognition system is based on a tied-mixture hidden Markov model, whose parameters are updated using the expectation maximization technique, the maximum likelihood linear regression algorithm and a new discriminative adaptation technique, the scaled likelihood linear regression. Contrary to the usual approach of adapting a writer-independent system to a specific writer we propose to adapt the system to the writer-independent data of a specific post office. The resulting system for each post office yields up to 16% lower word recognition errors.

The subject of this paper is the integration of the traditional vector quantizer (VQ) and discrete hidden Markov models (HMM) combination in the mixture emission density framework commonly used in automatic speech recognition (ASR). It is shown that the probability density of a system that consists of a VQ and a discrete classifier can be interpreted as a special case of a semi-continuous mixture model. Thus, the VQ parameters and the classifier can be trained jointly. In this framework, a gradient based VQ training method for single and multiple feature stream systems is derived. This leads to an approach that is directly related to the paradigm of maximum mutual information (MMI) neural networks, that has been successfully applied as VQ in ASR earlier. In continuous speech recognition experiments that were carried out for the Resource Management and Wall Street Journal databases the presented systems achieve recognition accuracies that compete well with comparable Gaussian mixture HMMs. Thus, we demonstrate that the performance degradations, often reported for discrete HMM systems, are not mainly caused by the vector quantization process in itself, but that they are due to the traditional separation of the VQ and the HMM during parameter estimation. These degradations can be avoided by training of the entire system as described here, while keeping the attractive computational speed of discrete HMMs.

This paper presents a method to improve the recognition rate of hybrid connectionist/HMM speech recognition systems. At the same time this approach allows the easy introduction of context dependent models in the hybrid framework. The approach is based on a standard hybrid connectionist/HMM recognizer, in which the neural nets are trained to estimate the a posteriori probabilities for all phones in each input frame. In the approach presented here, the probabilities of the neural nets are used to replace the codebook of a tied-mixture HMM system. Therefore the resulting system is called tied posterior. The advantages of this structure are that an arbitrary HMM-topology can be used, and that all context dependency and all clustering techniques used in tied-mixture systems can be applied to this hybrid speech recognition system. The approach has been evaluated on the Wall Street Journal (WSJ) database, with the result, that it outperforms the standard hybrid approach on this task.

Decision tree-based state clustering has emerged in as the most popular approach for clustering the states of context dependent hidden Markov model based speech recognizers. The application of sets of phones, mainly phonetically motivated, that limit the possible clusters, results in a reasonably good modeling of unseen phones while it still enables to model specific phones very precisely whenever this is necessary and enough training data is available. Formal concept analysis, a young mathematical discipline, provides means for the treatment of sets and sets of sets that are well suited for further improving tree-based state clustering. The possible refinements are outlined and evaluated in this paper. The major merit is the proposal of procedures for the adaptation of the number of sets used for clustering to the amount of available training data, and of a method that generates suitable sets automatically without the incorporation of additional knowledge.

This paper describes, how to perform speaker adaptation for a hybrid large vocabulary speech recognition system. The hybrid system is based on a Maximum Mutual Information Neural Network (MMINN), which is used as a Vector Quantizer (VQ) for a discrete HMM speech recognizer. The combination of MMINNs and HMMs has shown good performance on several large vocabulary speech recognition tasks like RM and WSJ. This paper now presents two approaches to perform speaker adaptation with this hybrid system. The first approach is a transformation of the feature space, which is performed by a neural network with maximum likelihood (ML) as objective function for the complete system, which means, that the parameters of the NN are estimated in order to match the HMM-parameters of the pre-trained speaker independent system. The second approach is to adapt the HMM parameters depending on the amount of training data available per HMM, using a regularization approach. Both approaches can be applied join...

We present a new adaptation technique for our hybrid large vocabulary continuous speech recognition system. In most adaptation approaches the HMM parameters are reestimated. In our approach, however, we train a speaker independent continuous speech recognizer, then we keep the HMM parameters fixed and we train a second network, which transforms the features of the adaptation data to fit the HMM parameters. Thus, less parameters have to be estimated, and therefore this approach performs well even for a small number of adaptation data. With this approach we achieve relative improvements in recognition rates on the Wall Street Journal (WSJ) task of 16.5%.

This paper describes, how to train Maximum Mutual Information Neural Networks (MMINN) in an efficient way, with a new topology. Large vocabulary speech recognition systems, based on a Hybrid MMI/connectionist HMM combination, have shown good performance on several tasks [1] and [2]. MMINNs are trained to maximize the mutual information between the index of the winning output neuron (Winner-Takes-All network) and the phonetical class of the corresponding acoustic frame. One major problem of MMI-neural networks is the high computational effort, which is needed for the training of the neural networks. The computational effort is proportional to the input and output size of the neural network and to the number of training samples. This paper shows two approaches, that demonstrate, how these long training times can be reduced with very low or even no loss in recognition accuracy. This is achieved by the use of phonetical knowledge, to build a network topology based on phonetical classes.

In this paper, we present the basic design principles and architecture of a dialogue system for scheduling appointments. This mixed-initiative dialogue system integrates an automatic speaker-independent speech recognition engine for continuously spoken German, a speech synthesizer and a scheduler database application to build up a scheduler that is purely driven by natural continuous speech and thus, does not need any visual display device. With these properties it is a prototype for a speech driven palm-size computer application and could be integrated in miniature computers that come along with no display device at all. 1. INTRODUCTION Dialogue systems enable the user to fulfill some well defined interaction with the machine by natural conversational speech in a spoken dialogue, in which the computer takes the part of one of the dialogue participants. The techniques and principles for the development of robust dialogue systems have attracted a lot of attention in the recent years. ...

This paper presents an extensive investigation of the use of trigraphs for online cursive handwriting recognition based on hidden Markov models (HMMs). Trigraphs are context dependent HMMs representing a single written character in its left and right context, similar to triphones in speech recognition. Looking at the great success of triphones in continuous speech recognition, it was always a challenging and open question, if the introduction of trigraphs could lead to substantially improved handwriting recognition systems. The results of this investigation are indeed extremely encouraging: the introduction of suitable trigraphs led to a 50% relative error reduction for a writer dependent 1000 word handwriting recognition system, and to a 35% relative error reduction for the same system with an extended 30000 word vocabulary for cursive handwriting recognition.

A hybrid MMI-connectionist/hidden Markov model (HMM) speech recognition system for the Wall Street Journal (WSJ) database is presented. The HMM part of this system uses discrete probability density functions (PDF). The neural network (NN) is used to replace a classical vector quantizer (VQ) like a k-means or LBG algorithm, which are typically used in discrete HMM systems. The NN is trained on an algorithm, that tries to achieve maximum mutual information (MMI) between the generated output labels and the underlying phonetic description. The system has been trained and tested with the five thousand word speaker independent WSJ task. The error rates of the MMI-connectionist approach are 21% lower than the error rates of a k-means system. The system achieves error rates which have been achieved before only by the best continuous/semi-continuous HMM speech recognizers, with the advantage of a faster recognition algorithm.

The estimation of the HMM parameters has always been a major issue in the design of speech recognition systems. Discriminative objectives like maximum mutual information (MMI) or minimum classification error (MCE) have proved to be superior over the common maximum likelihood estimation (MLE) in cases where a robust estimation of the probabilistic density functions (PDFs) is not possible. The determination of the overall likelihood of an acoustic observation is the most crucial point of the MMI-parameter estimation when applied to continuous speech systems. Contrary to the common approaches that estimate the overall likelihood of the training observations by evaluating the most confusing sentences or by applying global state frequencies, this paper suggests a dictionary analysis in order to get estimates for the dictionary-based risk of mixing two HMM states. These estimates are used to estimate the observations' likelihood and to control the discriminative MMI training procedure. Results on a monophone SCHMM speech recognition system are presented that prove the practicability of the new approach.

The paper presents the introduction of context dependent hidden Markov models for cursive, unconstrained handwriting recognition with large vocabularies. Since context dependent models were successfully introduced to speech recognition (R. Bahl et al., 1980; R. Schwartz et al., 1984; K. Lee, 1990), it seems obvious, that the use of trigraphs could also lead to improved online handwriting recognition systems (A. Kosmala et al., 1997). In analogy to triphones in speech recognition, trigraphs are context dependent sub word units representing a single written character in its left and right context. The tests were conducted on a writer dependent system with three different writers and two different vocabulary sizes (1000 words and 30000 words). The results we obtained with the trigaph based system compared to the monograph system, are very encouraging: a mean relative error reduction of 46% for the 1000 word handwriting recognition system and a mean relative error reduction of 37% for the same system with the 30000 word vocabulary. We believe that this represents one of the first systematic investigations of the influence of context dependent models and parameter reduction methods for a difficult large vocabulary handwriting recognition task.

In this paper we present a context dependent hybrid MMI-connectionist / Hidden Markov Model (HMM) speech recognition system for the Wall Street Journal (WSJ) database. The hybrid system is build with a neural network, which is used as a vector quantizer (VQ) and an HMM with discrete probablility density functions, which has the advantage of a faster decoding. The neural network is trained on an algorithm, that tries to maximize the mutual information between the classes of the input features (e.g. phones, triphones, etc.) and the neural firing sequence of the network.

We present a hybrid speech recognition system for speaker independent continuous speech recognition. The system combines a novel information theory based neural network (NN) paradigm and discrete Hidden Markov models (HMMs) including state-of-the-art techniques like state clustered triphones. The novel NN type is trained by an algorithm based on principles of self-organization that achieves maximum mutual information between the generated output labels and the basic phonetic classes. The structure of the hybrid system is quite similar to a classical VQ-HMM system but the vector quantizer (VQ) is replaced by the NN. To evaluate the system we use the speaker independent part of the resource management (RM) database. We obtained an important improvement by introducing a novel kind of context dependent basic classes used by the acoustic processor. The average RM recognition result with a word-pair grammar is now 95.2% what is significantly better than a classical VQ-system, slightly better than a different hybrid system with a recurrent network as probability estimator, and very close to the best continuous probability density function (PDF) HMM speech recognizers.

This paper presents the results of the comparison of continuous and discrete density hidden Markov models (HMMs) used for cursive handwriting recognition. For comparison, a subset of a large vocabulary (1000 word), writer-independent online handwriting recognition system for word and sentence recognition was used, which was developed at Duisburg University. This system has some unique features that are rarely found in other HMM-based character recognition systems, such as: (1) option between discrete, continuous, or hybrid modeling of HMM probability density distributions; (2) large vocabulary recognition based on either printed or cursive word or complete sentence input; (3) optimized HMM topology with an unusually large number of HMM states; and (4) use of multiple label streams for coding of handwritten information. Emphasis in this paper is on the comparison between continuous and discrete density HMMs, since this is still an open question in handwriting recognition, and is crucial for the future development of the system. However, in order to give a complete description of the basic system architecture, some of the above mentioned issues are also addressed. The surprising result of our investigation was the fact that discrete density models led to better results than continuous models, although this is generally not the case for HMM-based speech recognition systems. With the optimized system, a 70% word recognition rate was obtained for a challenging large-vocabulary, writer-independent sentence input task.

This paper presents a new hybrid system for speaker independent continuous speech recognition in a large vocabulary task. The hybrid system is a combination of context dependent discrete Hidden Markov Models and artificial neural networks that are trained by an information theory based algorithm. This algorithm maximizes the Mutual Information (MMI) between the network output and the phone descriptions by applying a self-organizing learning approach instead of forcing constrained network outputs. Recognition results have shown that the new hybrid system outperforms a classical k-means-VQ-based HMM-system. For the speaker independent DARPA Resource Management (RM) task (perplexity 60) we report a decrease in word recognition error rate up to 35% (close to the best continuous pdf systems). 1. INTRODUCTION Research during recent years has shown that Hidden Markov Models (HMMs) are the most successful approach for the recognition of speaker-independent continuous speech (e.g. see [1]). B...