Hidden Markov models (HMMs) are powerful generative models for sequential data that have been used in automatic speech recognition for more than two decades. Despite their popularity, HMMs make inaccurate assumptions about speech signals, thereby limiting the achievable performance of the conventional speech recognizer. Penalized logistic regression (PLR) is a well-founded discriminative classifier with long roots in the history of statistics. Its classification performance is often compared with that of the popular support vector machine (SVM). However, for speech classification, only limited success with PLR has been reported, partially due to the difficulty with sequential data. In this paper, we present an elegant way of incorporating HMMs in the PLR framework. This leads to a powerful discriminative classifier that naturally handles sequential data. In this approach, speech classification is done using affine combinations of HMM log-likelihoods. We believe that such combinations of HMMs lead to a more accurate classifier than the conventional HMM-based classifier. Unlike similar approaches, we jointly estimate the HMM parameters and the PLR parameters using a single training criterion. The extension to continuous speech recognition is done via rescoring of N-best lists or lattices.

State-of-the-art pattern recognition approaches like neural networks or kernel methods have only had limited success in speech recognition. The difficulties often encountered include the varying lengths of speech signals as well as how to deal with sequences of labels (e.g., digit strings) and unknown segmentation. In this paper we present a combined hidden Markov model (HMM) and penalized logistic regression machine (PLRM) approach to continuous speech recognition that can cope with both of these difficulties. The key ingredients of our approach are N-best rescoring and PLRM with garbage class. Experiments on the Aurora2 connected digits database show significant increase in recognition accuracy relative to a purely HMM-based system.

We propose a new approach to isolated-word speech recognition based on penalized logistic regression machines (PLRMs). With this approach we combine the hidden Markov model (HMM) with multiclass logistic regression resulting in a powerful speech recognizer which provides us with the posterior probability for each word. Experiments on the English E-set show significant improvements compared to conventional HMM-based speech recognition