We introduce in this paper a hierarchical linear discounting class n-gram language modeling technique that has the advantage of combining several language models, trained at different nodes in a class hierarchy. The approach hierarchically clusters the word vocabulary into a word-tree. The closer a tree node is to the leaves, the more specific the corresponding word class is. The tree is used to balance generalization ability and word specificity when estimating the likelihood of an n-gram event. Experiments are conducted on Wall Street Journal corpus using a vocabulary of 20,000 words. Results show a reduction on the test perplexity over the standard n-gram approaches by 10%. We also report considerable improvement in the accuracy of the speech recognition task.

We investigate the possibility of predicting lost packets for packet loss concealment using n-gram predictive models. Unlike the conventional repetition-based algorithms, the proposed algorithm is based on the Shannon game, which serves as a principle for predicting the speech parameters of lost packets using the previously received parameters. During the training phase, we construct statistical backoff n-gram models. In the test phase, the models are used to predict the speech parameters of lost packets. Experiments were performed on a switchboard telephone speech database and the proposed algorithm is compared with the conventional repetition-based algorithm. The performance is evaluated in terms of the spectral distortion between the original and the predicted (or repeated) speech. The algorithm based on the back-off n-gram models reduces the spectral distortion by 8.7% over the conventional repetition-based algorithm for the first lost packet after receiving one. Further, it maintains about 6.2% improvement for up to six consecutive lost packets. In terms of perplexity of the predictive models, the backoff n-gram approach outperforms the repetition-based algorithm by 8.65%, which is very close to the improvement rate obtained from the spectral distortion measurement.

In this paper, we investigate the possibility of predicting lost packets for packet loss concealment using n-gram predictive models. Unlike the conventional repetition-based algorithms, the proposed algorithm is based on the Shannon game, which serves as a principle for predicting the speech parameters of lost packets using the previously received parameters. During training phase, we construct statistical backoff n-gram models. In test phase, the models are used to predict the speech parameters of lost packets. Experiments were performed on switchboard telephone speech database and the proposed algorithm is compared with the conventional repetitionbased algorithm. The performance is evaluated in terms of the spectral distortion between the original and the predicted (or repeated) speech. The algorithm based on the backoff n-gram models reduces the spectral distortion by 8.7% over the conventional repetition-based algorithm for the first lost packet after receiving one. Further it maintains about 6.2% improvement up to six consecutive lost packets. In terms of perplexity of the predictive models, backoff n-gram approach outperforms the repetition-based algorithm by 8.65%, which is very close to the improvement rate obtained from the spectral distortion measurement.

Enrollment is a necessary session in speaker verification. Automatic verification of collected training utterances, so called automatic enrollment, is critical to the performance of any speaker verification system. In this paper, we propose one solution including two separate approaches for automatic enrollment. First, at the utterance level, we propose to use an N-best algorithm to perform spoken content verification for training utterance selection. Second, at the word level, we employ an accurate utterance verification algorithm to conduct word verification for training data selection. Finally, we combine the two techniques together and propose a solution for automatic enrollment. Our experiments show that the N-best approach and the utterance verification approach can provide very low error rates. Based on the experimental results, we expect that the combined solution is close to error free and is feasible for real-world applications.

When automatic speech recognition (ASR) and speaker verification (SV) are applied in adverse acoustic environments, endpoint detection and energy normalization can be crucial to the functioning of both systems. In low signal-to-noise ratio (SNR) and nonstationary environments, conventional approaches to endpoint detection and energy normalization often fail and ASR performances usually degrade dramatically. The purpose of this paper is to address the endpoint problem. For ASR, we propose a real-time approach. It uses an optimal filter plus a three-state transition diagram for endpoint detection. The filter is designed utilizing several criteria to ensure accuracy and robustness. It has almost invariant response at various background noise levels. The detected endpoints are then applied to energy normalization sequentially. Evaluation results show that the proposed algorithm significantly reduces the string error rates in low SNR situations. The reduction rates even exceed 50% in several evaluated databases. For SV, we propose a batch-mode approach. It uses the optimal filter plus a two-mixture energy model for endpoint detection. The experiments show that the batch-mode algorithm can detect endpoints as accurately as using HMM forced alignment while the proposed one has much less computational complexity.

When automatic speech recognition (ASR) is applied to hands-free or other adverse acoustic environments, endpoint detection and energy normalization can be crucial to the entire system. In low signal-to-noise (SNR) situations, conventional approaches of endpointing and energy normalization often fail and ASR performances usually degrade dramatically. The goal of this paper is to find a fast, accurate, and robust endpointing algorithm for real-time ASR. We propose a novel approach of using a special filter plus a 3-state decision logic for endpoint detection. The filter has been designed under several criteria to ensure the accuracy and robustness of detection. The detected endpoints are then applied to energy normalization simultaneously. Evaluation results show that the proposed algorithm significantly reduce the string error rates on 7 out of 12 tested databases. The reduction rates even exceeded 50% on two of them. The algorithm only uses one-dimensional energy with 24-frame lookahead; therefore, it has a low complexity and is suitable for real-time ASR.

This paper describes a collaboration between Bell Labs and NHK (Japan Broadcasting Corp.) STRL to develop a real-time large vocabulary speech recognition system for live closed-captioning of NHK news programs. Bell Labs broadcast news recognition engine consists of a two-pass decoder using bigram language models (LM) and right biphone models during the first pass, and trigram LM with within-word triphone models in the second pass. Various pruning strategies are used to achieve real time decoding, together with a noise compensation procedure aimed at improving recognition on noisy segments of the program. The system operates in a real-time mode and delivers less than 2% of word error rate (WER) on studio news conditions and about 5% of WER on noisy news and reporter speech when evaluated on a real broadcast news program.

We present TclBLASR, a framework to integrate a proprietary speech recognition engine, an open source script language, such as Tcl/Tk and an open source sound analysis toolkit, such as Snack from KTH, into a user friendly platform that a user can write a Tcl/Tk script application quickly for speech recognition evaluation, speech data collection and automatic annotation, and speech technology demonstration. This framework is extremely useful for third party customer evaluation of speech technologies that do not involve heavy C/C++ program development and extensive knowledge on low-level speech engine APIs. Using the Bell Labs Automatic Speech Recognition (BLASR) engine, coupled with the realtime audio I/O and visualization provided by Snack and the flexible graphical user interface tools embedded in Tcl/Tk, the TclBLASR platform proves to be a useful framework for quick packaging of ASR engines for customer evaluation of the technology without extensive customization of interf aces to meet different needs from a wide range of customers.

ABSTRACT In this paper, a dialogue system for natural language based call steering is described and studied. The system is based on natural language speech recognition and understanding within a mixed initiative dialogue. The system is implemented on Bell Labs. Speech Technology Integration Platform (BLSTIP) using dialogue and natural language understanding components from BT laboratories. A prototype system in the operator service domain [2] is described. In order to improve the acoustic and language modeling for natural language based dialogue applications, various approaches are described and studied. The structure of the dialogue manager is also presented in which mixed-initiative dialogue can be supported with efficiency. Call classification and steering experiments were performed. The results confirm the efficacy of the proposed approach. 1. INTRODUCTION Natural language dialogue between human and machine is a challenge. In order to make a natural language based dialogue system successful, various efforts are made to improve the accuracy, flexibility and robustness of the system component technologies, such as speech recognition, speech understanding, dialogue generation and dialogue manager, text-to-speech synthesis, etc. Such a complex dialogue application imposes stringent requirements on the flexibility of the system platform. One of the drawbacks in systems deployed in the past is the limitation imposed by the finite state grammar on the language that a user can use to communicate with the machine. Although such constraint alleviates the complexity and problem in recognizing human speech, it becomes an obstacle to support more powerful, user friendly and flexible dialogue systems for mixed-initiative dialogues. In this paper, we study issues encountered in designing and implementing a natural language based call steering application for telephone service calls. This is a complicated application, and it performs a detailed diagnostic dialogue to identify the service problem, such as a troubled telephone line and etc., that the user is experiencing. It provides the desired service after receiving user’s consent and confirmation [2]. In the prototype system studied in this paper, the dialogue can go deep through many turns. The natural language based request and query from the user is recognized through natural language based automatic speech recognition. There is no constraint on the way that the user should communicate to the system. It allows the user to make direct requests as well as provide a description of the problem where the final action will be identified as the outcome of the dialogue. A call classifier provides natural language understanding based on the word string from the speech recognition output. The dialogue manager uses this understanding to determine the next appropriate system action. The organization of this paper is as follows. In Section 2, the dialogue system architecture and design are presented which support natural language based mixed-initiative dialogue applications such as call steering, movie locator, etc. Section 3 is devoted to natural language based speech recognition and statistical language modeling for dialogue applications. Section 4 is concentrated on the dialogue manager design and automatic query generation. Call classification and steering are studied in Section 5 and results are given based on a case study in a telephone service application.

This paper presents some recent enhancement to Bell Labs Speech Technology Integration Platform (BLSTIP), a common platform to integrate Bell Lab’s speech, telephony, Internet, and dialogue technologies for spoken and multi-modal dialogue system research and prototyping. Last year, we introduced BLSTIP to our partners as a speech technology platform for collaborative research and new application development. BLSTIP software is packaged as a single, network downloadable installation file. It supports a variety of speech applications such as natural language call routing/steering for call centers, natural language information system, messaging system with voice user interface, speaker verification, speech application trial and data collection, etc. As an enhancement to BLSTIP, we also designed a VoiceXML (Voice eXtensible Markup Language) integration infrastructure to study emerging web hosted speech applications such as voice portal, multimodal internet access, and wireless internet access.

In this paper, an approach to continuous speech recognition based on a layered self-adjusting decoding graph is described. It utilizes a scaffolding layer to support fast network expansion and releasing. A two level hashing structure is also described. It introduces self-adjusting capability in dynamic decoding on a general re-entrant decoding network. In stack decoding, the scaffolding layer in the proposed approach enables the decoder to look several layers into the future so that long span inter-word context dependency can be exactly preserved. Experimental results indicate that highly efficient decoding can be achieved with a significant savings on recognition resources.

We present a generic speech technology integration platform for application development and research across different domains. The goal of the design is two-fold: On the application development side, the system provides an intuitive developer’s interface defined by a high level application definition language and a set of convenient speech application building tools. It allows a novice developer to rapidly deploy and modify a spoken language dialogue application. On the system research and development side, the system uses a thin, ‘broker’ layer to separate the system application programming interface from the service provider interface. It makes the system easy to incorporate new technologies and new functional components. We also use a domain independent acoustic model set to cover US English phones for general speech applications. The system grammar and lexicon engine creates grammars and lexicon dictionaries on the fly to enable a practically unrestricted vocabulary for many recognition and synthesis applications.

In this paper, a wave decoder based on the general re-entrant network for continuous speech recognition is described.The decoder design is based on the concept of self-adjusting decoding graph in which the decoding network is expanded and released frame-synchronously.The fast network expansion and release are made possible by utilizing a novel dynamic network scaolding layer.The self-adjusting decoding graph is obtained by slicing the traditional decoding network horizontally for separation of dierent knowledge sources and vertically according to each time instant in search.A two l a y er hashing structure and an admissible arc predication scheme are described.These methods signicantly reduce the arc mortality rate, a problem which plagues the eciency of the dynamic decoder.Experimental results demonstrate that an order of magnitudereduction of decoding resources can be achieved based on the proposed approach.