Deepak Ramachandran, Mark Fanty, Ronald Provine, Peter Yeh, William Jarrold, Adwait Ratnaparkhi, Benjamin Douglas. Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2015.

A mapping between words and pronunciations (potential phonetic realizations) is a key component of speech recognition systems. Traditionally, this has been encoded in a lexicon where each pronunciation is transcribed by a linguist or generated by a grapheme-to-phoneme algorithm. For large vocabulary recognition systems, this process is highly susceptible to errors. We present an off-line data driven algorithm to correct suboptimal pronunciations using transcribed utterances. Unlike previous data driven algorithms that struggle to balance acoustic representation and multi-speaker generalization, our multi-pass approach maximizes both criteria, instead of compromising between the two. We demonstrate on an automated name dialing task that our multi-pass algorithm achieves a 70% error rate reduction when compared to a baseline grapheme-to-phoneme generated lexicon.

In this paper, we argue for a paradigm shift in spoken language technology, from transcription tasks to interactive systems. The current paradigm evaluates speech recognition technology in terms of word recognition accuracy on large vocabulary transcription tasks, such as telephone conversations or media broadcasts. Systems are evaluated in international competitions, with strict rules for participation and well-defined evaluation metrics. Participation in these competitions is limited to a few elite laboratories that have the resources to develop and field systems. We propose a new, more productive and more accessible paradigm for spoken language research, in which research advances are evaluated in the context of interactive systems that allow people to perform useful tasks, such as accessing information from the World Wide Web, while driving a car. These systems are made available for daily use by ordinary citizens through telephone networks or placement in easily accessible kiosks in public institutions. It has previously been argued that this new paradigm, which focuses on the goal of universal access to information for all people, better serves the needs of the research community, as well as the welfare of our citizens. We discuss the challenges and rewards of an interactive system approach to spoken language research, and discuss our initial attempts to stimulate a paradigm shift and engage a large community of researchers through free distribution of the CSLU toolkit.

Neural network training targets for speech recognition are estimated using a novel method. Rather than use zero and one, continuous targets are generated using forward-backward probabilities. Each training pattern has more than one class active. Experiments showed that the new method effectively decreased the error rate by 15% in a continuous digits recognition task.

Three perceptual experiments were conducted to test the relative importance of vowels vs. consonants to recognition of fluent speech. Sentences were selected from the TIMIT corpus to obtain approximately equal numbers of vowels and consonants within each sentence and equal durations across the set of sentences. In experiments 1 and 2, subjects listened to (a) unaltered TIMIT sentences; (b) sentences in which all of the vowels were replaced by noise; or (c) sentences in which all of the consonants were replaced by noise. The subjects listened to each sentence five times, and attempted to transcribe what they heard. The results of these experiments show that recognition of words depends more upon vowels than consonants-about twice as many words are recognized when vowels are retained in the speech. The effect was observed when occurrences of [1], [r], [w], [y] [m], [n], were included in the sentences (experiment 1) or replaced by noise (experiment 2). Experiment 3 tested the hypothesis that vowel boundaries contain more information about the neighboring consonants than vice versa.

We report on utterance verification of putative recognitions in both open-set and closed-set recognition tasks using telephone speech. For open-set recognition, we report on rejection of out-of-vocabulary utterances. In a two-keyword task ("male" and "female") using 50% out-ofvocabulary utterances, utterance verification reduced errors by 60%, from 12% to 4.8% compared to our baseline rejection strategy. For closed-set recognition, we report on re-ordering the N-best hypotheses. In a 58-phrase task, utterance verification reduced closed-set recognition errors by 30%, from 6.5% to 4.5%. 1. INTRODUCTION Recognition based on the combination of phonetic likelihoods from short fixed-width frames is the dominate paradigm for speech recognition systems. While this approach has numerous advantages, it is reasonable to think that better word-level recognition is possible using whole-word classifiers. Building such recognizers presents a number of difficulties, such as finding word boundaries ...

This paper describes a study conducted to determine the feasibility of using a spoken questionnaire to collect information for the Year 2000 Census in the USA. To refine the dialogue and to train recognizers, we collected complete protocols from over 4000 callers. For the responses labeled (about half), over 99 percent of the answers contain the desired information. The recognizers trained so far range in performance from 75 percent correct on year of birth to over 99 percent for marital status. We developed a prototype system that engages the callers in a dialogue to obtain the desired information, reviews the recognized information at the end of the call, and asks the caller to identify the response categories that are incorrect. 1. INTRODUCTION We have conducted a study to determine the feasibility of using an automated spoken questionnaire to collect information for the Year 2000 Census in the United States of America. The goal of the study was to develop and evaluate a telephone ...

This paper analyzes inter-labeler agreement of label choice and boundary placement for human phonetic transcriptions of continuous telephone speech in different languages. In experiment one, English, German, Mandarin and Spanish are labeled by fluent speakers of the languages. In experiment two, German and Hindi are labeled by linguists who do not speak the languages. Experiment two uses a somewhat finer phonetic transcription set than experiment one. We compare the transcriptions of the utterances in terms of the minimum number of substitutions, insertions and deletions needed to map one transcription to the other. Native speakers agree on the average 67.52% of the time at the finest level of labeling, including diacritics. Non-native linguists agree 34.41% of the time. The implications of the results are discussed for evaluation of phonetic recognition algorithms. 1. INTRODUCTION Phonetically transcribed continuous speech databases are important for understanding the phonological st...

This report describes a telephone speech corpus collected at the Oregon Graduate Institute's Center for Spoken Language Understanding. Over four thousand people called in response to public requests. They were prompted by a recorded voice to say and spell their first and last names---with and without pauses, to say what city they grew up in and what city they were calling from, and to answer two yes/no questions. In order to collect sufficient instances of each letter, about 1000 callers also recited the alphabet. Each call is checked and transcribed by two people. In addition, a subset of the calls is being phonetically labeled. INTRODUCTION A successful telephone speech recognition system must deal with variations among speakers, telephone handsets, and channel characteristics. Modeling these sources of variability requires a corpus of speech samples provided by many speakers calling from different locations using different telephones. We describe the status of a telephone speech c...

A system that recognizes names spelled with pauses between letters using high quality speech is described. The system uses neural network classifiers to locate and classify letters, then searches a database of names to find the best match to the letter scores. The directory name retrieval system was evaluated on 1020 names provided by 34 speakers who were not used to train the system. Using a database of 50000 names, 972, or 95.3%, were correctly identified as the first choice. Of the remaining 48 names, all but 10 were in the top three choices. 99% of letters were correctly located, although speakers failed to pause completely about 10% of the time. Classification of individual spoken letters that were correctly located was 93%.>

A description is presented of EAR, an English alphabet recognizer that performs speaker-independent recognition of letters spoken in isolation. During recognition, (a) signal processing routines transform the digitized speech into useful representations, (b) rules are applied to the representations to locate segment boundaries, (c) feature measurements are computed on the speech segments, and (d) a neural network uses the feature measurements to classify the letter. The system was trained on one token of each letter from 120 speakers. Performance was 95% when tested on a new set of 30 speakers. Performance was 96% when tested on a second token of each letter from the original 120 speakers. The recognition accuracy is 6% higher than that of previously reported systems. The high level of performance is attributed to accurate and explicit phonetic segmentation, the use of speech knowledge to select features that measure the important linguistic information, and the ability of the neural classifier to model the variability of the data