This article describes the [corpus-name] corpus developed as part of the [project-name] project. To the best of our knowledge, this is one of the largest collections of children’s conversational speech that is freely available for non-commercial use under the creative commons license (CC BY-NC-SA 4.0). It comprises approximately 400 hours of speech, spanning some 230K utterances spread across about 10,500 virtual tutor sessions. Roughly 1,300 third, fourth and fifth grade students contributed to this corpus. The current release contains roughly 100K transcribed utterances. It is our hope that the corpus can be used to improve automatic speech recognition models and algorithms. We report the word error rate achieved on the test set using a model trained on the training and development portion of the corpus. The git repository of the corpus contains the complete training and evaluation setup in order to facilitate a fair and consistent evaluation. It is our hope that this corpus will contribute to the creation and evaluation of conversational AI agents having a better understanding of children’s speech, potentially opening doors to novel, effective, learning and therapeutic interventions.

We present a novel approach to synthesizing accurate visible speech based on searching and concatenating optimal variable-length units in a large corpus of motion capture data. Based on a set of visual prototypes selected on a source face and a corresponding set designated for a target face, we propose a machine learning technique to automatically map the facial motions observed on the source face to the target face. In order to model the long distance coarticulation effects in visible speech, a large-scale corpus that covers the most common syllables in English was collected, annotated and analyzed. For any input text, a search algorithm to locate the optimal sequences of concatenated units for synthesis is desrcribed. A new algorithm to adapt lip motions from a generic 3D face model to a specific 3D face model is also proposed. A complete, end-to-end visible speech animation system is implemented based on the approach. This system is currently used in more than 60 kindergarten through third grade classrooms to teach students to read using a lifelike conversational animated agent. To evaluate the quality of the visible speech produced by the animation system, both subjective evaluation and objective evaluation are conducted. The evaluation results show that the proposed approach is accurate and powerful for visible speech synthesis.

Abstract We present a technique for accurate automatic visible speech synthesis from textual input. When provided with a speech waveform and the text of a spoken sentence, the system produces accurate visible speech synchronized with the audio signal. To develop the system, we collected motion capture data from a speaker's face during production of a set of words containing all diviseme sequences in English. The motion capture points from the speaker's face are retargeted to the vertices of the polygons of a 3D face model. When synthesizing a new utterance, the system locates the required sequence of divisemes, shrinks or expands each diviseme based on the desired phoneme segment durations in the target utterance, then moves the polygons in the regions of the lips and lower face to correspond to the spatial coordinates of the motion capture data. The motion mapping is realized by a key‐shape mapping function learned by a set of viseme examples in the source and target faces. A well‐posed numerical algorithm estimates the shape blending coefficients. Time warping and motion vector blending at the juncture of two divisemes and the algorithm to search the optimal concatenated visible speech are also developed to provide the final concatenative motion sequence. Copyright © 2004 John Wiley & Sons, Ltd.

This article presents thoughts on the future of speech technology research, and a vision of the near future in which computer interaction is characterized by natural face-to-face conversations with lifelike characters that speak, emote and gesture. A first generation of these perceptive animated interfaces are now under development in a project called the Colorado Literacy Tutor, which uses perceptive animated agents in a computer-based literacy program

This paper presents a vision of the near future in which computer interaction is characterized by natural face-to-face conversations with lifelike characters that speak, emote, and gesture. These animated agents will converse with people much like people converse effectively with assistants in a variety of focused applications. Despite the research advances required to realize this vision, and the lack of strong experimental evidence that animated agents improve human-computer interaction, we argue that initial prototypes of perceptive animated interfaces can be developed today, and that the resulting systems will provide more effective and engaging communication experiences than existing systems. In support of this hypothesis, we first describe initial experiments using an animated character to teach speech and language skills to children with hearing problems, and classroom subjects and social skills to children with autistic spectrum disorder. We then show how existing dialogue system architectures can be transformed into perceptive animated interfaces by integrating computer vision and animation capabilities. We conclude by describing the Colorado Literacy Tutor, a computer-based literacy program that provides an ideal testbed for research and development of perceptive animated interfaces, and consider next steps required to realize the vision.

Detection of burst-related impulses, such as those accompanying plosive stop consonants, is an important problem for accurate measurement of acoustic features for recogntion (e.g., voice-onset-time) and for accurate automatic phonetic alignment. The proposed method of burst detection utilizes techniques for identifying and combining information about specific acoustic characteristics of bursts. One key element of the proposed method is the use of a measurement of intensity discrimination based on models from perceptual studies. Our experiments compared the proposed method of burst detection to the support vector machine (SVM) method, described below. The total error rate for the proposed method is 13.2 % on the test-set partition of the TIMIT corpus, compared to a total error rate of 24 % for the SVM method. 1.

We describe a corpus of children’s speech, called the OGI Kids’ Speech corpus, and a speaker- and vocabulary-independent recognition system trained and evaluated with these data. The corpus is composed of both prompted and spontaneous speech from 1100 children from kindergarten through grade 10. The prompted speech was presented as text appearing below an animated character (Baldi) that produced accurate visible speech synchronized with recorded prompts. The speech and text consists of isolated words, sentences, and digit strings. A phonetic recognizer was trained using an HMM/ANN framework, with training data taken from intervals of speech associated with phonetic segments in the isolated words in the corpus. Phonetic segments were derived using automatic phonetic alignment. To find out how well the recognizer is able to generalize to new words not found in the training set, we performed two test-set evaluations: one using a new set of utterances from the set of 205 words spoken in isolation (similar to the data used to train the recognizer) and one using words from the prompted sentences. Results were dramatically different (97.5 % for isolated vs. 37.9 % for words in sentences), and we explore methods that may be used to improve the recognizer’s ability to generalize to new words. 1.

In this paper, we argue for a paradigm shift in spoken language technology, from transcription tasks to interactive systems. The current paradigm evaluates speech recognition technology in terms of word recognition accuracy on large vocabulary transcription tasks, such as telephone conversations or media broadcasts. Systems are evaluated in international competitions, with strict rules for participation and well-defined evaluation metrics. Participation in these competitions is limited to a few elite laboratories that have the resources to develop and field systems. We propose a new, more productive and more accessible paradigm for spoken language research, in which research advances are evaluated in the context of interactive systems that allow people to perform useful tasks, such as accessing information from the World Wide Web, while driving a car. These systems are made available for daily use by ordinary citizens through telephone networks or placement in easily accessible kiosks in public institutions. It has previously been argued that this new paradigm, which focuses on the goal of universal access to information for all people, better serves the needs of the research community, as well as the welfare of our citizens. We discuss the challenges and rewards of an interactive system approach to spoken language research, and discuss our initial attempts to stimulate a paradigm shift and engage a large community of researchers through free distribution of the CSLU toolkit.

This paper describes the CSLU Speaker Recognition Corpus data collection. The corpus was motivated by a need for speech data from many speakers, under different environmental conditions, with each speaker providing data over a significant period of time. The corpus was designed to provide sufficient data to study phonetic variability within and across sessions, and to design and evaluate systems for both vocabulary independent and vocabulary specific recognition and verification tasks. The protocol includes fixed vocabulary phrases, digit strings, personal utterances (e.g., eye color), and fluent speech. The resulting Speaker Recognition Corpus is a collection of telephone speech recordings from over 500 participants collected over a two-year period. We describe the data collection procedure, the protocol, the transcription methods and the current status of the Speaker Recognition Corpus. 1. OVERVIEW Advances in speech technology require language resources to enable researchers to st...

This paper describes a set of experiments on neural-network training and search techniques that, when combined, have resulted in a 54% reduction in error on the continuous digits recognition task. The best system had word-level accuracy of 97.52% on a test set of the OGI 30K Numbers corpus, which contains naturally-produced continuous digit strings recorded over telephone channels. Experiments investigated effects of the feature set, the amount of data used for training, the type of context-dependent categories to be recognized, the values for duration limits, and the type of grammar. The experiments indicate that the grammar and duration limits had a greater effect on recognition accuracy than the output categories, cepstral features, or a 50% increase in the amount of training data. 1. INTRODUCTION The recognizers in the CSLU Toolkit use a hybrid HMM/ANN framework [1]. In these systems, frame-based recognition is done with context-dependent sub-phonetic states, where the state prob...

A set of freely available, universal speech tools is needed to accelerate progress in the speech technology. The CSLU Toolkit represents an effort to make the core technology and fundamental infrastructure accessible, affordable and easy to use. The CSLU Toolkit has been under development for five years. This paper describes recent improvements, additions and uses of the CSLU Toolkit. 1. INTRODUCTION Since 1993, the Center for Spoken Language Understanding (CSLU) has focused on incorporating state-of-the-art spokenlanguage technology into a portable, comprehensive and easyto -use software environment. The result of these efforts is the CSLU Toolkit. The toolkit integrates learning materials, authoring tools and core technologies such as speech recognition, text-to-speech synthesis, facial animation and speech reading. The toolkit is designed to support basic research, development and education activities related to spoken language systems and human-computer interfaces. What are our ...

In exploring new ways of looking at speech data, we have developed an alternative method of segmentation for training a neural-network-based digit-recognition system. Whereas previous methods segment the data into monophones, biphones, or triphones and train on each sub-phone unit in several broad-category contexts, our new method uses modified diphones to train on the regions of greatest spectral change as well as the regions of greatest stability. Although we account for regions of spectral stability, we do not require their presence in our word models. Empirical evidence for the advantage of this new method is seen by the 13% reduction in word-level error that was achieved on a test set of the OGI Numbers corpus. Comparison was made to a baseline system that used context-independent monophones and context-dependent biphones and triphones.

Neural network training targets for speech recognition are estimated using a novel method. Rather than use zero and one, continuous targets are generated using forward-backward probabilities. Each training pattern has more than one class active. Experiments showed that the new method effectively decreased the error rate by 15% in a continuous digits recognition task.

Currently, there are few opportunities for people to learn about and experiment with the latest spoken language technology. Furthermore, most research and development activities are restricted to a handful of academic and industrial labs. In order to make the technology less exclusive, it must become more accessible to the general population. This is now feasible with the development of the CSLU Toolkit which combines easy-to-use authoring tools with state-of-the-art human language technology. In this paper, we focus on the educational role of the toolkit and describe how it is being used in several local schools. 1. INTRODUCTION Research and development of spoken language systems is currently limited to relatively few academic and industrial laboratories. Because building such systems requires multidisciplinary expertise, sophisticated system development tools, language resources (e.g., pronunciation dictionaries), substantial computer resources and advanced technologies such as spee...

No abstract available.

Three perceptual experiments were conducted to test the relative importance of vowels vs. consonants to recognition of fluent speech. Sentences were selected from the TIMIT corpus to obtain approximately equal numbers of vowels and consonants within each sentence and equal durations across the set of sentences. In experiments 1 and 2, subjects listened to (a) unaltered TIMIT sentences; (b) sentences in which all of the vowels were replaced by noise; or (c) sentences in which all of the consonants were replaced by noise. The subjects listened to each sentence five times, and attempted to transcribe what they heard. The results of these experiments show that recognition of words depends more upon vowels than consonants-about twice as many words are recognized when vowels are retained in the speech. The effect was observed when occurrences of [1], [r], [w], [y] [m], [n], were included in the sentences (experiment 1) or replaced by noise (experiment 2). Experiment 3 tested the hypothesis that vowel boundaries contain more information about the neighboring consonants than vice versa.

The Spoken Dialogue Systems Laboratory at OGI gives students hands-on experience developing spoken dialogue systems (SDSs) in a rapid prototyping setting. The CSLU rapid prototyper (CSLUrp) allows students to quickly build and operate SDSs for dialogues of arbitrary complexity. CSLUrp consists of a graphical user interface that allows users to create SDSs with speech recognition, speech generation and arbitrary computation. When an application is designed, CSLUrp configures the system from the appropriate libraries in CSLUsh, the CSLU shell. The course explores CSLUrp in enough depth that students can craft and integrate their own low-level components (such as special purpose recognizers). In their final projects, students successfully built and demonstrated SDSs to do a variety of interesting tasks such as voice mail and directory assistance. Students found the course helpful and rated it highly.

We present a new approach to speaker dependent template generation which uses dramatically less storage to represent a speaker's words, with minimal degradation in recognition accuracy. In this approach, the symbolic string produced by a speaker-independent phonetic recognizer is used to represent utterances. We investigate effective procedures for template generation, and compare the results of these procedures to templates represented by acoustic parameters for utterances produced with different telephone handsets. The use of speaker-specific templates led to a reduction of about 1:500 in data-storage requirements with comparable recognition accuracy. We also compare recognition performance for speaker-specific and speaker-independent templates, and for combinations of the two. The results showed that combining speaker-specific and speaker-independent templates produces better recognition performance than either alone. A voice dialing system is described which incorporates the speaker-specific templates.

The Center for Spoken Language Understanding (CSLU) collects, annotates and distributes telephone speech data to enable research in spoken language understanding and automatic language identification. This paper gives a brief overview of recent activities in pursuit of this mission. We summarize corpus development activities at CSLU and describe new corpora useful for research on specific tasks: alphabet recognition, numbers recognition, large vocabulary word recognition, and yes/no recognition. We then discuss our two newest data collection efforts, Cellular Speech and the 22-Language Telephone Speech Corpus. All CSLU corpora are available at no charge to academic institutions.

We report on utterance verification of putative recognitions in both open-set and closed-set recognition tasks using telephone speech. For open-set recognition, we report on rejection of out-of-vocabulary utterances. In a two-keyword task ("male" and "female") using 50% out-ofvocabulary utterances, utterance verification reduced errors by 60%, from 12% to 4.8% compared to our baseline rejection strategy. For closed-set recognition, we report on re-ordering the N-best hypotheses. In a 58-phrase task, utterance verification reduced closed-set recognition errors by 30%, from 6.5% to 4.5%. 1. INTRODUCTION Recognition based on the combination of phonetic likelihoods from short fixed-width frames is the dominate paradigm for speech recognition systems. While this approach has numerous advantages, it is reasonable to think that better word-level recognition is possible using whole-word classifiers. Building such recognizers presents a number of difficulties, such as finding word boundaries ...

Current frame-based speech recognition systems sample speech at a fixed set of locations relative to each frame. Modeling the temporal dynamic behavior of speech is thereby complicated. This work shows that by explicitly using transitional information when extracting features, one can better model the acoustic phonetic structure, resulting in higher word level recognition performance. In this proposed approach, features representing local transitional information are used (a constant number of features are selected at each time frame, but the features are sampled near areas of greatest spectrum change within a relatively long window.) By explicitly modeling transitions in this way, we can also model local contextual information. Using this technique, the word level error rate decreased up to 30% on the databases we tested. 1. Introduction Current speech recognition systems can be categorized either as frame-based or segment-based [1]. Frame-based systems are currently more popular si...

A spoken language system combines speech recognition, natural language processing and human interface technology. It functions by recognizing the person's words, interpreting the sequence of words to obtain a meaning in terms of the application, and providing an appropriate response back to the user. Potential applications of spoken language systems range from simple tasks, such as retrieving information from an existing database (traffic reports, airline schedules), to interactive problem solving tasks involving complex planning and reasoning (travel planning, traffic routing), to support for multilingual interactions. We examine eight key areas in which basic research is needed to produce spoken language systems: (1) robust speech recognition; (2) automatic training and adaptation; (3) spontaneous speech; (4) dialogue models; (5) natural language response generation; (6) speech synthesis and speech generation; (7) multilingual systems; and (8) interactive multimodal systems. In each area, we identify key research challenges, the infrastructure needed to support research, and the expected benefits. We conclude by reviewing the need for multidisciplinary research, for development of shared corpora and related resources, for computational support and far rapid communication among researchers. The successful development of this technology will increase accessibility of computers to a wide range of users, will facilitate multinational communication and trade, and will create new research specialties and jobs in this rapidly expanding area.>

We describe a study to determine the feasibility of using automatic speech recognition to collect information over the telephone for the Year 2000 census in the United States. The stages of research are protocol development, speech data collection and transcription, system development, and system evaluation. We developed a rapid prototyping methodology to produce two protocols now being used to collect data from callers solicited by the bureau of the census. We have collected speech from over 1000 callers, with a goal of 3000 complete calls. Two hundred calls have been transcribed at the word level. Initial results are reported for a baseline system tested on the transcribed calls. The baseline system is both speaker and vocabulary independent-it has not been trained on words from the recognition vocabulary. The system's error rate ranges between 2% and 21% for responses to the different prompts. We describe research now underway intended to produce a system that is sufficiently accurate and graceful to demonstrate feasibility of a voice response questionnaire.>

There has been renewed interest in the field of automatic language identification over the past two years. The advent of a public-domain ten-language corpus of telephone speech has made the evaluation of different approaches to automatic language identification feasible. In an effort to provide benchmarks for evaluating machine performance, we conducted perceptual experiments on 1-, 2-, 4- and 6-second excerpts of telephone speech excised from spontaneous speech utterances in this corpus. The subject population consisted of 10 native speakers of English and 2 speakers from each of the remaining 9 languages. Statistical analyses of our results indicate that duration of the excerpt, familiarity with the language, and number of languages known are important factors affecting a subject's performance on the identification task.>

This paper describes a study conducted to determine the feasibility of using a spoken questionnaire to collect information for the Year 2000 Census in the USA. To refine the dialogue and to train recognizers, we collected complete protocols from over 4000 callers. For the responses labeled (about half), over 99 percent of the answers contain the desired information. The recognizers trained so far range in performance from 75 percent correct on year of birth to over 99 percent for marital status. We developed a prototype system that engages the callers in a dialogue to obtain the desired information, reviews the recognized information at the end of the call, and asks the caller to identify the response categories that are incorrect. 1. INTRODUCTION We have conducted a study to determine the feasibility of using an automated spoken questionnaire to collect information for the Year 2000 Census in the United States of America. The goal of the study was to develop and evaluate a telephone ...

This paper analyzes inter-labeler agreement of label choice and boundary placement for human phonetic transcriptions of continuous telephone speech in different languages. In experiment one, English, German, Mandarin and Spanish are labeled by fluent speakers of the languages. In experiment two, German and Hindi are labeled by linguists who do not speak the languages. Experiment two uses a somewhat finer phonetic transcription set than experiment one. We compare the transcriptions of the utterances in terms of the minimum number of substitutions, insertions and deletions needed to map one transcription to the other. Native speakers agree on the average 67.52% of the time at the finest level of labeling, including diacritics. Non-native linguists agree 34.41% of the time. The implications of the results are discussed for evaluation of phonetic recognition algorithms. 1. INTRODUCTION Phonetically transcribed continuous speech databases are important for understanding the phonological st...

A variety of approaches to language identification, based on (a) acoustic features, (b) broad-category segmentation, and (c) fine phonetic classification, are introduced. These approaches are evaluated in terms of their ability to distinguish between English and Japanese utterances spoken over a telephone channel. It is found that the best performance (86.3 % accurate classification of utterances with a mean length of 13.4 sec) is obtained when fine phonetic features are employed. In addition, the results show the importance of discriminatory training rather than likelihood estimation. 1. INTRODUCTION As developments in telecommunications and long-distance travel cause national borders to become increasingly transparent, the ability to identify which language is being spoken is growing in importance. The utility of tasks such as directory assistance or automatic translation is, for instance, improved substantially by the availability of a means of identifying which language is being s...

This report describes a telephone speech corpus collected at the Oregon Graduate Institute's Center for Spoken Language Understanding. Over four thousand people called in response to public requests. They were prompted by a recorded voice to say and spell their first and last names---with and without pauses, to say what city they grew up in and what city they were calling from, and to answer two yes/no questions. In order to collect sufficient instances of each letter, about 1000 callers also recited the alphabet. Each call is checked and transcribed by two people. In addition, a subset of the calls is being phonetically labeled. INTRODUCTION A successful telephone speech recognition system must deal with variations among speakers, telephone handsets, and channel characteristics. Modeling these sources of variability requires a corpus of speech samples provided by many speakers calling from different locations using different telephones. We describe the status of a telephone speech c...

The OGI Multi-language Telephone Speech Corpus is designed to support research on automatic language identification and multi-language speech recognition. The corpus consists of up to nine separate responses from each caller, ranging from single words to short topic-specific descriptions to 60 seconds of unconstrained spontaneous speech. The utterances were spoken over commercial telephone lines by speakers of English, Farsi (Persian), French, German, Japanese, Korean, Mandarin Chinese, Spanish, Tamil, and Vietnamese. We have completed the initial phase of our data acquisition effort: the recording and initial verification of utterances produced by 100 different speakers in each of the 10 languages. We describe the recording protocol, data collection procedure, ongoing corpus development, preliminary results of the statistical evaluation of the 10 languages, and plans to provide orthographic transcriptions of the speech. INTRODUCTION Research in multi-language recognition systems wou...

A system that recognizes names spelled with pauses between letters using high quality speech is described. The system uses neural network classifiers to locate and classify letters, then searches a database of names to find the best match to the letter scores. The directory name retrieval system was evaluated on 1020 names provided by 34 speakers who were not used to train the system. Using a database of 50000 names, 972, or 95.3%, were correctly identified as the first choice. Of the remaining 48 names, all but 10 were in the top three choices. 99% of letters were correctly located, although speakers failed to pause completely about 10% of the time. Classification of individual spoken letters that were correctly located was 93%.>

A segment-based approach to automatic language identification is discussed which is based on the idea that the acoustic structure of languages can be estimated by segmenting speech into broad phonetic categories. Automatic language identification can then be achieved by computing features that describe the phonetic and prosodic characteristics of the language, and using these feature measurements to train a classifier to distinguish between languages. As a first step in this approach, a multilanguage neural-network-based segmentation and broad classification algorithm using seven broad phonetic categories has been built. The algorithm was trained and tested on separate sets of speakers of American English, Japanese, Mandarin Chinese, and Tamil. It currently performs with an accuracy of 82.3% on the utterances of the test set.>

The ability of multilayer perceptrons (MLPs) trained with backpropagation to classify vowels excised from natural continuous speech is examined. Two spectral representations are compared: spectrograms and cochleagrams. The features used to train the MLPs include discrete Fourier transform (DFT) or cochleagram coefficients from a single frame in the middle of the vowel, or coefficients from each third of the vowel. The effects of estimates of pitch, duration, and the relative amplitude of the vowel were investigated. The experiments show that with coefficients alone, the cochleagram is superior to the spectrogram in classification performance for all experimental conditions. With the three additional features, however, the results are comparable. Perceptual experiments with trained human listeners on the same data revealed that MLPs perform much better than humans on vowels excised from context.>

A description is presented of EAR, an English alphabet recognizer that performs speaker-independent recognition of letters spoken in isolation. During recognition, (a) signal processing routines transform the digitized speech into useful representations, (b) rules are applied to the representations to locate segment boundaries, (c) feature measurements are computed on the speech segments, and (d) a neural network uses the feature measurements to classify the letter. The system was trained on one token of each letter from 120 speakers. Performance was 95% when tested on a new set of 30 speakers. Performance was 96% when tested on a second token of each letter from the original 120 speakers. The recognition accuracy is 6% higher than that of previously reported systems. The high level of performance is attributed to accurate and explicit phonetic segmentation, the use of speech knowledge to select features that measure the important linguistic information, and the ability of the neural classifier to model the variability of the data

Multilayer perceptrons and trained classification trees are two very different techniques which have recently become popular. Giving enough data and time, both methods are capable of performing arbitrary nonlinear classification. The two techniques have not previously been compared on real-world problems. The authors first consider the important differences between multilayer perceptrons and classification trees and conclude that there is not enough theoretical basis for the clear-cut superiority of one technique over the other. They then present results of a number of empirical tests on quite different problems in power system load forecasting and speaker-independent vowel identification. They compare the performance for classification and prediction in terms of accuracy outside the training set. In all cases, even with various sizes of training sets, the multilayer perceptron performed as well as or better than the trained classification trees. The authors are confident that the univariate version of the trained classification trees do not perform as well as the multilayer perceptron. More studies are needed, however, on the comparative performance of the linear combination version of the classification trees.>

Describes an algorithm that performs speaker-independent segmentation and broad classification of continuous speech. The algorithm is implemented as a set of knowledge sources that apply rules to speech parameters to locate segment boundaries and assign broad category labels to the resulting segments. The output of the algorithm is a network of segments with broad category labels. The structure of the algorithm, the manner in which acoustic phonetic knowledge is applied, and the performance of the algorithm on 200 utterances produced by 50 speakers is described.>

FEATURE is a speaker-independent isolated letter recognition system. The system performs a series of feature measurements on an input utterance, then classifies the sound as one of 26 English letters using statistical pattern classification techniques. Performance was evaluated for 10 male and 10 female speakers. For each speaker tested, the system was trained on 4 tokens of each letter provided by the remaining 19 speakers. The average error rate was 10.5%. The system can be used in either a speaker-independent or dynamic adaptation mode. In this latter mode, the user provides feedback when an error is made, and the system changes the statistical parameters that are used during classification. In this way, the system dynamically adapts to the speech patterns of the current user. The use of tuning produced a decrease in the error rate from 10.5% to 6.2%, averaged across the 20 speakers. FEATURE is significant because it is able to perform fine phonetic distinctions (such as between the letters B-D-E, P-T-G, V-Z, M-N, J-K, I-R) in a speaker- independent mode.

Template-based recognition systems overcome errors in the short-term matching process by comparing whole sequences of acoustic events. In many vocabularies, each word has a highly distinctive sequence. Some vocabularies have confusable words with very similar sequences, leading to poor recognition performance. Improvements in discriminability among similar words may be achieved by altering the matching algorithm, or by improving the reference template set. Both techniques are instances of multi-exemplar learning techniques which improve recognition performance through automatic evaluation of training data. This paper examines several such techniques using isolated utterances and highly ambiguous vocabularies (e.g., the "E" set; 3 B C D E G P V T Z) in a speaker-dependent recognition system. A system which combined both featural and template information led to the best performance for six out of eight speakers. Using this technique, E-set error rates improved from 37% to 10%.

This paper presents the results of several spectrogram reading experiments that were designed to determine the amount of phonetic information that is contained in the speech signal. The task involved identifying the phonetic content of an unknown utterance only from a visual examination of the spectrogram. In the first experiment, one of the authors attempted to phonetically label spectrograms of normal and anomalous English utterances as well as words in a known carrier phrase. The results, when compared with the transcriptions of three phoneticians who listened to the utterances, indicated an overall agreement of better than 85% on the sentences, and 93% for words in a carrier phrase. In the second and third experiments, we investigated the speed at which spectrogram reading can be accomplished. In the final experiment, five students read spectrograms of normal English sentences after a 13-week course in acoustic phonetics. Working as a group, the class agreed with the transcribers on over 80% of the segments. Implications for speech recognition and aids for the deaf are discussed.