Increasing amounts of public, corporate, and private audio data are available for use, but limited in usefulness by the lack of tools to permit their browsing and search. In this paper, we describe SCANMail, a system that employs automatic speech recognition, information retrieval, information extraction, and human computer interaction technology to permit users to browse and search their voicemail messages by content through a graphical user interface interface. The SCANMail client also provides note-taking capabilities as well as browsing and querying features. A CallerId server also proposes caller names from existing caller acoustic models and is trained from user feedback. An Email server sends the original message plus its transcription to a mailing address specified in the user's profile. 1.

This paper describes the design and evaluation of an automatic speech recognition (ASR) system on the Naval Research Laboratory Speech In Noise (SPINE) speech corpus. This corpus represents a task which involves human-human interaction on a constrained problem solving scenario under six di erent simulated noisy environments. Acoustic and language modeling were performed using a small dataset taken entirely from a subset of the acoustic environments. Speech recognition was performed on continuous conversations by detecting speech utterances, performing acoustic feature analysis and normalization, and adapting HMMmodels in multiple passes over each conversation-side. The ASR word accuracy (WAC) ranged from 77 percent in an o ce environment to 61 percent in conditions that include signi cant levels of background speech and noise.

In the frame of the INCO-Copernicus program of European Commission we have started to develop an audio-visual pronunciation teaching and training method and software system for hearing and speech-handicapped persons to help them to control their speech production. A teaching method is drawn up for progression from the individual sound preparation to practice of the sounds in sentences. The main aim is to develop an audio-visual articulation training and teaching system for all participant languages, these being English, Swedish, Slovenian and Hungarian. The basic part is a general language-independent measuring system and database editor. This database editor makes it possible to construct modules for all participant languages and for different speech disabilities. Two modules are under development for its construction in all languages, one of them being for teaching and training vowels for hearing-impaired children, while the other one is for correction of misarticulated fricative sounds.

Previous examinations of search in textual archives have assumed that users first retrieve a ranked set of documents relevant to their query, and then visually scan through these documents, to identify the information they seek.While document scanning is possible in text, it is much more laborious in speech archives, due to the inherently serial nature of speech.Yet, in developing tools for speech access, little attention has so far been paid to users' problems in scanning and extracting information from within "speech documents".We demonstrate the extent of these problems in two user studies.We show that users experience severe problems with local navigation in extracting relevant information from within "speech documents".Based on these results, we propose a new user interface (UI) design paradigm: What You See Is (Almost) What You Hear, (WYSIAWYH) -a multimodal method for accessing speech archives.This paradigm presents a visual analogue to the underlying speech, enabling visual scanning for effective local navigation.We empirically evaluate a UI based on this paradigm.We compare our WYSIAWYH UI with a visual "tape recorder", in relevance ranking, fact-finding, and summarization tasks involving broadcast news data.Our findings indicate that an interface supporting local navigation multimodally helps relevance ranking and fact-finding, but not summarization.We analyze the reasons for system success and identify outstanding research issues in UI design for speech archives.

This paper presents experimental results comparing two different designs for a spoken language interface to entail.We compare a mixed-initiative dialogue style, in which users can flexibly control the dialogue, to a systemiuitiative dialogue style, in which the system controls the dialogue.Our results show that even though the mixedinitiative system is more eflicient, as measured by number of turns, or elapsed time to complete a set of email tasks, users prefer the system-initiative interface.We posit that these preferences arise from the fact that the system initiative inter&e is easier to learn and more predictable.

We combine our earlier approach to context-dependent network representation with our algorithm for determining weighted networks to build optimized networks for large-vocabulary speech recognition combining an n-gram language model, a pronunciation dictionary and context-dependency modeling. While fully-expanded networks have been used before in restrictive settings (medium vocabulary or no cross-word contexts), we demonstrate that our network determination method makes it practical to use fully-expanded networks also in large-vocabulary recognition with full cross-word context modeling. For the DARPA North American Business News task (NAB), we give network sizes and recognition speeds and accuracies using bigram and trigram grammars with vocabulary sizes ranging from 10000 to 160000 words. With our construction, the fully-expanded NAB context-dependent networks contain only about twice as many arcs as the corresponding language models. Interestingly, we also find that, with these networks, real-time word accuracy is improved by increasing the vocabulary size and n-gram order.

This paper reports experimental results comparing a mixed-initiative to a system-initiative dialog strategy in the context of a personal voice email agent. To independently test the effects of dialog strategy and user expertise, users interact with either the system-initiative or the mixed-initiative agent to perform three successive tasks which are identical for both agents. We report performance comparisons across agent strategies as well as over tasks. This evaluation utilizes and tests the PARADISE evaluation framework, and discusses the performance function derivable from the experimental data.

A method of determining the similarity of nouns on the basis of a metric derived from the distribution of subject, verb and object in a large text corpus is described. The resulting quasi-semantic classification of nouns demonstrates the plausibility of the distributional hypothesis, and has potential application to a variety of tasks, including automatic indexing, resolving nominal compounds, and determining the scope of modification.

An effective procedure for automatically acquiring a new set of disambiguation rules for an existing deterministic parser on the basis of tagged text is presented. Performance of the automatically acquired rules is much better than the existing hand-written disambiguation rules. The success of the acquired rules depends on using the linguistic information encoded in the parser; enhancements to various components of the parser improves the acquired rule set. This work suggests a path toward more robust and comprehensive syntactic analyzers.

It is often remarked that natural language, used naturally, is unnaturally ungrammatical. *Spontaneous speech contains all manner of false starts, hesitations, and self-corrections that disrupt the well-formedness of strings. It is a mystery then, that despite this apparent wide deviation from grammatical norms, people have little difficulty understanding the non-fluent speech that is the essential medium of everyday life. And it is a still greater mystery that children can succeed in acquiring the grammar of a language on the basis of evidence provided by a mixed set of apparently grammatical and ungrammatical strings.

Linguists, including computational linguists, have always been fond of talking about trees. In this paper, we outline a theory of linguistic structure which talks about talking about trees; we call this theory Description theory (D-theory). While important issues must be resolved before a complete picture of D-theory emerges (and also before we can build programs which utilize it), we believe that this theory will ultimately provide a framework for explaining the syntax and semantics of natural language in a manner which is intrinsically computational. This paper will focus primarily on one set of motivations for this theory, those engendered by attempts to handle certain syntactic phenomena within the framework of deterministic parsing.

If natural language understanding systems are ever to cope with the full range of English language forms, their designers will have to incorporate a number of features of the spoken vernacular language. This communication discusses such features as non-standard grammatical rules, hesitations and false starts due to self-correction, systematic errors due to mismatches between the grammar and sentence generator, and uncorrected true errors.