We describe a statistical approach for modeling dialogue acts in conversational speech, i.e., speech-act-like units such as STATEMENT, Question, BACKCHANNEL, Agreement, Disagreement, and Apology. Our model detects and predicts dialogue acts based on lexical, collocational, and prosodic cues, as well as on the discourse coherence of the dialogue act sequence. The dialogue model is based on treating the discourse structure of a conversation as a hidden Markov model and the individual dialogue acts as observations emanating from the model states. Constraints on the likely sequence of dialogue acts are modeled via a dialogue act n-gram. The statistical dialogue grammar is combined with word n-grams, decision trees, and neural networks modeling the idiosyncratic lexical and prosodic manifestations of each dialogue act. We develop a probabilistic integration of speech recognition with dialogue modeling, to improve both speech recognition and dialogue act classification accuracy. Models are trained and evaluated using a large hand-labeled database of 1,155 conversations from the Switchboard corpus of spontaneous human-to-human telephone speech. We achieved good dialogue act labeling accuracy (65% based on errorful, automatically recognized words and prosody, and 71% based on word transcripts, compared to a chance baseline accuracy of 35% and human accuracy of 84%) and a small reduction in word recognition error.

According to discourse theories in linguistics, conversational utterances possess an informational structure that partitions each sentence into two portions: a given and new. We explore this idea by building sub-sentence discourse language models for conversational speech recognition. The internal sentence structure is captured in statistical language modeling by training multiple n-gram models using the expectation-maximization algorithm on the Switchboard corpus. The resulting model contributes to a 30% reduction in language model perplexity and a small gain in word error rate.

Jeanrenaud et al. (1993) introduced the notion of event spotting and showed that the detection of events could be approached as a word spotting problem. The present authors first concentrate on the problem of spotting complex units such as grammatic events and describe the three issues that have to be addressed. These issues are: 1) hypothesizing-how to detect an event, 2) scoring-how to generate a consistent score for the event, 3) identification-what word sequence constitutes the particular realization of the event. The authors also discuss ways to evaluate the performance of an event spotter and how to characterize the complexity of an event. They then present two approaches to spot events. The first approach is based on posterior probability scoring using sub-grammars, and the second uses a large vocabulary recognizer. They show that both approaches have comparable performance on an event spotting task using the Switchboard corpus. In the posterior probability scoring approach, however, there is the advantage of being able to choose an operating point.>

A novel system for extracting information from stereotyped voice traffic is described. Off-the-air recordings of commercial air traffic control communications are interpreted in order to identify the flights present and determine the scenario (e.g., takeoff, landing) that they are following. The system combines algorithms from signal segmentation, speaker segregation, speech recognition, natural language parsing, and topic classification into a single system. Initial evaluation of the algorithm on data recorded at Dallas-Fort Worth airport yields performance of 68% detection of flights with 98% precision at an operating point where 76% of the flight identifications are correctly recognized. In tower recording containing both takeoff and landing scenarios, flights are correctly classified as takeoff or landing 94% of the time.>