In this paper, we focus on the problem of using sentence compression techniques to improve multi-document summarization.We propose an innovative sentence compression method by considering every node in the constituent parse tree and deciding its status -remove or retain.Integer liner programming with discriminative training is used to solve the problem.Under this model, we incorporate various constraints to improve the linguistic quality of the compressed sentences.Then we utilize a pipeline summarization framework where sentences are first compressed by our proposed compression model to obtain top-n candidates and then a sentence selection module is used to generate the final summary.Compared with state-ofthe-art algorithms, our model has similar ROUGE-2 scores but better linguistic quality on TAC data.

Joint compression and summarization has been used recently to generate high quality summaries.However, such word-based joint optimization is computationally expensive.In this paper we adopt the 'sentence compression + sentence selection' pipeline approach for compressive summarization, but propose to perform summary guided compression, rather than generic sentence-based compression.To create an annotated corpus, the human annotators were asked to compress sentences while explicitly given the important summary words in the sentences.Using this corpus, we train a supervised sentence compression model using a set of word-, syntax-, and documentlevel features.During summarization, we use multiple compressed sentences in the integer linear programming framework to select salient summary sentences.Our results on the TAC 2008 and 2011 summarization data sets show that by incorporating the guided sentence compression model, our summarization system can yield significant performance gain as compared to the state-of-the-art.

A finger-written, camera-based, hand gesture recognition framework for English letters in an in-vehicle environment based on Hidden Markov models is proposed. Due to the nature of the constrained hand-movement situations on the steering column, we are confronted with at least two challenging research issues, namely varying illumination conditions and noisy hand gestures. The first difficulty is alleviated by utilizing the contrast for background-foreground separation and skin model adaptation. We also adopt sub-letter stroke modeling to reduce the noisy frames of the beginning and ending parts of the letter gestures followed by the trajectory re-normalization. Moreover, the geometric relationship between letter pairs is also utilized to distinguish highly confusable letters. Finally, score fusion between whole-letter and sub-stroke models can be used to further improve the performance. When compared with the baseline system with simple features, our experimental results show that an overall relative error reduction of 66.03% can be achieved by integrating the above four new pieces of information.

We propose two very effective high-level binary-class features to enhance model-based skin color detection. First we find that the log likelihood ratio of the testing data between skin and non-skin RGB models can be a good discriminative feature. We also find that namely the background-foreground correlation provides another complementary feature compared to the conventional low-level RGB feature. Further improvement can be accomplished by Bayesian model adaptation and feature fusion. By jointly considering both schemes of Bayesian model adaptation and feature fusion, we attain the best system performance. Experimental results show that the proposed joint framework improves the 68% to 84% baseline F1scores to as high as almost 90% in a wide range of lighting conditions.

One of the main challenges in the development of robust dialog systems is to deal with noisy input due to imperfect results from any speech recognition module. A key step in addressing this noisy input is the computation of confidence for the portions so that the subsequent dialog modules can make use of the confidence scores to design corresponding dialog strategies. While past work in computing confidence scores have been focusing on recognized words, semantic slots, or utterances, this paper is extending the investigation on computing confidence scores for all phrases of a sentence in a dialog system setting. We demonstrated that using a Conditional Maximum Entropy (CME) classifier in combination with features in acoustic, syntactic, and semantic categories, we are able to obtain a high performance for the dialog system application in a restaurant finding domain, specifically, an annotation error rate of 5.1% is reached, which is a very good result for practical user.

Abstract We explore the relationship between question answering and constraint relaxation in spoken dialogue systems and develop dialogue strategies for selecting and presenting information succinctly. In particular, we describe methods for dealing with the results of database queries in information-seeking dialogues. Our goal is to structure the dialogue in such a way that the user is neither overwhelmed with information nor left uncertain as to how to refine the query further. We present two sets of evaluation results for a restaurant selection task: one is a system performance evaluation experiment involving twenty subjects, the other is an experimental evaluation of the use of suggestions involving sixteen subjects.

Recent developments in statistical modeling of various linguistic phenomena have shown that additional features give consistent performance improvements. Quite often, improvements are limited by the number of features a system is able to explore. This paper describes a novel progressive training algorithm that selects features from virtually unlimited feature spaces for conditional maximum entropy (CME) modeling. Experimental results in edit region identification demonstrate the benefits of the progressive feature selection (PFS) algorithm: the PFS algorithm maintains the same accuracy performance as previous CME feature selection algorithms (e.g., Zhou et al., 2003) when the same feature spaces are used. When additional features and their combinations are used, the PFS gives 17.66% relative improvement over the previously reported best result in edit region identification on Switchboard corpus (Kahn et al., 2005), which leads to a 20% relative error reduction in parsing the Switchboard corpus when gold edits are used as the upper bound.

In this paper we present a framework for managing information presentation in spoken dialogue systems. We describe a content optimization module that makes use of ontological relationships in information-seeking dialogues in order to organize knowledge base items and perform adjustments such as relaxing or tightening user constraints. We present the results of an experimental evaluation comparing two response strategies: (a) one that uses the content optimization module to offer suggestions and (b) one that gives no suggestions. The results indicate that giving such suggestions is preferred when a user query matches either no items or many items in the knowledge base, and may also lead to more efficient dialogues. Index Terms: spoken dialogue systems, content management 1.

Spoken dialogue interfaces, mostly command-and-control, become more visible in applications where attention needs to be shared with other tasks, such as driving a car. The deployment of the simple dialog systems, instead of more sophisticated ones, is partly because the computing platforms used for such tasks have been less powerful and partly because certain issues from these cognitively challenging tasks have not been well addressed even in the most advanced dialog systems. This paper reports the progress of our research effort in developing a robust, wide-coverage, and cognitive load-sensitive spoken dialog interface called CHAT: Conversational Helper for Automotive Tasks. Our research in the past few years has led to promising results, including high task completion rate, dialog efficiency, and improved user experience. Index Terms: dialog systems, cognitive load, robustness

Most existing statistical surface realizers either make use of hand-crafted grammars to provide coverage or are tuned to specific applications. This paper describes an initial effort toward building a statistical surface realization model that provides both precision and coverage. We trained a Maximum Entropy model that given a predicate-argument semantic representation, predicts the surface form for realizing a semantic concept and the ordering of sibling semantic concepts and their parent, on the Penn TreeBank and Proposition Bank corpora. Initial results have shown that the precisions for predicting surface forms and orderings reached 80% and 90% respectively, on a held-out part of Penn TreeBank. We use the model to generate sentences from our domain representations. We are in the process of evaluating the model on a corpus collected for our in-car applications.

Abstract This paper describes a data collection process aimed atgathering human-computer dialogs in high-stress or “busy”do-mains where the user is concentrating on tasks other than theconversation, for example, when driving a car. Designing spo-ken dialog interfaces for suchdomains is extremely challengingand the data collected will help us improve the dialog systemin-terfaceand performance,understandhowhumansperformthesetasks with respect to stressful situations, and obtain speech ut-terances for extracting prosodic features. This paper describesthe experimental design for collecting speech data in a simu-lated driving environment. 1. Background Research in human-computer interfaces has been carried outinapplications where the useris focusedon taskssuch as driving acar [4] or operating other machinery,with the goal of designinginterfaces that will help reduce the user’s overall cognitive load.In such applications, the user normally controls several devicessimultaneously. Existing applications maintain little or no dia-log context, and require the userto learn and remember compli-cated sets of device-specific commands. To overcome some ofthe shortcomings of such systems, researchers have been inves-tigating designing spokeninterface systems which can conversewith the user more naturally, allowing more flexibility in th euser’s speech and keeping track of the dialog context, similarto how a human speech partner would [6, 7]. However, human-humanspeechin suchscenariosis highly context-and situation-dependent,full of disfluencies(e.g., false starts and paus es)andsentence fragments (abandoned or repaired utterances), and ishighly interactive and collaborative. We believe that the easiestinterfaces to use will be those that mimic human-human inter-action in some, though perhaps not all, respects. Therefore,our data collection focuses on collecting the kind of speech thatwould occur between a human and a system that is as flexibleand capable as that user would desire.Our goal is a system should mimic human-human interac-tions by understanding the user’s requests and producing re-sponsesbased on the user’s knowledge, the conversationalcon-text, and the external situation. We use the car-driving domainas a testbed ofsucha dialog interface for operating in-carequip-ment, such as obtaining navigation information (e.g., turn-by-turn instructions) and information about local points of interest.Figure 1 illustrates the systemcomponents,which include a lan-guage understanding component,a response generator, a dialogmanager and a prosody classifier. We use off-the-shelf tech-nologies and tools for speechrecognition, speechsynthesis,andknowledge management.1.1. Purposes for Data CollectionThe ultimate goal of our dialog system is to enable natural in-teractions between the driver and the system to be like thosebetween humans. Therefore collecting human-human dialogsfor the above tasks helps us to develop and tune the system tosimulate such interactions. As the first step of our system de -velopment, data collection has the following specific purpo ses.Improve the system interface and performance: Languagecoverage has been a bottleneck for existing dialog systems. Arobust dialog system should allow the user to speak freely andbe ableto understandthe user’sintention expressedthroughvar-ious utterances. The robustness of a system can only be en-hanced using a large amount of data that are expected to covermost language phenomena in the target application. Thereforewe aim to collect dialogs from many subjects and to use thesedata to train the language understanding component. The datawill also provide evidence as to what features users would de-sire in an in-car conversationalsystem.Understand how humans give navigation instructions in adriving situation: Although human navigation data has beencollected for developing systems that automatically generatenavigation instructions, e.g. [1], the data is often written de-scriptions based on the subject’s mental recap of the route. Ina driving environment,humans might chooseto give navigationinformation differently with respect to the current position ofthe vehicle (e.g., close to a turn) and external situations (e.g.,emergency stop). There is a need, therefore, to collect new datato discover what kinds of strategies humans would use to con-vey navigation information in a real-time setting.Obtain speech utterances for extracting prosody features:Drivers are likely to produce disfluent and distracted speec hwith potentially complex syntax when focusing on tasks otherthan talking. Such data contain rich prosodic information thatcaptures variations in timing (e.g., lengthened sounds, pauses),intonation (e.g., pitch rise/fall at the end of utterance), and loud-ness. These features convey information beyond that carried bythe words themselves. They can help a dialog system detectutterance boundaries, driver intention and stress level, and sub-sequently generate appropriate responses which take into ac-count the driver’s emotional state. They can also help augmentthe information available to a natural language parser, to help

Spoken dialogue interfaces are gaining increased acceptance in a wide range of applications. Most current examples of such systems, however, rely on using restricted language and scripted dialogue interactions. We argue that speech interfaces in highly stressed or cognitively overloaded domains, i.e. those involving a user concentrating on other tasks, call for more flexible dialogue with robust, wide-coverage language understanding. We describe an initial effort at addressing flexible and rich dialogue in a system with a number of features, such as full spoken language understanding, a multithreaded dialogue manager, dynamic update of information, and recognition of partial proper names.

This article presents an approach for parsing natural language queries that integrates multiple subparsers and subgrammars, in contrast to the traditional single grammar and parser approach. In using LR(k) parsers for natural language processing, we are faced with the problem of rapid growth in parsing table sizes as the number of grammar rules increases. We propose to partition the grammar into multiple subgrammars, each having its own parsing table and parser. Grammar partitioning helps reduce the overall parsing table size when compared to using a single grammar. We used the GLR parser with an LR(1) parsing table in our framework because GLR parsers can handle ambiguity in natural language. A parser composition technique then combines the parsers' outputs to produce an overall parse that is the same as the output parse of single parser. Two different strategies were used for parser composition: (i) parser composition by cascading; and (ii) parser composition with predictive pruning.Our experiments were conducted with natural language queries from the ATIS (Air Travel Information Service) domain. We have manually translated the ATIS-3 corpora into Chinese, and consequently we could experiment with grammar partitioning on parallel linguistic corpora. For English, the unpartitioned ATIS grammar has 72,869 states in its parsing table, while the partitioned English grammar has 3,350 states in total. For Chinese, grammar partitioning reduced the overall parsing table size from 29,734 states to 3,894 states. Both results show that grammar partitioning greatly economizes on the overall parsing table size. Language understanding performances were also examined. Parser composition imparts a robust parsing capability in our framework, and hence obtains a higher understanding performance when compared to using a single GLR parser.

Natural language queries provide a natural means for common people to interact with computers and access to on-line information. Due to the complexity of natural language, the traditional way of using a single grammar for a single language parser leads to an inefficient, fragile, and often very large language processing system. Multi-Parser Architecture (MPA) intends to alleviate these problems, and the modularized MPA also has the advantage of easier portability to new domains and distributed computing. In this paper, we investigate the effect of using different types of parsers on different types of query data in MPA. Three data sets and two types of sub-parsers have been examined. Results show that partitioning grammars leads to superior speed performance for the Earley parser 1 across the three data sets. The GLR parser is faster than the Earley parser for the partitioned grammar, but the GLR parser may have excessive memory usage for the un-partitioned grammars. 1.

Variations in rate of speech (ROS) produce changes in both spectral features and word pronunciations that affect ASR systems. To cope with these effects, we propose to use rate specific phone models and pronunciations for ROS modeling at the word level. Words are given three types of pronunciations fast, slow, and medium-consisting of rate-specific phone models, respectively. This approach allows us to model within sentence rate variation. To better model coarticulation effects, we introduce the concept, of zero-length phones, which enables short phones to be skipped without having to change their neighboring phones' contexts. A data-driven approach is used to prune the pronunciation dictionary derived from rules for phone reduction. We tested these approaches on the Hub 4 database and achieved a relative improvement of 2.0% over the baseline-an evaluation-quality version of SRI's DECIPHER continuous speech recognition system-for clean native speech in the 1996 development set.

In large-vocabulary, multi-pass speech recognition systems, it is desirable to generate word lattices incorporating a large number of hypotheses while keeping the lattice sizes small. We describe two new techniques for reducing word lattice sizes without eliminating hypotheses. The first technique is an algorithm to reduce the size of non-deterministic bigram word lattices. The algorithm iteratively combines lattice nodes and transitions if local properties show that this does not change the set of allowed hypotheses. On bigram word lattices generated from Hub4 Broadcast News speech, it reduces lattice sizes by half on average. It was also found to produce smaller lattices than the standard finite state automaton determinization and minimization algorithms. The second technique is an improved algorithm for expanding lattices with trigram language models. Instead of giving all nodes a unique trigram context, this algorithm only creates unique contexts for trigrams that are explicitly represented in the model. Backed-off trigram probabilities are encoded without node duplication by factoring the probabilities into bigram probabilities and backoff weights. Experiments on Broadcast News show that this method reduces trigram lattice sizes by a factor of 6, and reduces expansion time by more than a factor of 10. Compared to conventionally expanded lattices, recognition with the compactly expanded lattices was also found to be 40 % faster, without affecting recognition accuracy. 1 1.

This paper describes our work in developing multilingual (Swedish and English) speech recognition systems in the ATIS domain. The acoustic component of the multilingual systems is realized through sharing Gaussian codebooks across Swedish and English allophones. The language model (LM) components are constructed by training a statistical bigram model, with a common backoff node, on bilingual texts, and by combining two monolingual LMs into a probabilistic finite state grammar. This system uses a single decoder for Swedish and English sentences, and is capable of recognizing sentences with words from both languages. Preliminary experiments show that sharing acoustic models across the two languages has not resulted in improved performance, while sharing a backoff node at the LM component provides flexibility and ease in recognizing bilingual sentences at the expense of a slight increase in word error rate in some cases. As a by-product, the bilingual decoder also achieves good performance on language identification (LID). 1.

This paper describes and evaluates a simple and general solution to the handling of compound nouns in Swedish and other languages in which compounds can be formed by concatenation of single words.The basic idea is to split compounds into their components and treat these components as recognition units equivalent to other words in the language model.By using a principled grammar-based language-processing architecture, it is then possible to accommodate input in split-compound format. 1

No abstract available.