It is a common belief that repetitive users can adapt to a spoken dialog system. In this paper we describe a dialog design that allows experienced users to make their interactions with the system more efficient and present experimental evaluation of such adaptation. In our study we focused on the application of dialog systems as a tool for real-time data collection for healthcare. Specifically, we implemented a dialog system, Pain Monitoring Voice Diary, for monitoring chronic pain patients and conducted a usability study involving 171 dialog sessions with 24 users. Breakdown of the data according to the level of user experience indicates that experienced users adapt and take advantage of dialog design to make their interaction more efficient.

In this paper we address the problem of optimizing the decisions taken by a spoken dialog system based on a given business model. Although the model can be applied to other types of decisions, we refer to the choice on whether to continue a session or to escalate it to a human agent. We consider a family of business models based on costs and savings that are dependent on the decision taken and the automation outcome of each session. Based on a corpus of more than 61,000 dialogs, we show that an optimal classifier can be selected to minimize the cost for a given business model.

This papers reports on the development of spoken dialog system as a tool for real-time data collection for healthcare, life and behavioral science. Specifically, we implemented a dialog system, Pain Monitoring Voice Diary, for monitoring chronic pain patients. We discuss the requirements and characteristics of this application, specifically, the needs for adaptive level of user support and high and controllable accuracy of data capture, and show their implications for the design of Pain Monitoring Voice Diary. In usability study involving 118 dialog sessions with 24 volunteers we measured 98 % data capture and 80 % dialog efficiency, as estimated by percentage of task-oriented prompts. 1.

This paper focuses on the challenges that one encounters when building for commercial deployment an automated system for Directory Assistance (DA.) The design for an automated DA system needs to take into account constraints and requirements that arise from three distinct aspects of the application, namely, the business drivers, the user needs, and the strengths and weaknesses of voice technologies. 1.

The design and implementation of the AT&T Communicator mixed-initiative spoken dialog system is described.The Communicator project, sponsored by DARPA and launched in 1999, is a multi-year multi-site project on advanced spoken dialog systems research.The main focus of this paper is on issues related to the design of mixed-initiative systems.In addition to describing our architecture and implementation of the complex travel task, the paper reports on some preliminary evaluation results.

This paper describes our progress towards building a telephony-based spoken dialogue system for workshop/conference services. A mixed-initiative dialogue system has been developed that is engineered to o er users natural interaction with the system, ease-of-use and robustness towards ambiguous requests and machine errors. A prototype system, known as W99, is described in this paper

We propose a quantitative model for dialog systems that can be used for learning the dialog strategy. We claim that the problem of dialog design can be formalized as an optimization problem with an objective function reflecting different dialog dimensions relevant for a given application. We also show that any dialog system can be formally described as a sequential decision process in terms of its state space, action set, and strategy. With additional assumptions about the state transition probabilities and cost assignment, a dialog system can be mapped to a stochastic model known as Markov decision process (MDP). A variety of data driven algorithms for finding the optimal strategy (i.e., the one that optimizes the criterion) is available within the MDP framework, based on reinforcement learning. For an effective use of the available training data we propose a combination of supervised and reinforcement learning: the supervised learning is used to estimate a model of the user, i.e., the MDP parameters that quantify the user's behavior. Then a reinforcement learning algorithm is used to estimate the optimal strategy while the system interacts with the simulated user. This approach is tested for learning the strategy in an air travel information system (ATIS) task. The experimental results we present in this paper show that it is indeed possible to find a simple criterion, a state space representation, and a simulated user parameterization in order to automatically learn a relatively complex dialog behavior, similar to one that was heuristically designed by several research groups.

We introduce a stochastic model for dialogue systems based on Markov decision process. Within this framework we show that the problem of dialogue strategy design can be stated as an optimization problem, and solved by a variety of methods, including the reinforcement learning approach. The advantages of this new paradigm include objective evaluation of dialogue systems and their automatic design and adaptation. We show some preliminary results on learning a dialogue strategy for an air travel information system.

Recent progress in the field of spoken natural language understanding expanded the scope of spoken language systems to include mixed initiative dialogue. Currently there are no agreed upon theoretical foundations for the design of such systems. In this work we propose a stochastic model of computer-human interactions. This model can be used for learning and adaptation of the dialogue strategy and for objective evaluation. 1. INTRODUCTION Man-machine interactions, from simple touch tone menus to more complex speech based systems are ubiquitous in today's world. In the natural language and AI research communities there are even more complex examples of dialogue systems that make use of natural ways of communication like speech and language [1, 2, 3, 6]. In this paper we show that such dialogue systems can be formally described in terms of their state space, action set and strategy. The state of a dialogue system represents all the knowledge the system has at a certain time about interna...

The conventional n-gram language model exploits dependencies between words and their fixed-length past. This letter presents a model that represents sentences as a concatenation of variable-length sequences of units and describes an algorithm for unsupervised estimation of the model parameters. The approach is illustrated for the segmentation of sequences of letters into subword-like units. It is evaluated as a language model on a corpus of transcribed spoken sentences. Multigrams can provide a significantly lower test set perplexity than n-gram models.>

A method for measuring the capacity of learning machines is described. The method is based on fitting a theoretically derived function to empirical measurements of the maximal difference between the error rates on two separate data sets of varying sizes. Experimental measurements of the capacity of various types of linear classifiers are presented.

In this paper we discuss learning paradigms for the problem of understanding spoken language. The basic idea consists in redefining the language understanding problem in terms of translation between a natural language and a formal language that represents the meaning of sentences. Within this framework, with the assumption that input and output sentences can be put into sequential correspondence, understanding can be seen as a problem of sequential transduction. In this case several techniques exist for learning the corresponding transducers, some of which can be properly stated in terms of Hidden Markov modeling (conceptual HMMs). If the sequential assumption does not hold, there are new algorithms that also seem able to solve the learning problem. This view of a language understanding system opens new perspectives in the field of automatic learning of language models.

Recently, much interest has been generated regarding speech recognition systems based on Hidden Markov Models (HMMs) and neural network (NN) hybrids. Such systems attempt to combine the best features of both models: the temporal structure of HMMs and the discriminative power of neural networks. In this work we establish one more relation between the HMM and the NN paradigms by introducing the time-warping network (TWN) that is a generalization of both an HMM-based recognizer and a backpropagation net. The basic element of such a network, a time- warping neuron, extends the operation of the formal neuron of a backpropagation network by warping the input pattern to match it optimally to its weights. We show that a single-layer network of TW neurons is equivalent to a Gaussian density HMM-based recognition system. This equivalent neural representation suggests ways to improve the discriminative power of this system by using backpropagation discriminative training, and/or by generalizing the structure of the recognizer to a multi-layer net. The performance of the proposed network was evaluated on a highly confusable, isolated word, multi-speaker recognition task. The results indicate that not only does the recognition performance improve, but the separation between classes is enhanced, allowing us to set up a rejection criterion to improve the confidence of the system.

The application of neural networks to modeling time-invariant nonlinear systems has been difficult for complicated nonstationary signals, such as speech, because the networks are unable to characterize temporal variability. This problem is addressed by proposing a network architecture, called the hidden control neural network (HCNN), for modeling signals generated by nonlinear dynamical systems with restricted time variability. The mapping implemented by a multilayered neural network is allowed to change with time as a function of an additional control input signal. The network is trained using an algorithm based on ;backpropagation' and segmentation algorithms for estimating the unknown control together with the network's parameters. Application of the network to the segmentation and modeling of a signal produced by a time-varying nonlinear system, speaker-independent recognition of spoken connected digits, and online recognition of handwritten characters demonstrates the ability of the HCNN to learn time-varying nonlinear dynamics and its potential for high-performance recognition of signals produced by time-varying sources.

The authors extend the dynamic time warping (DTW) algorithm, widely used in automatic speech recognition (ASR), to a dynamic plane warping (DPW) algorithm, for application in the field of optical character recognition (OCR) or similar applications. Although direct application of the optimality principle reduced the computational complexity somewhat, the DPW (or image alignment) problem is exponential in the dimensions of the image. It is shown that by applying constraints to the image alignment problem, e.g., limiting the class of possible distortions, one can reduce the computational complexity dramatically, and find the optimal solution to the constrained problem in linear time. A statistical model, the planar hidden Markov model (PHMM), describing statistical properties of images is proposed. The PHMM approach was evaluated using a set of isolated handwritten digits. An overall digit recognition accuracy of 95% was achieved. It is expected that the advantage of this approach will be even more significant for harder tasks, such cursive-writing recognition and spotting.>

An understanding system, designed for both speech and text input, has been implemented based on statistical representation of task specific semantic knowledge. The core of the system is the conceptual decoder, which extracts the words and their association to the conceptual structure of the task directly from the acoustic signal. The conceptual information, which is also used to clarify the English sentences, is encoded following a statistical paradigm. A template generator and an SQL (structured query language) translator process the sentence and produce SQL code for querying a relational database. Results of the system on the official DARPA test are given.>

Neural networks are used to model nonlinear and time-varying systems. The proposed model attempts to cope with the time variability systems by adding an undetermined control input which modulates the mapping implemented by the network. The network architecture proposed, the hidden control neural network (HCNN), combines nonlinear prediction of conventional neural networks with hidden Markov modeling. This network is trained using an algorithm that is based on back-propagation and segmentation algorithms for estimating the unknown control together with the network's parameters. The HCNN approach is evaluated on multispeaker recognition of connected digits, yielding a word accuracy of 99.3%.>

A general statistical description of the problem of learning from examples is presented. Learning in layered networks is posed as a search in the network parameter space for a network that minimizes an additive error function of a statistically independent examples. By imposing the equivalence of the minimum error and the maximum likelihood criteria for training the network, the Gibbs distribution on the ensemble of networks with a fixed architecture is derived. The probability of correct prediction of a novel example can be expressed using the ensemble, serving as a measure to the network's generalization ability. The entropy of the prediction distribution is shown to be a consistent measure of the network's performance. The proposed formalism is applied to the problems of selecting an optimal architecture and the prediction of learning curves.>