Abstract The generation of referring expressions is a central topic in computational linguistics. Natural referring expressions – both definite references like ‘the baseball cap’ and pronouns like ‘it’ – are dependent on discourse context. We examine the practical implications of context-dependent referring expression generation for the design ofspoken systems. Currently, not all spoken systems have the goal of generating natural referring expressions. Many researchers believe that the context-dependency of natural referring expressions actually makes systemslessusable. Using the dual-task paradigm, we demonstrate that generating natural referring expressions that are dependent on discourse context reduces cognitive load. Somewhat surprisingly, we also demonstrate that practice does not improve cognitive load in systems that generate consistent (context-independent) referring expressions. We discuss practical implications for spoken systems as well as other areas of referring expression generation.

Agent-based modeling and simulation (ABMS) is of increasing interest for the analysis of complex systems consisting of a large number of subsystems interacting with each other in complex ways to accomplish their goals. The overall dynamic behavior of such a complex system typically emerges from the interactions among subsystems of the system. This paper describes a framework for modeling and simulating such complex systems. Of particular interest is developing a computational agent model of human operators interacting with other technical systems in a complex system. We present an example of developing and incorporating a human agent modeling system into an agent-based simulation system to support the design and evaluation of advanced air transportation concepts.

One way to model human behavior easily and accurately is to decompose a complex task into a set of primitive operations to 'which performance parameters may be assigned. This allows reuse of models at the task level by means of behavioral templates. Performance predictions generated from reusable templates were tested against data from an experiment on space shuttle procedures. The experiment used different participant populations (novice pilots vs. expert astronauts) in different workload conditions (single vs. multiple malfunctions). A phrase-reading template was found to predict performance for the different groups in the different conditions, and a screen-touching template was found to predict performance for astronauts, but there was not enough data to evaluate cross-group and cross-condition predictions. Templates appear to be a useful tool for making predictions of human performance.

Although computational modeling of human performance provides significant value in complex domains, its use has been mostly confined to simple human computer interaction (HCI) tasks due to difficulty in constructing models in such domains. The aim of this paper is to present the construction of a reactive human performance model from detailed cognitive task analysis in the apex cognitive architecture [Freed, M (1998)]. We show the construction of a sequence of predicted human behavior from elementary human performance templates and further validate the templates by comparing the task completion times to the human-in-the-loop simulation data obtained from the federal aviation administration.

A priori prediction of skilled human performance has the potential to be of great practical value but is difficult to carry out. This article reports on an approach that facilitates modeling of human behavior at the level of cognitive, perceptual, and motor operations, following the CPM-GOMS method (John, 1990). CPM-GOMS is a powerful modeling method that has remained underused because of the expertise and labor required. We describe a process for automatically generating CPM-GOMS models from a hierarchical task decomposition expressed in a computational modeling tool, taking advantage of reusable behavior templates and their efficacy for generating zero-parameter a priori predictions of complex human behavior. To demonstrate the process, we present a model of automated teller machine interaction. The model shows that it is possible to string together existing behavioral templates that compose basic HCI tasks, (e.g., mousing to a button and clicking on it) to generate powerful human performance predictions. Because interleaving of templates is now automated, it becomes possible to construct arbitrarily long sequences of behavior. In addition, the manipulation and adaptation of complete models has the potential of becoming dramatically easier. Thus, the tool described here provides an engine for CPM-GOMS that may facilitate computational modeling of human performance at the millisecond level.

Models of human performance typically focus on the mental components of task processing from discrete task trials. This approach neglects the advance planning of actions and overlapping of tasks characteristic of natural settings. The present research measures the joint timing of eye movements and manual responses in a typing-like task with the goal of extending models of discrete task performance to continuous domains. Following Pashler [1994] participants made separate choice responses to a series of five letters spread over a wide viewing area. Replicating Pashler's results, significant preview effects were found in both response time and eye movement data. Response to the first stimulus was delayed, with inter-response intervals for subsequent items rapid and flat across items. The eyes moved toward the next letter about 800 ms before the corresponding manual response (eye-hand span). Fixation dwell time was affected by stimulus luminance as well as difficulty of response mapping. The results suggest that fixation duration entails more than perceptual analyses. Implications of the results are discussed.

As speech interfaces become more prevalent, it i s becoming more crucial that they be developed in a way that minimizes cognitive load for users. One major barrier to creating systems that are more human-centered has been the lack of an accepted online methodology for directly evaluating the cognitive resource demands of different systems. The present study extends a classic tool from cognitive psychology, the dual-task paradigm, to speech interface evaluation. Participants follow simple instructions generated by a system, while simultaneously monitoring for a simple visual probe. Performance on the monitoring task is used as a measure of cognitive resource demands; whenever language understanding is more demanding, performance on the monitoring task suffers. In the present study we used this methodology to investigate patterns of reference generation and how they impact human understanding.

Previous studies for menu and Web search tasks have suggested differing advice on the optimal number of selections per page. In this article, we examine this discrepancy through the use of a computational model of information navigation that simulates users navigating through a Web site. By varying the quality of the link labels in our simulations, we find that the optimal structure depends on the quality of the labels and are thus able to account for the results in the previous studies. We present additional empirical results to further validate the model and corroborate our findings. Finally we discuss our findings' implications for the information architecture of Web sites.

CPM-GOMS is a modeling method that combines the task decomposition of a GOMS analysis with a model of human resource usage at the level of cognitive, perceptual, and motor operations. CPM-GOMS models have made accurate predictions about skilled user behavior in routine tasks, but developing such models is tedious and error-prone. We describe a process for automatically generating CPM-GOMS models from a hierarchical task decomposition expressed in a cognitive modeling tool called Apex. Resource scheduling in Apex automates the difficult task of interleaving the cognitive, perceptual, and motor resources underlying common task operators (e.g. mouse move-and-click). Apex’s UI automatically generates PERT charts, which allow modelers to visualize a model’s complex parallel behavior. Because interleaving and visualization is now automated, it is feasible to construct arbitrarily long sequences of behavior. To demonstrate the process, we present a model of automated teller interactions in Apex and discuss implications for user modeling.