A Playbook delegation approach was evaluated for human-autonomy teaming (HAT) in Single Pilot Operations (SPO). In SPO, a single pilot makes flight decisions, performs flight tasks, and collaborates with an autonomous teammate. The autonomous teammate shares responsibility, authority, and tasks. Challenges include the design of functions, interactions, and teaming skills. HAT often requires the ability to dynamically allocate functions, and timely methods to accurately express intent to teammates. A Playbook delegation interface was developed to enables the pilot to call and modify plays in collaboration with the autonomous teammate. Twenty pilots evaluated the Playbook interface to explore real-time function allocation, and identified teaming skills needed to support HAT. Pilots preferred the Playbook interface for better collaboration with the autonomous teammate. Interviews revealed that supporting human-like communication in HAT is critical to facilitate decision-making. Four major teaming skills (communication, coordination, cooperation, and cognition) are discussed to support HAT in SPO.

A proposed redesign of the shuttle cockpit display formats improves the correspondence between system summary displays and crew members' mental models of systems architecture and functional mode. We report the results of a part task simulation that assessed the impact of the redesigned displays on participants' ability to perform various steps in the process of diagnosing and recovering from systems malfunctions. Participants were airline pilots who received a modest amount of training on the tasks required of shuttle crews during nominal and off nominal ascents. With respect to fault management performance, both errors of omission and commission were reduced with the redesigned displays. Fault management errors were further categorized within a cognitive stage information processing framework. Error rates increased steadily from early to late stages of processing, but more so for the current displays than for the redesigned displays. We conclude that classifying and analyzing errors made by participants with relatively low levels of training provides a useful methodology for assessing and evaluating human centered design modifications to spacecraft displays.

One way to model human behavior easily and accurately is to decompose a complex task into a set of primitive operations to 'which performance parameters may be assigned. This allows reuse of models at the task level by means of behavioral templates. Performance predictions generated from reusable templates were tested against data from an experiment on space shuttle procedures. The experiment used different participant populations (novice pilots vs. expert astronauts) in different workload conditions (single vs. multiple malfunctions). A phrase-reading template was found to predict performance for the different groups in the different conditions, and a screen-touching template was found to predict performance for astronauts, but there was not enough data to evaluate cross-group and cross-condition predictions. Templates appear to be a useful tool for making predictions of human performance.

A priori prediction of skilled human performance has the potential to be of great practical value but is difficult to carry out. This article reports on an approach that facilitates modeling of human behavior at the level of cognitive, perceptual, and motor operations, following the CPM-GOMS method (John, 1990). CPM-GOMS is a powerful modeling method that has remained underused because of the expertise and labor required. We describe a process for automatically generating CPM-GOMS models from a hierarchical task decomposition expressed in a computational modeling tool, taking advantage of reusable behavior templates and their efficacy for generating zero-parameter a priori predictions of complex human behavior. To demonstrate the process, we present a model of automated teller machine interaction. The model shows that it is possible to string together existing behavioral templates that compose basic HCI tasks, (e.g., mousing to a button and clicking on it) to generate powerful human performance predictions. Because interleaving of templates is now automated, it becomes possible to construct arbitrarily long sequences of behavior. In addition, the manipulation and adaptation of complete models has the potential of becoming dramatically easier. Thus, the tool described here provides an engine for CPM-GOMS that may facilitate computational modeling of human performance at the millisecond level.

CPM-GOMS is a modeling method that combines the task decomposition of a GOMS analysis with a model of human resource usage at the level of cognitive, perceptual, and motor operations. CPM-GOMS models have made accurate predictions about skilled user behavior in routine tasks, but developing such models is tedious and error-prone. We describe a process for automatically generating CPM-GOMS models from a hierarchical task decomposition expressed in a cognitive modeling tool called Apex. Resource scheduling in Apex automates the difficult task of interleaving the cognitive, perceptual, and motor resources underlying common task operators (e.g. mouse move-and-click). Apex’s UI automatically generates PERT charts, which allow modelers to visualize a model’s complex parallel behavior. Because interleaving and visualization is now automated, it is feasible to construct arbitrarily long sequences of behavior. To demonstrate the process, we present a model of automated teller interactions in Apex and discuss implications for user modeling.

Click-down (or pull-down) menus have long been a key component of graphical user interfaces, yet we know surprisingly little about how users actually interact with such menus. Nilsens [8] study on menu selection has led to the development of a number of models of how users perform the task [6, 21. However, the validity of these models has not been empirically assessed with respect to eye movements (though [l] presents some interesting data that bear on these models). The present study is an attempt to provide data that can help refine our understanding of how users interact with such menus.

The ACT-R system is a general system for modeling a wide range of higher level cognitive processes. Recently, it has been embellished with a theory of how its higher level processes interact with a visual interface. This includes a theory of how visual attention can move across the screen, encoding information into a form that can be processed by ACT-R. This system is applied to modeling several classic phenomena in the literature that depend on the speed and selectivity with which visual attention can move across a visual display. ACT-R is capable of interacting with the same computer screens that subjects do and, as such, is well suited to provide a model for tasks involving human-computer interaction. In this article, we discuss a demonstration of ACT-R's application to menu selection and show that the ACT-R theory makes unique predictions, without estimating any parameters, about the time to search a menu. These predictions are confirmed.

This paper describes the structure and operation of SOUL, or Semantically-Oriented Understanding of Language. SOUL is a knowledge intensive reasoning system which is opportunistically used to provide a more thorough, fine grained analysis of an input utterance following its processing by a case-frame speech parser. The SOUL postprocessor relies upon extensive semantic and pragmatic knowledge to correct, reject and/or clarify the outputs of the CMU PHOENIX case-frame parser for speech and speech transcripts. We describe briefly both some of the SOUL is a general purpose, in-depth knowledge based reasoning system. It relies primarily upon pragmatic and semantic knowledge which is organized into a multi-layered frame base system of hierarchies. 1 It reasons by using abductive reasoning and constraint satisfaction techniques. The SOUL system is designed to perform thorough analyses of individual, spoken ut-terances. The in-depth reasoning component augments the basic