The primary purpose of this article is to determine the impact of a simulated agent's transparency on human performance and related variables, such as response time, workload, and trust calibration. The agent supports participants as they complete a base defense task by managing a team of heterogeneous unmanned vehicles and serves as a decision aid to the human. Three conditions of transparency are explored. In condition 1, the agent displays only the basic information (map of vehicle location and proposed routes). In condition 2, the agent displays the basic information and an explanation of its reasoning. In condition 3, the agent displays the basic information, reasoning, and uncertainties involved in the plans. Results show that participants exhibit better performance and trust calibration in the high-transparency conditions without perceiving a significant increase in workload. However, response time also increased, likely due to the additional processing time needed for conditions with more information. Overall, our findings indicate that the increased agent transparency can improve human-agent decision making and performance, but with a small cost of the efficiency (timeliness) of task completion.

Communication of the inner workings of an intelligent agent, or agent transparency, is important for human operators' calibration of their trust in the agent. Recent work has focused on methods of communicating this information in the context of agent-assisted mission planning. This investigation looked at the framing of agent communication and how it might interact with level of agent transparency. Results showed that critical framing improved operator's perception and trust of the agent regardless of the agent's transparency level. Further, findings suggest that the effects of framing are especially pronounced in low transparency applications.

Two human factors studies were conducted to assess the effectiveness of intelligent agents' user interfaces that were designed based on the Situation awareness-based Agent Transparency (SAT) model. Results show that agents' transparency (based on the SAT model) can benefit operator performance and support proper calibration of trust in the agents. Increasing levels of transparency enhanced operator's perceived trust in the agents, but only to a degree. When uncertainty was added to the interface, operator's trust did not further increase. Finally, the subjective workload data suggest that the benefits of increasing agent transparency do not have to be associated with higher levels of operator workload.

The purpose of the paper is to discuss human centered design implications for shared decision making between humans and autonomous systems in complex environments. Design implications are generated based on empirical results from two research paradigms. In the first paradigm, an intelligent agent (RoboLeader) supervised multiple subordinate systems and was in turn supervised by the human operator. The Robo Leader research varied number of subordinate units, task difficulty, agent reliability, type of agent errors, and partial autonomy. The second paradigm involved human interaction with partially and fully autonomous systems. Design implications from both paradigms are evaluated -- relating to multitasking, adaptive systems, false alarms, individual differences, operator trust, and allocation of human and agent tasks for partial autonomy. We conclude that mixed-initiative decision sharing depends on designing interfaces that support human-agent transparency.

We discuss the Situation awareness-based Agent Transparency (SAT) model and its three levels of information to support agent transparency and operator situation awareness of the agent: agent's current action or plan, agent's reasoning information, and agent's projections of future outcomes. Two studies -- Autonomous Squad Member and IMPACT -- are summarized to illustrate the utility of agent transparency for the overall human-agent team performance.

The human factors literature on intelligent systems was reviewed in relation to the following: efficient human supervision of multiple robots, appropriate human trust in the automated systems, maintenance of human operator's situation awareness, individual differences in human-agent (H-A) interaction, and retention of human decision authority. A number of approaches-from flexible automation to autonomous agents-were reviewed, and their advantages and disadvantages were discussed. In addition, two key human performance issues (trust and situation awareness) related to H-A teaming for multirobot control and some promising user interface design solutions to address these issues were discussed. Some major individual differences factors (operator spatial ability, attentional control ability, and gaming experience) were identified that may impact H-A teaming in the context of robotics control.

A military multitasking environment was simulated to examine the effects of unreliable automation on the performance of robotics operators. The main task was to manage a team of four ground robots with the assistance of RoboLeader, an intelligent agent capable of coordinating the robots and changing their routes based upon developments in the mission environment. RoboLeader's recommendations were manipulated to be either false-alarm prone or miss prone, with a reliability level of either 60% or 90%. The visual density of the targeting environment was manipulated by the presence or absence of friendly soldiers. Results showed that the type of RoboLeader unreliability (false-alarm vs. miss prone) affected operator's performance of tasks involving visual scanning (target detection, route editing, and situation awareness). There was a consistent effect of visual density for multiple performance measures. Participants with higher spatial ability performed better on the two tasks that required the most visual scanning (target detection and route editing). Participants' attentional control impacted their overall multitasking performance, especially during their execution of the secondary tasks (communication and gauge monitoring).

The purpose of this paper is to review research pertaining to the limitations and advantages of supervisory control for unmanned systems. We identify and discuss results showing technologies that mitigate the observed problems such as specialized interfaces, and adaptive systems. In the report, we first present an overview of definitions and important terms of supervisory control and human-agent teaming. We then discuss human performance issues in supervisory control of multiple robots with regard to operator multitasking performance, trust in automation, situation awareness, and operator workload. In the following sections, we review research findings for specific areas of supervisory control of multiple ground robots, aerial robots, and heterogeneous robots (using different types of robots in the same mission). In the last section, we review innovative techniques and technologies designed to enhance operator performance and reduce potential performance degradations identified in the literature.

We developed an intelligent agent, RoboLeader, that could assist human operators in route planning for a team of ground robots. We compared the operators' target detection performance in the 4-robot and 8-robot conditions. Results showed that the participants detected significantly less targets and had significantly worse situation awareness when there were 8 robots compared to the 4-robot condition. Those participants with higher spatial ability detected more targets than did those with lower spatial ability. Participants' self-assessed workload was affected by the number of robots under control, their gender, and their attentional control ability.

This paper investigates the impact of framing and time pressure on human judgment performance in a complex multiattribute judgment task. We focus on the decision process of human participants who must choose between pairwise alternatives in a resource-allocation task. We used the Analytic Hierarchy Process (AHP) to calculate the relative weights of the four alternatives (i.e.,${\rm C}_{1}$,${\rm C}_{2}$,${\rm C}_{3}$, and${\rm C}_{4}$) and the judgment consistency. Using the AHP, we examined two sets of hypotheses that address the impact of task conditions on the weight prioritization of choice alternatives and the internal consistency of the judgment behavior under varying task conditions. The experiment simulated the allocation of robotic assets across the battlefield to collect data about an enemy. Participants had to make a judgment about which asset to allocate to a new area by taking into account three criteria related to the likelihood of success. We manipulated the information frame and the nature of the task. We found that, in general, participants gave significantly different weights to the same alternatives under different frames and task conditions. Specifically, in terms of ln-transformed priority weights, participants gave significantly lower weights to${\rm C}_{2}$and${\rm C}_{4}$and higher weight to${\rm C}_{3}$under gain frame than under loss frame, and also, under different task conditions (i.e., Tasks #1, #2, and #3), participants gave significantly higher weight to${ \rm C}_{4}$in Task #1, lower weights to${\rm C}_{1}$and${\rm C}_{4}$, higher weight to${\rm C}_{3}$in Task #2, and lower weight to${\rm C}_{3}$in Task #3. Furthermore, we found that the internal consistency of the decision behavior was worse, first, in the loss frame than the gain frame and, second, under time pressure. Our methodology complements utility-theoretic frameworks by assessing judgment consistency without requiring the use of task-performance outcomes. This work is a step toward establishing a coherence criterion to investigate judgment under naturalistic conditions. The results will be useful for the design of multiattribute interfaces and decision aiding tools for real-time judgments in time-pressured task environments.

We simulated a military mounted environment and examined the performance of the combined position of gunner and robotics operator and how aided target recognition (AiTR) capabilities (delivered either through tactile or tactile + visual cueing) for the gunnery task might benefit the concurrent robotics and communication tasks. Results showed that participants' teleoperation task improved significantly when the AiTR was available to assist them with their gunnery task. However, the same improvement was not found for semi-autonomous robotics task performance. Additionally, when teleoperating, those participants with higher spatial ability outperformed those with lower spatial ability. However, performance gap between those with higher and lower spatial ability appeared to be narrower when the AiTR was available to assist the gunnery task. Participants' communication task performance also improved significantly when the gunnery task was aided by AiTR. Finally, participant's perceived workload was significantly higher when they teleoperated a robotic asset and when their gunnery task was unassisted.

In the future, it will become more common for humans to team up with robotic systems to perform tasks that humans cannot realistically accomplish alone. Even for autonomous and semiautonomous systems, teleoperation will be an important default mode. However, teleoperation can be a challenging task because the operator is remotely located. As a result, the operator's situation awareness of the remote environment can be compromised and the mission effectiveness can suffer. This paper presents a detailed examination of more than 150 papers covering human performance issues and suggested mitigation solutions. The paper summarizes the performance decrements caused by video images bandwidth, time lags, frame rates, lack of proprioception, frame of reference, two-dimensional views, attention switches, and motion effects. Suggested solutions and their limitations include stereoscopic displays, synthetic overlay, multimodal interfaces, and various predicative and decision support systems.

An approach to conduct systematic analysis of risk visualization strategies in a homeland missile defense context is presented. The approach consists of three phases which take different perspectives toward characterizing the problems of resource management and allocation for defense against missile attacks. Decision makers in missile defense are supervisory controllers of highly automated and resource-constrained systems. Our approach is suggested as a method to assess the effectiveness of visualization strategies in a dynamic decision environment. The three phases of our approach included a test for subject utility preferences, a text for subject multi-attribute prefences using the Analytic Hierarchy Process (AHP), and a test for subject action selection using a human-in-the-loop simulation test bed. Our approach is proposed to systematically evaluate the effectiveness of multistage decision making strategies against immediate and delayed threats. The alternatives were calculated based on expected cases of lives lost as a result of attacks on defended areas. Implementation of the test bed enabled us to investigate the effect of visualization formats on decision behaviour. Some preliminary results based on a pilot study will be presented. The implications of our study extend beyond missile defense. In a world where catastrophic loss of civilian lives is seen by some as a victory, those entrusted with homeland security must effectively manage limited resources to combat terrorist activities. Effective visualization formats will help to promote decision that are informed with respect to the risks and outcomes.