This paper introduces a novel fan-out model that improves accuracy over previous models. The commonly used models rely on neglect time, the time an agent operates independently, which confounds both human and robot abilities. The proposed model separates neglect time into two functionally distinct concepts: the time a robot can operate self-sufficiently, and the time a human estimates the robot can do so. Previous research indicates fan-out is often overestimated. This work explains why robot ability provides an upper bound to fan-out, but that actual achieved fan-out is influenced by both the human and robot abilities. We conduct a study to validate this new model and show improved performance over the two most common fan-out models. The results show that both previous models overestimate as predicted. Using the new fan-out model, we show that as the difference between human estimation and robot abilities grows, the actual fan-out will fall further from the upper bound potential fan-out. By including assessments of both the robotic and human elements, the new model provides a more nuanced understanding of the dynamics at play and the factors involved in scaling Human Multi-Robot Teams.

This paper presents an inductive learning algorithm to predict the performance of hub-based swarms solving the best-of-N problem. Since a major constraint in learning swarm behavior is the high computational cost of obtaining sample data, it is desirable to ensure the right samples are used to train the models. The paper’s main contribution is formulating and comparing various sampling techniques to improve performance prediction using manageable amounts of training data. We compare random sampling with in-distribution sampling and out-of-distribution sampling, and then apply the lessons learned to modify random sampling to improve sampling. Results show that in-distribution sampling has the best F1 across different sampling techniques for classifying slow versus fast convergence. Model performance indicates that an informed combination of in-distribution and out-of-distribution sampling produces the highest classification accuracy of the swarm’s time-to-converge.

As the techniques of autonomous robots advance, there is an increasing demand for robots to provide explanations for their behavior. There are two commonly used explanation types. The first type emphasizes that a robot's policy is the best (or only) option that satisfies a specific property produced by its decision-making algorithms. The second explanation type is used when a robot fails and describes the cause of an error state that led to the failure. This paper proposes a new explanation type derived from a robot's proficiency self-assessment. The proposed explanation type not only supplements the first explanation type under typical operating conditions but also includes the second explanation type when the robot fails. The proposed explanation type is based on assumption-alignment tracking (AAT), a novel method for robot proficiency self-assessment. AAT provides three pieces of information for explanation generation: (1) assessment of assumptions veracity on which the robot's generators rely; (2) proficiency assessment measured by the probability that the robot will successfully accomplish its task; (3) counterfactual proficiency assessment computed by hypothetically varying assumptions. The information provided by AAT fits the situation awareness-based framework for explainable artificial intelligence. Examples of generated explanations are demonstrated using a simulated robot setting up a table with different blocks.

Proficiency self-assessment (PSA), the ability to assess how well one can carry out a task, is a desirable capability of autonomous robot systems. Prior work has proposed assumption-alignment tracking (AAT) for performing PSA, and has shown that it can accurately predict robot performance in real-time given a dataset obtained from both normal and abnormal training runs. Obtaining data in abnormal conditions (i.e., conditions in which the robot is not prepared to operate) is difficult and is often not possible. As a result, many realistic datasets contain very few data points for abnormal conditions, making it difficult to apply AAT. This paper hypothesizes that a one-class classifier can be built to detect anomalies using only data collected under normal conditions. Two metrics, difference and separation, are proposed and used to demonstrate that AAT feature vectors from different running conditions tend to form distinct clusters that are identifiable by mainstream one-class classification algorithms. Thus, one-class classifiers trained on AAT feature vectors from normal data can detect anomalous conditions. Furthermore, preliminary results suggest that a few abnormal data points, if available, can be used to classify the abnormality type and, in turn, the degree to which the anomalies will likely impact robot performance. Empirical results from both a simulated navigation robot and a Sawyer robot manipulating blocks show the efficacy of the approach.

Biological inspiration from honeybees, insects, and other animals has been used to create interesting implementations of multi-robot swarms. When the robots in a swarm are completely distributed, that is they lack any form of centralized control, the swarm acts as an agent-based model (ABM) wherein each agent implements its own controller and collective behavior emerges from the interactions between agents. Differential equation and graph-based models of some types of swarms have been used to guarantee collective behavior, but guaranteeing or predicting outcomes for hub-based agent colonies with finite numbers of robots remains an open problem. This paper presents a case study of designing an agent-based, hub-based swarm that solves the best-of-N problem with predictable success rates and completion times. The key innovation is modifying a tripartite graph formulation (TGF) from previous work so that it acts as a graph schema which abstracts an ABM into a simplified four state model, which in turn leads to a large discrete time Markov chain (DTMC) that describes how the collective state evolves over time. The DTMC can be used to compute success rates and completion times, which act as predictions for the ABM. Deviations between observed ABM outcomes and DTMC predictions lead to modifications in the ABM so that the swarm becomes more predictable.

A robot is proficient if its performance for its task(s) satisfies a specific standard. While the design of autonomous robots often emphasizes such proficiency, another important attribute of autonomous robot systems is their ability to evaluate their own proficiency. A robot should be able to conduct proficiency self-assessment (PSA), i.e. assess how well it can perform a task before, during, and after it has attempted the task. We propose the assumption-alignment tracking (AAT) method, which provides time-indexed assessments of the veracity of robot generators' assumptions, for designing autonomous robots that can effectively evaluate their own performance. AAT can be considered as a general framework for using robot sensory data to extract useful features, which are then used to build data-driven PSA models. We develop various AAT-based data-driven approaches to PSA from different perspectives. First, we use AAT for estimating robot performance. AAT features encode how the robot's current running condition varies from the normal condition, which correlates with the deviation level between the robot's current performance and normal performance. We use the k-nearest neighbor algorithm to model that correlation. Second, AAT features are used for anomaly detection. We treat anomaly detection as a one-class classification problem where only data from the robot operating in normal conditions are used in training, decreasing the burden on acquiring data in various abnormal conditions. The cluster boundary of data points from normal conditions, which serves as the decision boundary between normal and abnormal conditions, can be identified by mainstream one-class classification algorithms. Third, we improve PSA models that predict robot success/failure by introducing meta-PSA models that assess the correctness of PSA models. The probability that a PSA model's prediction is correct is conditioned on four features: 1) the mean distance from a test sample to its nearest neighbors in the training set; 2) the predicted probability of success made by the PSA model; 3) the ratio between the robot's current performance and its performance standard; and 4) the percentage of the task the robot has already completed. Meta-PSA models trained on the four features using a Random Forest algorithm improve PSA models with respect to both discriminability and calibration. Finally, we explore how AAT can be used to generate a new type of explanation of robot behavior/policy from the perspective of a robot's proficiency. AAT provides three pieces of information for explanation generation: (1) veracity assessment of the assumptions on which the robot's generators rely; (2) proficiency assessment measured by the probability that the robot will successfully accomplish its task; and (3) counterfactual proficiency assessment computed with the veracity of some assumptions varied hypothetically. The information provided by AAT fits the situation awareness-based framework for explainable artificial intelligence. The efficacy of AAT is comprehensively evaluated using robot systems with a variety of robot types, generators, hardware, and tasks, including a simulated robot navigating in a maze-based (discrete time) Markov chain environment, a simulated robot navigating in a continuous environment, and both a simulated and a real-world robot arranging blocks of different shapes and colors in a specific order on a table.

While the design of autonomous robots often emphasizes developing proficient robots, another important attribute of autonomous robot systems is their ability to evaluate their own proficiency and limitations. A robot should be able to assess how well it can perform a task before, during, and after it attempts the task. Thus, we consider the following question: How can we design autonomous robots that know their own limits? Toward this end, this paper presents an approach, called assumption-alignment tracking (AAT), for designing autonomous robots that can effectively evaluate their own limits. In AAT, the robot combines (a) measures of how well its decision-making algorithms align with its environment and hardware systems with (b) its past experiences to assess its ability to succeed at a given task. The effectiveness of AAT in assessing a robot's limits are illustrated in a robot navigation task.

Autonomous robot systems are often designed to achieve specific goals. This paper restricts attention to a specific type of goal, namely reaching a desired state within a certain time bound. For such goals, a robot system’s competency and resilience can be defined as the probability of reaching the desired state as a function of the time bound under a nominal unperturbed condition and under known perturbation conditions, respectively. Two metrics taken from prior work for measuring competency and resilience, power and efficiency, are modified so that they do not require subjective parameters. This paper formalizes the adapted metrics for discrete time Markov chains. The adapted metrics are applied to a best-of-N case study that is solved by a graph-based approach and modeled as a discrete time Markov chain. The case study demonstrates that the modified metrics allow power-efficiency trade-offs to be more easily visualized than the cluttered visualizations produced by the original metrics.

As development of robots with the ability to self-assess their proficiency for accomplishing tasks continues to grow, metrics are needed to evaluate the characteristics and performance of these robot systems and their interactions with humans. This proficiency-based human-robot interaction (HRI) use case can occur before, during, or after the performance of a task. This article presents a set of metrics for this use case, driven by a four-stage cyclical interaction flow: (1) robot self-assessment of proficiency (RSA), (2) robot communication of proficiency to the human (RCP), (3) human understanding of proficiency (HUP), and (4) robot perception of the human’s intentions, values, and assessments (RPH). This effort leverages work from related fields including explainability, transparency, and introspection, by repurposing metrics under the context of proficiency self-assessment. Considerations for temporal level (a priori, in situ, and post hoc) on the metrics are reviewed, as are the connections between metrics within or across stages in the proficiency-based interaction flow. This article provides a common framework and language for metrics to enhance the development and measurement of HRI in the field of proficiency self-assessment.

Measuring how well a potential solution to a problem matches the problem-holder's intent and detecting when a current solution no longer matches intent is important when designing resilient human-robot teams. This paper addresses intent-matching for a robot path-planning problem that includes multiple objectives and where human intent is represented as a vector in the multi-objective payoff space. The paper introduces a new metric called the intent threshold margin and shows that it can be used to rank paths by how close they match a specified intent. The rankings induced by the metric correlate with average human rankings (obtained in an MTurk study) of how closely different paths match a specified intent. The intuition of the intent threshold margin is that it represents how much the human's intent must be "relaxed" to match the payoffs for a specified path.

Algorithms used in networking, operation research and optimization can be created using bio-inspired swarm behaviors, but it is difficult to mimic swarm behaviors that generalize through diverse environments. State-machine-based artificial collective behaviors evolved by standard Grammatical Evolution (GE) provide promise for general swarm behaviors but may not scale to large problems. This paper introduces an algorithm that evolves problem-specific swarm behaviors by combining multi-agent grammatical evolution and Behavior Trees (BTs). We present a BT-based BNF grammar, supported by different fitness function types, which overcomes some of the limitations in using GEs to evolve swarm behavior. Given human-provided, problem-specific fitness-functions, the learned BT programs encode individual agent behaviors that produce desired swarm behaviors. We empirically verify the algorithm's effectiveness on three different problems: single-source foraging, collective transport, and nest maintenance. Agent diversity is key for the evolved behaviors to outperform hand-coded solutions in each task.

In human-swarm systems, human input to a robot swarm can both inhibit desirable swarm behaviors and allow the operator to properly guide the swarm to achieve mission goals. Indeed, the way that control is shared between the human operator and the inherent collective robot behaviors determines in large part the success of the human-swarm system. In this paper, we seek to understand how to design human-swarm systems that effectively moderate human influence over a robot swarm. To do this, we implement a simulated swarm system based on honeybees, and study how interacting with this swarm using various methods of moderating human influence impacts the success of the resulting human-swarm system. Our results demonstrate that moderating human influence is essential to achieving effective human-swarm systems, and highlight the need for future work in determining how to better moderate human influence in human-swarm systems.

A robotic swarm can perform various tasks. However, a human is required to task the swarm. Human control over the swarm can be enabled through a set of influential agents which can be either leaders or predators. In the presence of multiple tasks, the swarm may need to split into sub-swarms to accomplish the task and re-group as a swarm to execute larger tasks. The response of the swarm in the presence of influential agents depends on the swarm dynamics. A precise measure of influence using leaders or predators or a combination of leaders and predators to achieve the mission is not adequately studied. In this paper, we analyze the effect of using only leaders, only predators and a combination of leaders and predators on three swarm models namely, shepherding model, Couzin's model and a physicomimetic models while they perform foraging tasks and carry out Monte-Carlo simulations to evaluate the performance of the influential agents on different swarms. We also propose a novel way to split a swarm into smaller sub-swarms using influential agents. Our results show that the predator based swarm splitting and steering to a task based on shepherding model performs far better than any other combination of leaders and predators. This result is consistent even when the number of agents is increased to 500.

For goal-based robot navigation in a dynamic environment, human intent includes expectations about what performance objectives are satisfied by a planned path in terms of objectives to be met. If the planned path drifts from the human's intent as a result of environment changes, the path needs to be replanned. This paper presents a replanning framework with three elements: (a) the integration of fast online path-planning algorithms that generate trajectories conforming to the given intent; (b) a mathematical model that says when replanning must happen; and (c) an evaluation of events that trigger replanning. An interactive graphical user interface enables a human to accept or reject replanned paths when a trigger happens. A study of 50 MTurk participants is used to assess what replanning triggers best enable a human-robot collaboration to persistently satisfy intent?

Particle Swarm Optimization is an effective algorithm because of the combination of the stochastic behavior of particles and the swarm structure. Unfortunately these features also make it difficult to understand the dynamics of PSO. Common methods of analyzing PSO rely on simplifying the algorithm, e.g., assuming stagnation (a state where the swarm ceases finding better solutions) or treating the stochastic factors as constants. In this paper, we expand on earlier work to understand the dynamics of PSO which used input-to-state stability analysis. In particular, we decompose PSO more completely and use the properties of combinations of input-to-state stable components to model convergence at all levels up to and including the entire swarm. This approach allows us to conceptualize the swam as a leader-follower structure and analyze the swarm under a variety of conditions including various fitness functions.

Animals such as bees, ants, birds, fish, and others are able to perform complex coordinated tasks like foraging, nest-selection, flocking and escaping predators efficiently without centralized control or coordination. Conventionally, mimicking these behaviors with robots requires researchers to study actual behaviors, derive mathematical models, and implement these models as algorithms. We propose a distributed algorithm, Grammatical Evolution algorithm for Evolution of Swarm bEhaviors (GEESE), which uses genetic methods to generate collective behaviors for robot swarms. GEESE uses grammatical evolution to evolve a primitive set of human-provided rules into productive individual behaviors. The GEESE algorithm is evaluated in two different ways. First, GEESE is compared to state-of-the-art genetic algorithms on the canonical Santa Fe Trail problem. Results show that GEESE outperforms the state-of-the-art by (a) providing better solution quality given sufficient population size while (b) utilizing fewer evolutionary steps. Second, GEESE outperforms both a hand-coded and a Grammatical Evolution-generated solution on a collective swarm foraging task.

Researchers in the field of human-robot interaction (HRI) are celebrating the transition to an official ACM Transactions journal.It is a significant achievement for HRI.This field has faced many challenges, including scarce resources for conducting HRI research, the difficulty of conducting the research itself-integrating hardware and software; building robots that can interact with people; piecing together the intricacies of seeing, thinking, and acting when a person encounters a robot; understanding and shaping ethical norms and standards; and stepping behind the mythologies that have long governed cultural attitudes for and against robots.HRI are presented here.Even as HRI becomes more mature, the core disciplines of the field face many controversies and subtleties that complicate our research questions and activities.Here's an example: It was long thought that human beings can discriminate only the seven so-called universal emotions of surprise, happiness, sadness, anger, disgust, fear, and contempt in others' faces and, therefore, that modeling this ability in a robot would be soon possible.But now, psychologists have reported that people can make much finer-grained emotional distinctions, raising the bar for designing a robot.Advances in technology, machine learning, and data science also present higher standards for what a robot can do with, and for, people.What Do We Need to Learn About Humans?At its core, the science of HRI is a science of interaction, but we cannot assume that the state of being human is a known entity that does not change.We still need to learn more about the following:• the causes and manifestations of people's intentions, so that robots can ascertain them • trust, and what it takes in different cultures to develop and destroy trust

There is a growing need to develop effective interaction methods that enable a single operator to manage a team of multiple robots. This paper presents a novel approach that involves treating the team as a moldable volume, in which deformations of the volume correspond to changes in team shape. The team possesses a level of autonomy that allows the team to travel to and surround buildings of interest in a patrol and cordon scenario. During surround mode, the operator explores or manipulates the team shape to create desired formations around a building. A spacing interaction method also allows the operator to adjust how robots are spaced within the current shape. Separate haptic feedback is developed for each method to allow the operator to "feel" the shape or spacing manipulation. During travel mode, the operator chooses desired travel locations and receives feedback to help identify how and where the team travels. Results from a user study suggest that haptic feedback significantly improves operator performance in a reconnaissance task when task demand is higher, but may slightly increase operator workload. In the context of the experimental setup, these results suggest that haptic feedback may contribute to heads-up control of a team of autonomous robots. There were no significant differences in levels of situation awareness due to haptic feedback in this study.

An important part of expressing human intent is identifying acceptable tradeoffs among competing performance objectives. We present and evaluate a set of graphical user interfaces (GUIs), that are designed to allow a human to express intent by expressing desirable tradeoffs. The GUIs require an algorithm that identifies the set of Pareto optimal solutions to the multi-objective decision problem, which means that all the solutions are equally good in the sense that there are no other solutions better for every objective. Given the Pareto set, the GUIs provide different ways for a human to express intent by exploring tradeoffs between objectives; once a tradeoff is selected, the solution is chosen. The GUI designs are applied to interactive human-robot path-selection for a robot in an urban environment, but they can be applied to other tradeoff problems. A user study evaluates GUI designs by requiring users to select a tradeoff that satisfies a specified mission intent. Results of the user study suggest that GUIs designed to support an artist's palette-metaphor can be used to express intent without incurring unacceptable levels of human workload.

Traditional path-planning involves (1) choosing start and goal points, (2) calculating a path, and (3) following that path. There are, however, many real world scenarios where an agent might need to change its goal and replan, which frequently includes expensively calculating a new path from scratch. We propose an adaptation to RRT∗ that locally rewires the RRT∗ tree as the robot moves and path costs change. Local rewiring takes advantage of information already existing in the tree and makes small adaptations that accommodate changes. Rewiring adds computational overhead during robot travel, but allows replanning in real time with approximately constant overhead. Empirical studies demonstrate that computational costs are lower than alternative replanners in 2D worlds with moderate obstacle density, and the resulting paths approach optimality as more time is allowed to replan.

In interactive human-robot path-planning, a capability for expressing the path topology provides a natural mechanism for describing task requirements. We propose a topology-aware RRT∗ algorithm that can explore in parallel any given set of topologies. The topological information used by the algorithm can either be assigned by the human prior to the planning or be selected from the human in posterior path selection. Theoretical analyses and experimental results are given to show that the optimal path of any topology can be found, including a winding topological constraint wherein the robot must circle one or more objects of interest.

Many fields of study involve decision-making where trade-offs between different desirable outcomes are to be made. This paper introduces four designs for a graphical user interface (GUI) called the Adverb Palette (AP) to support interactive robot path-planning under multiple performance objectives. The GUI designs apply when the robot must go from an initial configuration to a goal configuration while honoring competing objectives. This novel human-robot interface helps the operator issue commands to the robot to take a specific path from the many available paths, thereby helping the user in decision making process. AP is a metaphor that symbolizes an artist's palette where an artist mixes different colors to get a desired shade of color. For the principal GUI designs, each objective is represented as a color. Since we assume that the human specifies the objectives using language, each objective is metaphorically called an adverb. Two of the approaches, the sliders and the waypoints indicator, have been explored by the researchers in the past, and the other two, palette and prism are novel and may be more useful and intuitive.

An important problem in human-robot interaction is for a human to be able to tell the robot go to a particular location with instructions on how to get there or what to avoid on the way. This paper provides a solution to problems where the human wants the robot not only to optimize some objective but also to honor “soft” or “hard” topological constraints, i.e. “go quickly from A to B while avoiding C”. The paper presents the HARRT* (homotopy-aware RRT*) algorithm, which is a computationally scalable algorithm that a robot can use to plan optimal paths subject to the information provided by the human. The paper provides a theoretic justification for the key property of the algorithm, proposes a heuristic for RRT*, and uses a set of simulation case studies of the resulting algorithm to make a case for why these properties are compatible with the requirements of human-robot interactive path-planning.

Allowing a human to express topological requirements to a robot in language enables untrained users to guide robot movement without requiring the human to understand sophisticated robot algorithms. By using a homotopy class or classes to represent one or more topological requirements, we build a framework that helps a robot understand a human's intent. This paper reviews a homotopic decomposition method that is used to convert any path into a string, which allows homotopic path equivalence to be performed by comparing strings. We then integrate the Homotopic Distributed Correspondence Graph (HoDCG) to infer the homotopic constraint in the format of strings from a language instruction. Finally, we use a homotopic path-planning algorithm that finds the optimal paths for a given objective and homotopic constraint. Experiment results show how a language instruction is converted into a path driven by an implicit topological requirement.

The search for invariants is a fundamental aim of scientific endeavors. These invariants, such as Newton's laws of motion, allow us to model and predict the behavior of systems across many different problems. In the nascent field of Human-Swarm Interaction (HSI), a systematic identification of fundamental invariants is still lacking. Discovering and formalizing these invariants will provide a foundation for developing, and better understanding, effective methods for HSI. We propose two invariants underlying HSI for geometric-based swarms: (1) collective state is the fundamental percept associated with a bio-inspired swarm, and (2) a human's ability to influence and understand the collective state of a swarm is determined by the balance between the span and persistence. We provide evidence of these invariants by synthesizing much of our previous work in the area of HSI with several new results, including a novel user study where users manage multiple swarms simultaneously. We also discuss how these invariants can be applied to enable more efficient and successful teaming between humans and bio-inspired collectives and identify several promising directions for future research into the invariants of HSI.

This paper examines the dynamics of particle swarm optimization (PSO) by modeling PSO as a feedback cascade system and then applying input-to-state stability analysis. Using a feedback cascade system model we can include the effects of the global-best and personal-best values more directly in the model of the dynamics. Thus in contrast to previous study of PSO dynamics, the input-to-state stability property used here allows for the analysis of PSO both before and at stagnation. In addition, the use of input-to-state stability allows this analysis to preserve random terms which were heretofore simplified to constants. This analysis is important because it can inform the setting of PSO parameters and better characterize the nature of PSO as a dynamic system. This work also illuminates the way in which the personal-best and the global-best updates influence the bound on the particle's position and hence, how the algorithm exploits and explores the fitness landscape as a function of the personal best and global best.

Having a well-developed Graphical User Interface (GUI) is often necessary for a human-robot team, especially when the human and the robot are not in close proximity to each other or when the human does not interact with the robot in real time. Most current GUIs process and display information in real time, but the time to interact with these systems does not scale well when the complexity of the displayed information increases or when information must be fused to support decision-making. We propose a new interface concept, a Graphical Narrative Interface (GNI), which presents story-based summaries driven by accumulated data. We hypothesize that the GNI allows users to search and analyze spatiotemporal information more easily and quickly than a typical GUI. This paper (a) uses literature and preliminary GNI designs to identify a set of design requirements and (b) develops a conceptual GNI implementation that satisfies these requirements.

Leveraging the abilities of multiple affordable robots as a swarm is enticing because of the resulting robustness and emergent behaviors of a swarm. However, because swarms are composed of many different agents, it is difficult for a human to influence the swarm by managing individual agents. Instead, we propose that human influence should focus on (a)~managing the higher level attractors of the swarm system and (b)~managing trade-offs that appear in mission-relevant performance. We claim that managing attractors theoretically allows a human to abstract the details of individual agents and focus on managing the collective as a whole. Using a swarm model with two attractors, we demonstrate this concept by showing how limited human influence can cause the swarm to switch between attractors. We further claim that using quorum sensing allows a human to manage trade-offs between the scalability of interactions and mitigating the vulnerability of the swarm to agent failures.

Human interaction with bio-inspired collectives provides an interesting setting for studying shared control. A human will often have knowledge of global objectives and high-level plans, but the collective will often have more detailed lower-level knowledge about the particulars of the situation at hand. Thus it is important to understand how control can be appropriately shared between the human and the collective. We analyze human interaction with bio-inspired collectives using graph theory, and propose that there are two human-side elements that determine how well control is shared: span and persistence. We additionally propose that there is a collective-side element that determines how well control is shared: connectivity. We study two examples of shared-control between a human and a bio-inspired collective: shaping a spatial formation and causing a collective to switch between stable collective states. Our empirical results show that span, persistence, and connectivity combine to affect (1) how influence is shared between the human and the collective and (2) the resulting success of human-collective interactions.

One way for a human and a robot to collaborate on a search task is for the human to specify constraints on the robot's path and then allow the robot to find an optimal path subject to these constraints. This paper presents an anytime solution to the robot's path-planning problem when the human specifies a path constraint and an acceptable amount of deviation from this path. The robot's objective is to maximize information gathered during the search subject to this constraint. We first discretize the path constraint and then convert the resulting problem into a multi-partite graph. Information maximization becomes a submodular orienteering problem on this topology structure. Backtracking is used to generate an efficient heuristic for solving this problem, and an expanding tree is used to facilitate an anytime algorithm.

During unmanned aerial vehicle (UAV) search missions, efficient use of UAV flight time requires flight paths that maximize the probability of finding the desired subject. The probability of detecting the desired subject based on UAV sensor information can vary in different search areas due to environment elements like varying vegetation density or lighting conditions, making it likely that the UAV can only partially detect the subject. This adds another dimension of complexity to the already difficult (NP-Hard) problem of finding an optimal search path. We present a new class of algorithms that account for partial detection in the form of a task difficulty map and produce paths that approximate the payoff of optimal solutions. The algorithms use the mode goodness ratio heuristic that uses a Gaussian mixture model to prioritize search subregions. The algorithms search for effective paths through the parameter space at different levels of resolution. We compare the performance of the new algorithms against two published algorithms (Bourgault's algorithm and LHC-GW-CONV algorithm) in simulated searches with three real search and rescue scenarios, and show that the new algorithms outperform existing algorithms significantly and can yield efficient paths that yield payoffs near the optimal.

Unmanned Aerial Vehicles (UAVs) are increasingly becoming economical platforms for carrying a variety of sensors. Building flight plans that place sensors properly, temporally and spatially, is difficult. The goal of sensor-driven planning is to automatically generate flight plans based on desired sensor placement and temporal constraints. We present a hierarchical sensor-driven flight planning system capable of generating 2D flights that satisfy desired sensor placement and complex timing and dependency constraints. The system makes use of several well-known planning algorithms and includes a user interface. Results demonstrate that the algorithm is general enough for use by a human in several simulated wilderness search and rescue scenarios.

Currently, a single Unmanned Aerial System (UAS) requires several humans managing different aspects of the problem. Human roles often include vehicle operators, payload experts, and mission managers [1-3]. As a step toward reducing the number of humans required, it is desirable to reduce operator workload through effective distributed control, augmented autonomy, and intelligent user interfaces. Reliably doing this requires various roles in the system to be modeled. These roles naturally include the roles of the humans, but they also include roles delegated to autonomy and software decision-making algorithms, meaning the GUI and the unmanned aerial vehicle. This paper presents a conceptual model which models the roles of complex systems as a collection of actors, running in parallel. Results from applying this model to the UAS-enabled Wilderness Search and Rescue (WiSAR) domain indicate (a) it is possible to model the entire WiSAR system at varying degrees of abstraction (b) that building and evaluating the model provides insight into the best practices of WiSAR teams and (c) a way to model human machine interactions that works directly with the Java Pathfinder model checker to detect errors.

Human-swarm interaction methods often allow a human to influence a swarm through either leadership or predation. These methods of influence have two main limitations: (1) although leaders sustain influence over nominal agents for a long period of time, they tend to cause all collective structures to turn in to flocks (negating the benefit of other swarm formations) and (2) predators tend to cause collective structures to fragment. We introduce the use of mediators as a novel shared control method for human-swarm influence and use mediators to shape Couzin-like tori [1]. The mediator method uses special agents that operate from within the spatial center of a swarm. This approach allows a human operator to transform and move a dynamic torus formation while sustaining influence over the torus, avoiding fragmentation, and maintaining the torus' connectivity. The use of mediators allows a human to mold and adapt the torus' behavior and structure to a wide range of spatio-temporal tasks such as military protection and decontamination tasks.

One of the goals of the Journal of Human-Robot Interaction (JHRI) is to promote a stronger research community, including interactions among HRI-relevant conferences and journals. This issue is comprised of papers that first appeared in another format, usually a conference, and that exhibited special insights or area of interest. Authors were invited to submit extensions to these papers with at least 50% new material and told that the papers would be reviewed relative to the standard of scholarship suited to an archival journal. Submitted papers were then reviewed by at least three reviewers, who were instructed to hold the papers to a journal-level standard for scholarship. Of the papers that were invited, five papers were selected for publication.

Human-robot interaction (HRI) is of global importance, as exemplified by the human and financial resources dedicated to research and development in the area. The vision for this special issue was to provide a broad, global perspective on the field. More specifically, the vision was for researchers in representative projects from around the world to share lessons and perspectives on HRI obtained from those projects.

In wilderness search and rescue, objects not native or typical to a scene may provide clues that indicate the recent presence of the missing person. This paper presents the results of augmenting an aerial wilderness search-and-rescue system with an automated spectral anomaly detector for identifying unusually colored objects. The detector dynamically builds a model of the natural coloring in the scene and identifies outlier pixels, which are then filtered both spatially and temporally to find unusually colored objects. These objects are then highlighted in the search video as suggestions for the user, thus shifting a portion of the user's task from scanning the video to verifying the suggestions. This paper empirically evaluates multiple potential detectors then incorporates the best-performing detector into a suggestion system. User study results demonstrate that even with an imperfect detector users' detection increased significantly. Results further indicate that users' false positive rates did not increase, though performance in a secondary task did decrease. Furthermore, users subjectively reported that the use of detector-based suggestions made the overall task easier. These results suggest that such suggestion-based systems for search can increase overall searcher performance but that additional external tasks should be limited.

Swarm robots use simple local rules to create complex emergent behaviors. The simplicity of the local rules allows for large numbers of low-cost robots in deployment, but the same simplicity creates difficulties when deploying in many applicable environments. These complex missions sometimes require human operators to influence the swarms towards achieving the mission goals. Human swarm interaction (HSI) is a young field with few user studies exploring operator behavior. These studies all assume perfect information between the operator and the swarm, which is unrealistic in many applicable scenarios. Indoor search and rescue or underwater exploration may present environments where radio limitations restrict the bandwidth of the robots. This study explores this bandwidth restriction in a user study. Three levels of bandwidth are explored to determine what amount of information is necessary to accomplish a swarm foraging task. The lowest bandwidth condition performs poorly, but the medium and high bandwidth condition both perform well. The medium bandwidth condition does so by aggregating useful swarm information to compress the state information. Further, the study shows operators preferences that should have hindered task performance, but operator adaptation allowed for error correction.

Every year there are numerous cases of individuals becoming lost in remote wilderness environments. Principles of search theory have become a foundation for developing more efficient and successful search and rescue methods. Measurements can be taken that describe how easily a search object is to detect. These estimates allow the calculation of the probability of detection-the probability that an object would have been detected if in the area. This value only provides information about the search area as a whole; it does not provide details about which portions were searched more thoroughly than others. Ground searchers often carry portable GPS devices and their resulting GPS track logs have recently been used to fill in part of this knowledge gap. We created a system that provides a detection likelihood map that estimates the probability that each point in a search area was seen well enough to detect the search object if it was there. This map will be used to aid ground searchers as they search an assigned area, providing real time feedback of what has been “seen.” The maps will also assist incident commanders as they assess previous searches and plan future ones by providing more detail than is available by viewing GPS track logs.

Automated agents for electricity markets, social networks, and other distributed networks must repeatedly interact with other intelligent agents, often without observing associates' actions or payoffs (i.e., minimal information). Given this reality, our goal is to create algorithms that learn effectively in repeated games played with minimal information. As in other applications of machine learning, the success of a learning algorithm in repeated games depends on its learning bias. To better understand what learning biases are most successful, we analyze the learning biases of previously published multi-agent learning (MAL) algorithms. We then describe a new algorithm that adapts a successful learning bias from the literature to minimal information environments. Finally, we compare the performance of this algorithm with ten other algorithms in repeated games played with minimal information.

Robots excel at planning and performing tasks in controlled environments, but poor perception often leads to poor performance in unstructured environments. One typical way of improving robot performance is to give more control to a human operator and then design user interfaces that build the operator's situation awareness. As an alternative, humans can support robot perception to add structure to unstructured environments. We claim that when humans support robot perception, robots can spend more time acting autonomously, which can lead to reduced operator workload and increased overall performance. We present a design process, called perception by proxy, and apply it to a simple manipulation task.

Robots appear to be engaging to many children with autism, and evidence suggests that engagement can facilitate social interaction not only between child and robot but also between child and another human. To date, no objective evidence has established a link between short-term child-robot interactions and long-term child-human interactions. We report on a therapy model that uses a robot in no more than 20% of available therapy time, and describe how a humanoid robot can be used during that limited time to promote generalizable child-human interactions. Preliminary evidence indicates that such low-dose robotics can promote positive child-human interactions.

In this paper, we formalize the problem of human interaction with bio-inspired robot teams (HuBIRT). The formalism applies to a large class of bio-inspired team dynamics and uses simple algebraic graph theory representations to distinguish between interagent influence, environmental influence, and operator influence. These representations lead to metrics for interagent cohesiveness and responsiveness to human input. We then select two different classes of team dynamics, physicomimetics which encodes dynamics using artificial physics, and a biomimetic structure which encodes dynamics using a model of fish behavior. We then demonstrate the relevance of the metrics by conducting a series experiments that demonstrate differences between leader and predator styles of human influence, and conclude with a comparison of nearest-neighbor topologies to metric-based topologies.

Current practice in Wilderness Search and Rescue (WiSAR) is analogous to an intelligent system designed to gather and analyze information to find missing persons in remote areas. The system consists of multiple parts - various tools for information management (maps, GPS, etc) distributed across personnel with different skills and responsibilities. Introducing a camera-equipped mini-UAV into this task requires autonomy and information technology that itself is an integrated intelligent system to be used by a sub-team that must be integrated into the overall intelligent system. In this paper, we identify key elements of the integration challenges along two dimensions: (a) attributes of intelligent system and (b) scale, meaning individual or group. We then present component technology that offload or supplement many responsibilities to autonomous systems, and finally describe how autonomy and information are integrated into user interfaces to better support distributed search across time and space. The integrated system was demoed for Utah County Search and Rescue personnel. A real searcher flew the UAV after minimal training and successfully located the simulated missing person in a wilderness area.

Video-equipped mini unmanned aerial vehicles (mini-UAVs) are becoming increasingly popular for surveillance, remote sensing, law enforcement, and search and rescue operations, all of which rely on thorough coverage of a target observation area. However, coverage is not simply a matter of seeing the area (visibility) but of seeing it well enough to allow detection of targets of interest, a quality we here call "see-ability". Video flashlights, mosaics, or other geospatial compositions of the video may help place the video in context and convey that an area was observed, but not necessarily how well or how often. This paper presents a method for using UAV-acquired video georegistered to terrain and aerial reference imagery to create geospatial video coverage quality maps and indices that indicate relative video quality based on detection factors such as image resolution, number of observations, and variety of viewing angles. When used for offline post-analysis of the video, or for online review, these maps also enable geospatial quality-filtered or prioritized non-sequential access to the video. We present examples of static and dynamic see-ability coverage maps in wilderness search-and-rescue scenarios, along with examples of prioritized non-sequential video access. We also present the results of a user study demonstrating the correlation between see-ability computation and human detection performance.

This paper explores supervisory control of multiple, heterogeneous, independent robots by operator teams. Experimental evidence is presented which suggests that two cooperating operators may have free capacity that can be used to improve primary task performance without increasing average fan-out.

Learning is one way to determine how agents should act, but learning in multi-agent systems is more difficult than in single-agent systems because other learning agents modify their behavior. We introduce a particle-based algorithm called MMM-PHC. MMM-PHC promotes convergence to Nash equilibria in matrix games using the ideas of maxim in strategies and partial commitment. Partial commitment is implemented by restricting policies to a simplex. Simulations show that MMM-PHC performs on a larger class of games than WoLF-PHC.

Robot-based autism therapy is a rapidly developing area of research, with a wide variety of robots being developed for use in clinical settings. Specific, detailed requirements for robots and user interfaces are needed to provide guidelines for the creation of robots that more effectively assist therapists in autism therapy. This paper enumerates a set of requirements for a clinical humanoid robot and the associated human interface. The design of two humanoid robots and an intuitive and flexible user interface for use by therapists in the treatment of children with autism is described.

This paper explores multi-operator supervisory control (MOSC) of multiple independent robots using two complementary approaches: a human factors experiment and an agent-based simulation. The experiment identifies two task and environment limitations on MOSC: task saturation and task diffusion. It also identifies the correlation between task specialization and performance, and the possible existence of untapped spare capacity that emerges when multiple operators coordinate. The presence of untapped spare capacity is explored using agent-based simulation, resulting in evidence which suggests that operators may be more effective when they operate at less than maximum capacity.

Prior work suggests that the potential benefits of mixed initiative management of multiple robots are mitigated by situational factors, including workload and operator expertise. In this paper, we present an experiment where allowing a supervisor and group of searchers to jointly decide the correct level of autonomy for a given situation ("mixed initiative") results in better overall performance than giving an agent exclusive control over their level of autonomy ("adaptive autonomy") or giving a supervisor exclusive control over the agent's level of autonomy ("adjustable autonomy"), regardless of the supervisor's expertise or workload. In light of prior work, we identify two elements of our experiment that appear to be requirements for effective mixed initiative control of large-scale robotic teams: (a) Agents must be capable of making progress toward a goal without having to wait for human input in most circumstances. (b) The operator control interface must help the human to rapidly understand and modify the progress and intent of several agents.

In the priority search phase of Wilderness Search and Rescue, a probability distribution map is created. Areas with higher probabilities are searched first in order to find the missing person in the shortest expected time. When using a UAV to support search, the onboard video camera should cover as much of the important areas as possible within a set time. We explore several algorithms (with and without set destination) and describe some novel techniques in solving this problem and compare their performances against typical WiSAR scenarios. This problem is NP-hard, but our algorithms yield high quality solutions that approximate the optimal solution, making efficient use of the limited UAV flying time.

Mini unmanned aerial vehicles (mUAVs) have the potential to assist wilderness search and rescue groups by providing a bird's eye view of the search area. This paper proposes a method for augmenting visible-spectrum searching with infrared sensing in order to make use of thermal search clues. It details a method for combining the color and heat information from these two modalities into a single fused display to reduce needed screen space for remote field use. To align the video frames for fusion, a method for simultaneously pre-calibrating the intrinsic and extrinsic parameters of the cameras and their mount using a single multi-spectral calibration rig is also presented. A user study conducted to validate the proposed image fusion methods showed no reduction in performance when detecting various objects of interest in this single-screen fused display compared to side-by-side viewing. Most significantly, the users' increased performance on a simultaneous auditory task showed that their cognitive load was reduced when using the fused display.

Camera-equipped mini-UAVs are popular for many applications, including search and surveillance, but video from them is commonly plagued with distracting jittery motions and disorienting rotations that make it difficult for human viewers to detect objects of interest and infer spatial relationships. For time-critical search situations there are also inherent tradeoffs between detection and search speed. These problems make the use of dynamic mosaics to expand the spatiotemporal properties of the video appealing. However, for many applications it may not be necessary to maintain full mosaics of all of the video but to mosaic and retain only a number of recent (temporally local) frames, still providing a larger field of view and effectively longer temporal view as well as natural stabilization and consistent orientation. This paper presents and evaluates a real-time system for displaying live video to human observers in search situations by using temporally local mosaics while avoiding masking effects from dropped or noisy frames. Its primary contribution is an empirical study of the effectiveness of using such methods for enhancing human detection of objects of interest, which shows that temporally local mosaics increase task performance and are easier for humans to use than non-mosaiced methods, including stabilized video.

Wilderness search and rescue (WiSAR) is a challenging problem because of the large areas and often rough terrain that must be searched. Using mini-UAVs to deliver aerial video to searchers has potential to support WiSAR efforts, but a number of technology and human factors problems must be overcome to make this practical. At the source of many of these problems is a desire to manage the UAV using as few people as possible, so that more people can be used in ground-based search efforts. This paper uses observations from two informal studies and one formal experiment to identify what human operators may be unaware of as a function of autonomy and information display. Results suggest that progress is being made on designing autonomy and information displays that may make it possible for a single human to simultaneously manage the UAV and its camera in WiSAR, but that adaptable displays that support systematic navigation are probably needed.

We present a novel technique for behavioral animation through data‐driven behavior synthesis. This technique has two key features: it provides natural character behavior and has a programming‐by‐demonstration interface. Thus we can quickly create compelling autonomous virtual agents that exhibit stylized behavior. First, the human user demonstrates behavior for the character by specifying its high‐level actions (e.g., with a joystick) during an interactive session. Each demonstration is recorded as a sequence of discrete actions. Later, the character synthesizes novel behavior by concatenating segments of action sequences. The choice of segments is guided by simulations that predict fitness. Thus our technique operates such as a cognitive model, providing a character with deliberative decision making. The actions are abstract and can be mapped to any pertinent motions, even procedurally synthesized motions. Thus our technique complements character animation algorithms. We empirically show that our O(logn) technique is scalable, robust when provided with sufficient data, produces effective behavior for a number of problem domains, and is faster than traditional planning. Also, the interface is intuitive enough that character behavior can be created by nontechnical users.

It is often desirable for a human to manage multiple robots. Autonomy is required to keep workload within tolerable ranges, and dynamically adapting the type of autonomy may be useful for responding to environment and workload changes. We identify two management styles for managing multiple robots and present results from four experiments that have relevance to dynamic autonomy within these two management styles. These experiments, which involved 80 subjects, suggest that individual and team autonomy benefit from attention management aids, adaptive autonomy, and proper information abstraction.

Navigation is an essential element of many remote robot operations including search and rescue, reconnaissance, and space exploration. Previous reports on using remote mobile robots suggest that navigation is difficult due to poor situation awareness. It has been recommended by experts in human-robot interaction that interfaces between humans and robots provide more spatial information and better situational context in order to improve an operator's situation awareness. This paper presents an ecological interface paradigm that combines video, map, and robot-pose information into a 3-D mixed-reality display. The ecological paradigm is validated in planar worlds by comparing it against the standard interface paradigm in a series of simulated and real-world user studies. Based on the experiment results, observations in the literature, and working hypotheses, we present a series of principles for presenting information to an operator of a remote robot.

One of the fundamental aspects of robot teleoperation is the ability to successfully navigate a robot through an environment. We define successful navigation to mean that the robot minimizes collisions and arrives at the destination in a timely manner. Often video and map information is presented to a robot operator to aid in navigation tasks. This paper addresses the usefulness of map and video information in a navigation task by comparing a side-by-side (2D) representation and an integrated (3D) representation in both a simulated and a real world study. The results suggest that sometimes video is more helpful than a map and other times a map is more helpful than video. From a design perspective, an integrated representation seems to help navigation more than placing map and video side-by-side.

This paper describes an effort to identify common metrics for task-oriented human-robot interaction (HRI). We begin by discussing the need for a toolkit of HRI metrics. We then describe the framework of our work and identify important biasing factors that must be taken into consideration. Finally, we present suggested common metrics for standardization and a case study. Preparation of a larger, more detailed toolkit is in progress.

Transport selection mechanisms are designed to facilitate seamless connectivity in heterogeneous multi-transport environments, allowing access to the "best" available transport according to user requirements. Evaluating transport configurations dynamically according to the user's preferences and quality of service (QoS) requirements is a challenging task. This paper describes a quantitative approach that applies the Utility Theorem and Nash's Bargaining solution to heterogeneous wireless environments. The mathematical model presented generates and adjusts the transport preference list dynamically depending on the degree to which a transport satisfies user preferences and the application's QoS requirements. We incorporate a negotiation engine using the axiomatic multi-transport bargaining algorithm to integrate local and remote users' requirements in a mutually beneficial manner as devices are connected via a peer-to-peer link. The transport selection model discussed in this paper is computationally light with modest communication overhead, making it suitable for mobile devices

Learning algorithms often obtain relatively low average payoffs in repeated general-sum games between other learning agents due to a focus on myopic best-response and one-shot Nash equilibrium (NE) strategies. A less myopic approach places focus on NEs of the repeated game, which suggests that (at the least) a learning agent should possess two properties. First, an agent should never learn to play a strategy that produces average payoffs less than the minimax value of the game. Second, an agent should learn to cooperate/compromise when beneficial. No learning algorithm from the literature is known to possess both of these properties. We present a reinforcement learning algorithm (M-Qubed) that provably satisfies the first property and empirically displays (in self play) the second property in a wide range of games.

Target acquisition and continuous surveillance using fixed-wing UAVs is a difficult task due to the many degrees of freedom inherent in aircraft and gimbaled cameras. Mini-UAVs further complicate the problem by introducing severe restrictions on the size and weight of electro-optical sensor assemblies. We present a field-tested mini-UAV gimbal mechanism and flightpath generation algorithm as well as a human-UAV interaction scheme in which the operator manually flies the UAV to produce an estimate of the target position, then allows the aircraft to fly itself and control the gimbal while the operator re fines or moves the target position as required.

We discuss several techniques that assist in the field deployments of fixed-wing mini-UAVs to assist search teams, focusing on automatic takeoff and landing. We also present a real-time flightpath generation routine that performs a spiral search centered on a target point, and a path planner that creates waypoints for searches up mountain canyons.

The ability of robots to autonomously perform tasks is increasing. More autonomy in robots means that the human managing the robot may have available free time. It is desirable to use this free time productively, and a current trend is to use this available free time to manage multiple robots. We present the notion of neglect tolerance as a means for determining how robot autonomy and interface design determine how free time can be used to support multitasking, in general, and multirobot teams, in particular. We use neglect tolerance to 1) identify the maximum number of robots that can be managed; 2) identify feasible configurations of multirobot teams; and 3) predict performance of multirobot teams under certain independence assumptions. We present a measurement methodology, based on a secondary task paradigm, for obtaining neglect tolerance values that allow a human to balance workload with robot performance.

In some environments, a learning agent must learn to balance competing objectives. For example, a Q-learner agent may need to learn which choices expose the agent to risk and which choices lead to a goal. In this paper, we present a variant of Q-learning that learns a pair of utilities for worlds with dichotomous attributes and show that this algorithm properly balances the competing objectives and, as a result, efficiently identifies satisficing solutions. This occurs because exploration of the environment is restricted to those options which, according to current knowledge, are likely to avoid unjustifiable exposure to risk. We empirically validate the algorithm by (a) showing that the algorithm quickly converges to good policies in several simulated worlds of various complexities and (b) applying the algorithm to learning a force feedback profile for a gas pedal that helps drivers avoid risky situations.

We present several human-UAV interfaces that support real-time control of a small semi-autonomous UAV. These interfaces are designed for searching tasks and other missions that typically do not have a precise predetermined flight plan. We present a detailed analysis of a PDA-based interface and describe how our other interfaces relate to this analysis. We then offer quantative and qualitative performance comparisons of the interfaces, as well as an analysis of their possible real-world applications.

Most interfaces for robot control have focused on providing users with the most current information and giving status messages about what the robot is doing. While this may work for people that are already experienced in robotic, we need an alternative paradigm for enabling new users to control robots effectively. Instead of approaching the problem as an issue of what information could be useful, the focus should be on presenting essential information in an intuitive way. One way to do this is to leverage perceptual cues that people are accustomed to using. By displaying information in such contexts, people are able to understand and use the interface more effectively. This paper presents interfaces which allow users to navigate in 3-D worlds with integrated range and camera information.

This paper presents prioritized soft constraint satisfaction (PSCS), a novel approach to selecting the ''best" transport in dynamic wireless transport switching systems. PSCS maintains a satisfying connection to another end point by choosing transports based on a user-established range of preferences and priority for criteria such as speed, power, range and cost. Additionally, feedback is provided regarding tradeoffs among the criteria, thus enabling the user to adjust inputs according to the capabilities of the system. We also recommend guidelines for setting preferences and priorities.

In this paper we develop a method for predicting the performance of human-robot teams consisting of a single user and multiple robots. To predict the performance of a team, we first measure the neglect tolerance and interface efficiency of the interaction schemes employed by the team. We then describe a method that shows how these measurements can be used to estimate the team's performance. We validate the performance prediction algorithm by comparing predictions to actual results when a user guides three robots in an exploration and goal-finding mission; comparisons are made for various system configurations.

Advances in robot technology and artificial intelligence have increased the range of robot applications as well as the importance of supporting human interaction with robots and robot teams. Previous work by the authors has highlighted the importance of creating neglect tolerant autonomy and efficient interfaces. In this paper, lessons learned from evaluating neglect tolerance and interface efficiency are compiled into a set of principles for efficient interaction. Emphasis is placed on designing efficient interfaces, but many of the principles require autonomy levels that support the principles. Each principle is illustrated by an example and motivated by citing relevant factors from cognitive information processing.

Engineers, business managers, and governments are increasingly aware of the importance and difficulty of integrating technology and humans. The presence of technology can enhance human comfort, efficiency, and safety, but the absence of human-factors analysis can lead to uncomfortable, inefficient, and unsafe systems. Systematic human-centered design requires a basic understanding of how humans generate and manage tasks. A very useful model of human behavior generation can be obtained by recognizing the task-specific role of mental models in not only guiding execution of skills but also managing initiation and termination of these skills. By identifying the human operator's mental models and using them as templates for automating different tasks, we experimentally support the hypothesis that natural and safe interaction between human operator and automation is facilitated by this model-based human-centered approach. The design of adaptive cruise control (ACC) systems is used as a case study in the design of model-based task automation systems. Such designs include identifying ecologically appropriate perceptual states, identifying perceptual triggering events for managing transitions between skilled behaviors, and coordinating the actions of automation and operator.

Presents an end-to-end solution to the battlefield scenario where M unmanned air vehicles are assigned to strike N known targets, in the presence of dynamic threats. The problem is decomposed into the subproblems of (1) cooperative target assignment, (2) coordinated UAV intercept, (3) path planning, and (4) feasible trajectory generation. The design technique is based on a hierarchical approach to coordinated control. Detailed simulation results are presented.

Within the field of behavior-based robotics, a useful and popular technique for robot control is that of utilitarian voting, where behaviors representing objectives assign utilities to candidate actions. Utilitarian voting allows for distributed, modular control but is not well suited to systems with very high resolution or several, dependent degrees of freedom. Past systems have depended on exhaustive searches that choose the action with the highest combined utility. We present, instead, a framework that casts the action search problem as an anytime algorithm. This allows directed searches to be used with time limit constraints. Aspiration-based satisficing additionally helps to reduce CPU usage. In experiments so far, we have decreased CPU usage by UP to 42% while causing result quality to be no more than 6% worse.

Human-robot interaction is becoming an increasingly important research area. In this paper, we present a theoretical characterization of interaction efficiency with an aim towards designing a human-robot system with adjustable robot autonomy. In our approach, we analyze how modifying robot control schemes for a given autonomy mode can increase system performance and decrease workload demands on the human operator. We then perform a case study of the design of a shared-control teleoperation scheme and compare interaction efficiency against a traditional manual-control teleoperation scheme.

Presents an end-to-end solution to the cooperative control problem represented by the scenario where M unmanned air vehicles (UAVs) are assigned to transition through N known target locations in the presence of dynamic threats. The problem is decomposed into the subproblems of: 1) cooperative target assignment; 2) coordinated UAV intercept; 3) path planning; 4) feasible trajectory generation; and 5) asymptotic trajectory following. The design technique is based on a hierarchical approach to coordinated control. Simulation results are presented to demonstrate the effectiveness of the approach.

The amount of information available on the World Wide Web has been increasing dramatically in recent years. To enhance speedy searching and retrieving Web documents of interest, researchers and practitioners have partially relied on various information retrieval techniques. We propose a probabilistic model to classify Web documents into relevant documents and irrelevant documents with respect to a particular application ontology, which is a conceptual-model snippet of standard ontologies. Our probabilistic model is based on multivariate statistical analysis and is different from the conventional probabilistic information retrieval models. The experiments we have conducted on a set of representative Web documents indicate that the proposed probabilistic model is promising in binary-categorization of multiple-record Web documents.

Human-robot interaction is becoming an increasingly important research area. In this paper, we present our work on designing a human-robot system with adjustable autonomy and describe not only the prototype interface but also the corresponding, robot behaviors. In our approach, we grant the human meta-level control over the level of robot autonomy, but we allow the robot a varying amount of self-direction with each level. Within this framework of adjustable autonomy, we explore how existing, robot control approaches can be adapted and extended to be compatible with adjustable autonomy.

The design of artificial decision-making systems must be founded on some notion of rationality. Conventional multi-agent decision-making methodologies, such as von Neumann-Morgenstern game theory, are based on the paradigm of individual rationality, which requires decision makers to take the action that is best for themselves, regardless of its effect on other decision makers. Relaxing the demand for the "best possible" decision, however, opens the way to accommodate the preferences of others. Satisficing game theory is a new approach to multi-agent decision making that permits decision makers to adjust their preferences in a controlled way to give consideration to others by permitting conditional preferences whereby a decision maker is able to adjust its preferences as a function of the preferences of others.

In previous work, we addressed how the world sometimes mandates switches in human behaviors (M.A. Goodrich et al., 1999). This led to a characterization of how humans manage such mandatory transitions. In addition to mandatory behavior switches, there are also situations where humans exhibit discretionary behavior switches. The authors present a mathematical characterization of discretionary behavior switches which is applicable to both modeling human behavior generation as well as to developing action selection mechanisms for behavior based robotics. We support this model by analyzing behaviors observed in human subjects and by synthesizing behaviors in a mobile robot.

Human-centered automation problems have multiple attributes: an attribute reflecting human goals and capabilities, and an attribute reflecting automation goals and capabilities. In the absence of a general theory of human interaction with complex systems, it is difficult to define and find a unique optimal multiattribute resolution to these competing design requirements. We develop a systematic approach to such problems using a multiattribute decomposition of human and automation goals. This paradigm uses both the satisficing decision principle which is unique to two-attribute problems, and the domination principle which is a common manifestation of the optimality principle in multiattribute domains. As applied to human-centered automation in advanced vehicle systems, the decision method identifies performance evaluations and compares the safety benefit of a system intervention against the cost to the human operator. We illustrate the method by analyzing an automated system to prevent lane departures.

Model-predictive control, which is an alternative to conventional optimal control, provides controller solutions to many constrained and nonlinear control problems. However, even when a good model is available, it may be necessary for an expert to specify the relationship between local model predictions and global system performance. We present a satisficing fuzzy logic controller that is based on a receding control horizon, but which employs a fuzzy description of system consequences via model predictions. This controller considers the gains and losses associated with each control action, is compatible with robust design objectives, and permits flexible defuzzifier design. We demonstrate the controller's application to representative problems from the control of uncertain nonlinear systems.

In a navigation context, a mental model is defined as an internal representation employed to encode, predict, and evaluate the consequences of perceived and expected changes in the environment that result from both active planning and decision making as well as from external influences. A model is proposed to capture how human navigators dynamically represent the world to accomplish goal directed navigation tasks. This situated representation scheme not only embodies multi-granular notions of location and orientation, but also the ability to learn from observations and active exploration. The computational framework for this situated representation is based on evidence theory.

The existence of an optimal control policy and the techniques for finding it are grounded fundamentally in a global perspective. These techniques can be of limited value when the global behaviour of the system is difficult to characterize, as it may be when the system is nonlinear, when the input is constrained, or when only partial information is available regarding system dynamics or the environment. Satisficing control theory is an alternative approach that is compatible with the limited rationality associated with such systems. This theory is extended by the introduction of the notion of strong satisficing to provide a systematic procedure for the design of satisficing controls. The power of the satisficing approach is illustrated by applications to representative control problems.