A robust understanding of a human's internal state can greatly improve human–robot teaming, as estimating the human teammates' workload can inform more dynamic robot adaptations. Existing workload estimation methods use standard machine learning techniques to model the relationships between physiological metrics and workload. However, such methods are not sufficient for adaptive systems, as standard machine learning techniques struggle to make accurate workload estimates when the human–robot team performs unknown tasks. A meta-learning-based workload estimation algorithm is introduced and an initial analysis is conducted to show how adapting a machine learning model's parameters using task-specific information can improve result in more accurate workload estimates for unknown tasks.

Human-robot interaction has emerged over the last few decades as a critical aspect of deploying robot systems and numerous important challenges have been addressed; however, the deployment of heterogeneous robot systems that team with humans to reliably complete tasks in real dull, dirty and dangerous domains remains nascent. These domains, such as forestry management, infrastructure inspection and demolition, or agricultural crop management suffer from severe workforce shortages and high workforce turnover. These jobs also often require a skilled workforce and are some of the most dangerous. Robots can team with humans to increase safety and productivity, while reducing costs and not taking critical skilled labor jobs. Recent trends have moved the field forward for well-structured laboratory, home or office environments, however the resulting human-robot teaming technologies developed for these environments typically do not translate to the unstructured, uncertain and dynamic field environments represented by the dull, dirty and dangerous domains. Making the necessary leaps forward requires an increased focus on human-robot teaming, robot technologies, and the associated necessary field work. This keynote will focus on critical research directions for the next decade plus. These challenges will require systems level developments that support intelligent autonomy and teaming capabilities as well as the breadth of human-robot interaction roles - from supervisors to bystanders [1]–[3]. Future teams for these domains will span single human-single robot systems to highly heterogeneous multiple human-multiple robot teams that cooperative heavily across all team members and have a mix of capabilities to support humans, while also keeping humans safe.

Quantifying transparency requires evaluating the transparency embedded in the various system design elements to determine how they impact one another and influence human-collective interactions. Prior work demonstrated limitations of an abstract collective interaction. Interface designs to address these limitations and improve human-collective interaction transparency were evaluated for a sequential best-ofN decision-making task with four collectives, each consisting of 200 individual entities. The Informed and Simple visualizations’ predictive progress bars improved transparency and the overall human-collective team performance.

In a world with increasing numbers of robots operating in everyday human spaces, the employees at this robotics company are pioneers, with intelligent point-to-point path planning and autonomous transport operations in 150+ factory and warehouse locations in North America. At the time of research, this robotics company consisted of 250 employees. Unlike other industry models, their robots are designed to operate with people in mixed human-machine spaces, yet no HRI style evaluations had previously been run with their robots. As early observers of how factory workers and transport robot interact, across varied job roles ranging from technology design to customer relations, this work sought to leverage employee knowledge and experiences to identify opportunities for improving the communication capabilities of the robots, resulting in the addition of several robot state communications to their initial software set leveraging both employee- and social robotics literature- sourced ideas for communicating with lights. To achieve this a social robotics researcher spent a summer onsite at the robotics company, getting to know their software stack and culture. Her research activities included: (1) a company-wide survey relative to the robot’s light, sound, and motion communications was sent out and analyzed, (2) the development of three new light sets (car-like, sweeping, heartbeat) and five overall states (blocked, at goal, turning, idle), and (3) a user study evaluating the developed light sets relative to the current robot default light patterns, all significantly improving the overall legibility of the targeted robot state communications: at goal, blocked, turning, and idle. Our initial findings advance knowledge in which style of light patterns is best for different communication states, showing that eye-catching lights are best for high urgency states, such as blocked, and subtle lights are best for low urgency states, such as idle. Finally, the latest software release for this robot has deployed a subset of these light patterns to all of their currently operating client sites, i.e., anyone who updates their robots to the latest release will benefit from these research results. This deployment sets the ground for future researchers exploring how end-users at different sites have responded to the new, more communicative light patterns.

Collective robotic systems are biologically inspired and advantageous due to their apparent global intelligence and emergent behaviors. Many applications can benefit from the incorporation of collectives, including environmental monitoring, disaster response missions, and infrastructure support. Transparency research has primarily focused on how the design of the models, visualizations, and control mechanisms influence human-collective interactions. Traditionally most transparency research has evaluated one system design element. This article analyzed two models and visualizations to understand how the system design elements impacted human-collective interactions, to quantify which model and visualization combination provided the best transparency, and provide design guidance, based on remote supervision of collectives. The consensus decision-making and baseline models, as well as an individual collective entity and abstract visualizations, were analyzed for sequential best-of- n decision-making tasks involving four collectives, composed of 200 entities each. Both models and visualizations provided transparency and influenced human-collective interactions differently. No single combination provided the best transparency.

Robotic collectives are composed of hundreds or thousands of distributed robots using local sensing and communication that encompass characteristics of biological spatial swarms, colonies, or a combination of both. Interactions between the individual entities can result in emergent collective behaviors. Human operators in future disaster response or military engagement scenarios are likely to deploy semi-autonomous collectives to gather information and execute tasks within a wide area, while reducing the exposure of personnel to danger. This article presents and evaluates two action selection models in an experiment consisting of a single human operator supervising four simulated collectives. The action selection models have two parts: (1) a best-of- n decision-making model that attempts to choose the highest-quality target from a set of n targets and (2) a quorum sensing task sequencing model that enables autonomous target site occupation. An original biologically inspired insect colony decision model is compared to a bias-reducing model that attempts to reduce environmental bias, which can negatively influence collective best-of- n decisions when poorer-quality targets are easier to evaluate than higher-quality targets. The collective decision-making models are compared in both supervised and unsupervised trials. The bias-reducing model without human supervision is slower than the original model but is 57% more accurate for decisions where evaluating the optimal target is more difficult. Human-collective teams using the bias-reducing model require less operator influence and achieve 25% higher accuracy with difficult decisions compared to the teams using the original model.

High-stress environments, such as first-response or a NASA control room, require optimal task performance, as a single mistake may cause monetary loss or even the loss of human life. Robots can partner with humans in a collaborative or supervisory paradigm to augment the human’s abilities and increase task performance. Such teaming paradigms require the robot to appropriately interact with the human without decreasing either’s task performance. Workload is related to task performance; thus, a robot may use a human’s workload state to modify its interactions with the human. Assessing the human’s workload state may also allow for dynamic task (re-)allocation, as a robot can predict whether a task may overload the human and, if so, allocate it elsewhere. A diagnostic workload assessment algorithm that accurately estimates workload using results from two evaluations, one peer based and one supervisory based, is presented. The algorithm correctly classified workload at least 90% of the time when trained on data from the same human--robot teaming paradigm. This algorithm is an initial step toward robots that can adapt their interactions and intelligently (re-)allocate tasks.

Supervisory control environments, such as the NASA control room can induce high workload levels in situations where a single error is capable of costing millions of dollars. An intelligent system can improve human supervisor performance by monitoring the human's workload levels and intelligently adapting the system capabilities, such as adapting the interaction medium or reallocating roles and responsibilities between the human and the system. Systems capable of responding promptly and accurately to the human's changes in workload require a workload assessment algorithm that can detect changes to all components of workload in real time. A review of 24 workload assessment algorithms across six task domains is provided. Each algorithm is reviewed based on four criteria: sensitivity, diagnosticity, suitability, and generalizability. The majority of the reviewed algorithms were developed for a specific task domain and are unable to generalize different tasks. Further, the majority of the algorithms do not account for individual differences, only assess one or two workload components, and do not classify underload.

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement technique-the NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment, and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex human-machine systems.

Field studies indicate that European starlings in a flock coordinate utilizing a nearest neighbors approach. This model of communication is known in the biological swarm literature as the topological model. Each starling coordinates with a fixed number of nearest neighbors, and this number is referred to as the topological distance. The presented research evaluates the topological model in the context of a swarm robotics task, where a simulated artificial swarm is tasked to search for and go to a goal in the environment. Specifically, experiments are conducted on a high-fidelity, multi-robot simulator to analyze the performance of the tasked swarm as the topological distance is varied. Connections are drawn between the topological model based on starlings and the Delta-disk model, which is widely used in the robotics community to model agent interaction zones. Using graph measures, conditions under which the two models generate comparable networks are presented.

Reaction time and response time have been successfully measured in laboratory settings. As robots move into the real-world, such metrics are needed for human-robot team deployments when evaluating interaction naturalness, ability to maintain safety, and task performance. Potential real-world reaction and response time metrics for peer-based teams are presented. Primary and secondary task reaction and response times were measured via video and auditory coding for a subset of first response tasks. The successful application of the metrics showed that primary task reaction and response times were longer for human-robot teams.

Mass casualty events caused by a biological weapon require fully capable first response teams. However, human first responders are equipped with protective gear, which limits their capabilities to complete tasks. Robots can be employed to work collaboratively with the first responders in order to augment the human's reduced abilities. The robot needs to understand and adapt to the human's workload level in order for the human-machine team to effectively complete tasks. The automatic detection of human workload levels can provide valuable insight into the human's capabilities, as workload has a direct relationship with task performance. The robot can monitor the objective metrics of the human's workload level in order to accurately estimate workload via a workload assessment algorithm. The algorithm must be able to assess overall workload and the components of workload, in order for the robot to correctly adapt its interactions or reallocate tasks among the team. A novel workload assessment algorithm that provides an accurate estimate of overall workload and each workload component is presented and evaluated. The algorithm is capable of distinguishing between high and low workload conditions; however, the algorithm's workload values correlate poorly to a generated workload model. Modifications to enhance the algorithm's capabilities are discussed and will be investigated in future work.

The biological swarm literature presents communication models that attempt to capture the nature of interactions among the swarm's individuals. The evaluated hypothesis that the choice of a biologically inspired communication model can affect an artificial swarm's performance for a given task was supported.

Accurate performance models are important for interaction technique, application, and hardware design. The limited screen size of mobile devices and use of touch interaction require unique considerations, especially when interacting with large amounts of information. This paper considers the performance impact of target visibility on mobile smartphone applications that provide on- and offscreen content with the commonly used direct-touch interactions and four cursor-based interaction methods for precise selection. Three existing and 12 novel performance models are experimentally validated. Fitts' Law, which was not designed for modelling selection of offscreen targets, did not predict interaction times for mobile interaction methods as accurately as is commonly observed with desktop interaction with onscreen targets. Target visibility was found to greatly impact interaction times (particularly for direct-touch interaction). The presented models that incorporate variables related to target visibility greatly improve predicted interaction times. The use and merits of the top models are discussed, emphasizing the importance and implications of accepted user-interface design guidelines.

Successful human-robot (H-R) teams working with direct interaction and close coupling will have relationships that vary, depending on whether or not collaboration is prioritized and how collaboration changes the human mental workload and the team's task performance. Modeling of representative functions can provide predictions of changes in mental workload and the potential impact on team performance. The presented research focuses on modeling and quantifying mental workload for H-R teams in which team members have some individually assigned responsibilities but must also make joint decisions with a teammate. IMPRINT Pro, a discrete event simulation modeling tool created by the U.S. Army Research Laboratory, was used to model human-human (H-H) and H-R teams completing a reconnaissance task in a building. This research evaluated H-H and H-R teams completing the same reconnaissance tasks. Predictions of mental-workload levels from the model and the evaluation results showed that mental workload was lower for the H-R teams. The results for the closely coupled teams were compared to results from a prior evaluation with a master-slave relationship; similar results were found for both evaluations. Mental workload was lower in H-R teams than in H-H teams, but task performance did not differ between the two.

Human-swarm interaction seeks to significantly increase the number of robots, which also increases uncertainty. Most prior research ignores uncertainty. Sources of uncertainty based on biological swarms are presented.

Social networks provide opportunities to overshare information in a secondhand nature as information passes from primary to secondary to tertiary sources. This information can be supplemented with geographical information to obtain the approximate location of the primary information source without the primary source's consent, presenting a potential breach of privacy. Messages regarding severe illness were obtained from the popular social media site Twitter to understand how sensitive information may propagate unregulated through secondhand information sharing. Results indicate that 59% of tweets concerning severe illness are Retweets distributed without explicit consent from the primary source. A majority of these Retweets were propagated by family and friends of the primary source.

The coalition formation problem for task allocation is a difficult and increasingly studied problem (e.g., [1-3]); however, less studied is the notion of task preemption in multi-robot systems. A previously proposed ascending auction based task allocation protocol, RACHNA, is one of the few algorithms that explicitly allows task preemption. This paper demonstrates that the manner in which RACHNA permits task preemption has an undesirable side effect in which RACHNA needlessly changes the coalition assigned to a given task, even when the original coalition can still complete the task. A simultaneous descending auction based approach to task allocation is introduced that allows task preemption and never exhibits the unnecessary task reassignments exhibited by RACHNA.

When people interact with intelligent agents, they likely rely upon a wide range of existing knowledge about machines, minds, and intelligence. This knowledge not only guides these interactions, but it can be challenged and potentially changed by interaction experiences. We hypothesized that a key factor mediating conceptual change in response to human-machine interactions is cognitive conflict, or dissonance. In this experiment, we evaluated whether interactions with a robot partner during a realistic medical triage scenario caused increased levels of cognitive dissonance relative to a control condition in which the same task was performed with a human partner. In addition, we evaluated whether heightened levels of dissonance affected concepts about agents. We observed increased cognitive dissonance after the human-robot interaction and found that this dissonance was correlated with a significantly less intentional (e.g., less human-like) view of the intelligence inherent to computers.

This experiment investigates the spatial memory and attention when human acts as supervisor of one or two groups of distributed robot teams in a large virtual environment (VE). The problem is similar to learning a new environment and interpreting its spatial structure, e.g., [Mou and McNamara 2002], but less is known when attention is divided or locomotion is involved. Our motivation arises in the context of humans and robots acting cooperatively together as a team. Such teaming is becoming increasingly important in many scenarios, such as disaster relief, and wilderness search and rescue [Humphrey and Adams 2009].

The effect of a robotic teammate on a human partner's workload has not been fully quantified. Prior research found that human participants experienced lower workload when working with a robotic partner than when working with a human partner. An evaluation investigated whether a similar trend in workload exists for tasks requiring direct and collaborative interaction between the partners, and joint team decision-making. The subjective results indicate a similar trend to the prior results; participants rated workload lower for the more complex task when partnered with a robot than when partnered with a human.

Clips from an evaluation where participants, each paired with either a human or robot partner, were deployed to search a hallway for suspicious items, in a manner similar to tactics used by first responders handling bomb threats are presented. The teams used natural, verbal communication to collaborate, determine where hazards were located, and which items were suspicious. The video demonstrates that the investigations in both conditions played out in a similar manner and participants were able to complete the investigations successfully with a robot partner; however, sometimes the participants were uncertain how to interact with the robot.

This paper presents a mixed reality system for combining real robots, humans, and virtual robots. The system tracks and controls physical robots in local physical space, and inserts them into a virtual environment (VE). The system allows a human to locomote in a VE larger than the physically tracked space of the laboratory through a form of redirected walking. An evaluation assessed the conditions under which subjects found the system to be the most immersive.

Map-based interfaces have been developed to support collaborative control of unmanned vehicles (i.e., robots). Annotation on the map (or geospatial annotation) has been proposed as an effective way to support team collaboration; however, there is a lack of research focused on the design of geospatial annotation tools to promote usability and task performance. The utility of location reference in geospatial annotations for communication and information sharing is the focus of this article. Two annotation tool designs were developed. The annotation contents were directly anchored on the map in the first design, whereas in the second design annotations were summarized in a separate panel on the interface. Evaluation participants followed instructions from a simulated team leader and assigned unmanned vehicles to different tasks for two simulated scenarios that include searching for victims and collecting hazardous materials samples. The results demonstrate the potential of using geospatial annotations to enrich communication and support map-based unmanned vehicle control. Participants appreciated the direct location reference feature of the first design and had generally shorter response time, but felt that the second design provided better usability and lower task workload. These results suggest that the user experience depends on the manner of obtaining information from the annotation tools, and the integration of the tool with user's task flow and other interface components, such as the map display. The presented results can be used as a basis for designing geospatial annotation tools for team collaboration that better fit user needs and preferences.

Human-Robot peer-based teams are evolving from a far-off possibility into a reality. Human Performance Moderator Functions (HPMFs) can be used to predict human behavior by incorporating the effects of internal and external influences such as fatigue and workload. The applicability of HPMFs to human-robot teams is not proven. The presented research focuses on determining the applicability of workload HPMFs in team tasks for first response mass casualty triage incidents between a Human-Human and a Human-Robot team. A model representing workload for each team was developed using IMPRINT Pro. The results from an empirical evaluation were compared to the model results. While significant differences between the two conditions were not found in all data, there was a general trend that workload in the human-robot condition was slightly lower than the workload experienced in the human-human condition. This trend was predicted by the IMPRINT Pro models. These results are the first to indicate that existing HPMFs can be applied to human-robot peer-based teams.

Precise selection is one of the key difficulties of touch-based interaction. In some cases (such as interacting with a map-based interface), precise selection is crucial for the correct specification of information. This work investigates the difference in user input error and speed for line drawing tasks using touch and mouse-based interaction. A parameterized method of solving the nearest-point-on-a-curve problem was developed in order to obtain a quantifiable error difference between mouse and touch-based interaction for line drawing tasks. A user study was conducted and determined that mouse-based interaction occurs at a slower rate of speed and results in less error than touch-based interaction for line drawing tasks.

Robots are frequently presented with vast arrays of diverse data. Unfortunately, perfect memory and recall provides a mixed blessing. While flawless recollection of episodic data allows increased reasoning, photographic memory can hinder a robot's ability to operate in real-time dynamic environments. Human-inspired forgetting methods may enable robotic systems to rid themselves of out-dated, irrelevant, and erroneous data. This paper presents the use of human-inspired forgetting to act as a filter, removing unnecessary, erroneous, and out-of-date information. The novel ActSimple forgetting algorithm has been developed specifically to provide effective forgetting capabilities to robotic systems. This paper presents the ActSimple algorithm and how it was optimized and tested in a WiFi signal strength estimation task. The results generated by real-world testing suggest that human-inspired forgetting is an effective means of improving the ability of mobile robots to move and operate within complex and dynamic environments.

Coalition formation is a fundamental problem in multi-agent systems. In characteristic function games (CFGs), each coalition C of agents is assigned a value indicating the joint utility those agents will receive if C is formed. CFGs are an important class of cooperative games; however, determining the optimal coalition structure, partitioning of the agents into a set of coalitions that maximizes the social welfare, currently requires O(3n) time for n agents. In light of the high computational complexity of the coalition structure generation problem, a natural approach is to relax the optimality requirement and attempt to find an approximate solution that is guaranteed to be close to optimal. Unfortunately, it has been shown that guaranteeing a solution within any factor of the optimal requires Ω(2n) time. Thus, the best that can be hoped for is to find an algorithm that returns solutions that are guaranteed to be as close to the optimal as possible, in as close to O(2n) time as possible. This paper contributes to the state-of-the-art by presenting an algorithm that achieves better quality guarantees with lower worst case running times than all currently existing algorithms. Our approach is also the first algorithm to guarantee a constant factor approximation ratio, 1/8, in the optimal time of O(2n. The previous best ratio obtainable in O(2n) was 2/n.

This half-day tutorial will cover topics related to conducting cognitive task analysis and cognitive work analysis for purposes of informing human-robot interaction design and development. The goal of the tutorial is to provide attendees with an overview and comparison of various cognitive task analysis and cognitive work analysis methods, an understanding of how to conduct these types of analyses, collect the necessary data for analysis, and provide real-world case studies for specific cognitive task analysis and cognitive work analysis. The tutorial will include examples from actual analyses and data collection activities.

Interaction between humans and robots in peer-based teams can be dramatically affected by human performance. Our research is focused on determining if existing human performance moderator functions apply to peer-based human-robot interaction and if not, how such functions must be modified. Our initial work focuses on modeling workload. Validation of the models will require human subject evaluations. Future work will incorporate larger numbers of performance moderator functions and will apply the results to distributing tasks to team members.

The objective is to develop a mobile human-robot interface that is optimized for multi-touch input. Our existing interface was designed for mouse and keyboard input and was later adopted for voice and touch interaction. A new multi-touch interface permits multi-touch gestures, for example zooming and panning a map, and robot task specific touch interactions. An initial user evaluation found that the multi-touch interface is preferred and yields superior performance.

Prior results demonstrated that the general visualization abstraction (GVA) algorithm can perform information abstraction (i.e., selection and grouping) and determine how information items should be presented (i.e., size) while lowering workload and improving situational awareness and task performance. This paper presents results from a within-subject evaluation to ascertain the relative strengths and weaknesses of the GVA algorithm's components and associated learning effects. The results corroborate the previous results and demonstrate that the GVA algorithm's underlying subcomponent structural composition is beneficial. Furthermore, these results indicate that usage of the GVA algorithm requires some learning before the benefits are achieved.

This paper presents a novel visualization technique that provides integration, abstraction, and sharing of the information generated by remotely deployed robots or sensors. The General Visualization Abstraction (GVA) algorithm is designed to display the most useful information items at any moment by determining an importance value for each information item with a focus on two classes of information: historically relevant and currently relevant information, and novel and emerging information.

Robots in remote environments (e.g., emergency response) have many potential benefits and affordances, with imagery (or video) being a major, if not primary, affordance. However, remote imagery is usually affected by the keyhole effect, or viewing the world through a ¿soda straw.¿ This work focuses on reducing the keyhole effect by improving the viewing angle of the imagery using a novel method that produces results more akin to that provided by the human vision system. The method and early results are subsequently presented for this human eye like perspective for remote vision.

This paper presents results from a user evaluation of a real multiple robot system in which the human's perceived workload and performance were measured. Participants completed tasks with one, two and four real heterogeneous mobile ground-based robots for indoor material transportation tasks. Twelve participants completed four trials of each task over two days. Generally speaking, little difference was found between the one- and two-robot tasks; however, perceived workload significantly increased while performance decreased during the four-robot task. A correlation analysis found that perceived workload increased as the number of commands (in total and by command type) and completion times increased. The highest number of accidents and user errors occurred during the four-robot task. In order to increase the number of robots a single human can supervise, the human must be provided with capabilities that reduce the number of required commands while also optimising the human's interactions with the robots.

Compasses have been used for centuries to express directions and are commonplace in many user interfaces; however, there has not been work in human-robotic interaction (HRI) to ascertain how different compass visualizations affect the interaction. This paper presents a HRI evaluation comparing two representative compass visualizations: top-down and in-world world-aligned. The compass visualizations were evaluated to ascertain which one provides better metric judgment accuracy, lowers workload, provides better situational awareness, is perceived as easier to use, and is preferred. Twenty-four participants completed a within-subject repeated measures experiment. The results agreed with the existing principles relating to 2D and 3D views, or projections of a three-dimensional scene, in that a top-down (2D view) compass visualization is easier to use for metric judgment tasks and a world-aligned (3D view) compass visualization yields faster performance for general navigation tasks. The implication for HRI is that the choice in compass visualization has a definite and non-trivial impact on operator performance (world-aligned was faster), situational awareness (top-down was better), and perceived ease of use (top-down was easier).

As multiple robot systems become more common, it is necessary to develop scalable human-robot interfaces that permit the inclusion of additional robots without reducing the overall system performance. Workload and situational awareness play key roles in determining the ratio of m operators to n robots. A scalable interface, where m is much smaller than n, will have to manage the operator's workload and promote a high level of situation awareness. This work focused on the development of a scalable interface for a single human-multiple robot system. This interface introduces a relational "halo display that augments a camera view to promote situational awareness and the management of multiple robots by providing information regarding the robots' relative locations with respect to a selected robot. An evaluation was conducted to determine the scalability of the interface focusing on the effects of increasing the number of robots on workload, situation awareness, and robot usage. Twenty participants completed two bomb defusing tasks: one employing six robots, the other nine. The results indicated that increasing the number of robots increased overall workload and the operator's situation awareness.

The purpose of this paper is to present an initial delineation of the inherent components required for unmanned vehicles to possess situation awareness. A broadly adapted human situation awareness definition is directly applied to the notion of unmanned vehicle situation awareness. This work focuses on identifying the inherent components of unmanned vehicle situation based upon the components of human situation awareness. The presented work is foundational for developing a holistic unmanned vehicle situation awareness architecture. It is hypothesized that unmanned vehicles that possess situation awareness will better accommodate dynamic situations while improving human-robotic interaction.

This paper demonstrates a learning mechanism for complex tasks. Such tasks may be inherently expensive to learn in terms of training time and/or cost of obtaining each training pattern. Learning simple, safe tasks and extending them to more complex tasks can cause faster convergence to the solution. This method has been formalized and demonstrated on a simulated multiple robot (multi-robot) scenario. The objective is to effectively search out and destroy stationary hostile agents present in an unknown urban terrain map. Using the presented method, the robots learn how to effectively map the area, and then improve their learning modules for the complex task. The robots are simple behavioral agents with minimal communication

As the community strives towards autonomous multi-robot systems, there is a need for these systems to autonomously form coalitions to complete assigned missions. Numerous coalition formation algorithms have been proposed in the software agent literature. Algorithms exist that form agent coalitions in both super additive and non-super additive environments. The algorithmic techniques vary from negotiation-based protocols in multi-agent system (MAS) environments to those based on computation in distributed problem solving (DPS) environments. Coalition formation behaviors have also been discussed in relation to game theory. Despite the plethora of MAS coalition formation literature, to the best of our knowledge none of the proposed algorithms have been demonstrated with an actual multi-robot system. There exists a discrepancy between the multi-agent algorithms and their applicability to the multi-robot domain. This paper aims to bridge that discrepancy by unearthing the issues that arise while attempting to tailor these algorithms to the multi-robot domain. A well-known multi-agent coalition formation algorithm has been studied in order to identify the necessary modifications to facilitate its application to the multi-robot domain. This paper reports multi-robot coalition formation results based upon simulation and actual robot experiments. A multi-agent coalition formation algorithm has been demonstrated on an actual robot system.

A Personal Digital Assistant (PDA) based interface has been developed to provide teleoperation of a mobile robot. The interface provides three different screen designs, all of which employ touch (finger) based interaction rather than stylus based interaction. The interface provides general interaction capabilities for driving the robot based upon the information display. The Vision-only screen provides the forward facing camera image, the Sensory-only screen provides the on-board ultrasonic sensors and laser range finder information, while the Vision with sensory overlay screen integrates all three data sets. A user evaluation was conducted to evaluate the usability and perceived workload required for each screen. Thirty participants completed the quantitative evaluation. The focus of this paper is the obtained perceived workload results.

In this correspondence, we introduce our work on sketch understanding, focusing here on the analysis of a sketched route map. A route map is drawn to help someone navigate along a path for the purpose of reaching a goal. A hand-sketched route map does not generally contain complete map information and is not necessarily drawn to scale, but yet it contains the correct qualitative information for route navigation. Here we propose a methodology for extracting a qualitative model of a sketched route map, based on human navigation strategies, using spatial relationships. Linguistic descriptions are generated from the sketch, both in the form of detailed descriptions at discrete path steps and also as a high-level route description. To describe the path linguistically, one must first be able to understand the path in a qualitative sense. We assert that the translation of a sketch into linguistic descriptions illustrates that the essential qualitative path knowledge has been extracted. The methodology is demonstrated using example sketches drawn on a handheld PDA.

A human-robot interface for a mobile robot was extended to include a discrete geodesic dome called a Sensory EgoSphere (SES) The SES is a two-dimensional data structure, centered on the robot's coordinate frame. The SES provides the robot's perspective of a remote environment via images, sonar, and laser range finder representations. It was proposed that the SES would enhance the general interface usability by decreasing perceived workload and increasing situational awareness. A human factors evaluation was performed to evaluate the established hypothesis. Novice users participated in the evaluation. The purpose of this paper is to review some of the evaluation results.

There is a desire to provide small, lightweight mobile interaction devices. This paper describes a PDA-based teleoperation interface for mobile robot. This interface differs from previous PDA teleoperation interfaces in that it uses only touch based interactions rather than stylus interaction. Three screens were developed: an imagery-only screen, a sonar and laser range finder based screen, and image with sensory overlay screen. This paper presents the interface motivation and design and discusses issues that arise when employing a PDA as a touch screen. This paper also describes the planned human factors evaluation.

A novel implicit communication framework in human-machine interaction that is sensitive to human affective states is presented in this paper. The focus is to achieve detection and recognition of human affect based on physiological signals. This involves building an affect recognition system that accepts as input various physiological parameters and predicts the probable related affective state. Both decision tree and fuzzy logic methodologies have been applied to this problem. This paper presents the results of the two methods and discusses their comparative merit. Three human subject experiments were designed and trials were conducted with six participants. The experimental results demonstrate the feasibility of the proposed implicit human-machine interaction framework.

We are developing a complex situational management application for non-steady state events. The domain in which we are working is polyester film base manufacturing. The complex situational management application is an expert system that monitors the non-steady state events and assists human operators with the event tasks. This application currently implements three event checklists and system monitoring logic that represent standard operating procedures for large, complex, film base machines.

Presents a collaborative effort to design and implement a cooperative material handling system by a small team of human and robotic agents in an unstructured indoor environment. The authors' approach makes fundamental use of the human agents' expertise for aspects of task planning, task monitoring, and error recovery. The authors' system is neither fully autonomous nor fully teleoperated. It is designed to make effective use of the human's abilities within the present state of the art of autonomous systems. The authors' robotic agents refer to systems which are each equipped with at least one sensing modality and which possess some capability for self-orientation and/or mobility. The authors' robotic agents are not required to be homogeneous with respect to either capabilities or function. The authors' research stresses both paradigms and testbed experimentation. Theory issues include the requisite coordination principles and techniques which are fundamental to a cooperative multiagent system's basic functioning. The authors have constructed an experimental distributed multiagent-architecture testbed facility. The required modular components of this testbed are currently operational and have been tested individually. The authors' current research focuses on the agents' integration in a scenario for cooperative material handling.