In this paper, we present an asynchronous display method, coined image queue, which allows operators to search through a large amount of data gathered by autonomous robot teams. We discuss and investigate the advantages of an asynchronous display for foraging tasks with emphasis on Urban Search and Rescue. The image queue approach mines video data to present the operator with a relevant and comprehensive view of the environment in order to identify targets of interest such as injured victims. It fills the gap for comprehensive and scalable displays to obtain a network-centric perspective for UGVs. We compared the image queue to a traditional synchronous display with live video feeds and found that the image queue reduces errors and operator's workload. Furthermore, it disentangles target detection from concurrent system operations and enables a call center approach to target detection. With such an approach we can scale up to very large multi-robot systems gathering huge amounts of data that is then distributed to multiple operators.

Controlling a team of Unmanned Aerial Vehicles (UAV) requires the operator to perform continuous surveillance and path planning. The operator's situation awareness is likely to degrade as an increasing number of surveillance videos must be viewed and integrated. The Picture-in-Picture display (PiP) provides one solution for integrating multiple UAV camera video by allowing the operator to view the video feed in the context of surrounding terrain. The experimental SUAVE (Simple Unmanned Areal Vehicle Environment) display extends PiP methods by sampling imagery from the video stream to texture a 3D map of the terrain. The operator can then inspect this imagery using world in miniature (WIM) or fly-through methods. We investigate the properties and advantages of SUAVE in the context of a search mission with 11 UAVs finding a strong advantage for finding targets While performance is expected to improve with increasing numbers of UAVs we did not find differences in performance between models generated by 11 UAVs and those employing 22 UAVs.

Technology for multirobot systems has advanced to the point where we can consider their use in a variety of important domains, including urban search and rescue. A key to the practical usefulness of multirobot systems is the ability to have a large number of robots effectively controlled by small numbers of operators. In this paper, two modalities for controlling a team of 24 robots in a foraging task in an urban search and rescue environment are compared. In both modalities, multiple operators must monitor video streams from the robots to detect and mark victims on a map as well as teleoperating robots that cannot get themselves out of difficult situations. In the first modality, the operators must also provide waypoints for the robots to explore, using both video and a partially completed map to choose appropriate waypoints. In the second modality, the robots autonomously plan their paths, allowing operators to focus on monitoring the video, but without being able to interpret video streams to guide exploration. Experimental results show that significantly better overall performance is achieved with autonomous path planning, although the reduction in operator workload is not significant.

In this paper, we address the problem of distributed path planning for large teams of hundreds of robots in constrained environments. We introduce two distributed prioritized planning algorithms: an efficient, complete method which is shown to converge to the centralized prioritized planner solution, and a sparse method in which robots discover collisions probabilistically. Planning is divided into a number of iterations, during which every robot simultaneously and independently computes a planning solution based on other robots' path information from the previous iteration. Paths are exchanged in ways that exploit the cooperative nature of the team and a statistical phenomenon known as the “birthday paradox”. Performance is measured in simulated 2D environments with teams of up to 240 robots. We find that in moderately constrained environments, these methods generate solutions of similar quality to a centralized prioritized planner, but display interesting communication and planning time characteristics.

We are developing a theory for human control of robot teams based on considering how control varies across different task allocations. Our current work focuses on domains such as foraging in which robots perform largely independent tasks. The present study addresses the interaction between automation and organization of human teams in controlling large robot teams performing an Urban Search and Rescue (USAR) task. We identify three subtasks: perceptual search-visual search for victims, assistance-teleoperation to assist robot, and navigation-path planning and coordination. For the studies reported here, navigation was selected for automation because it involves weak dependencies among robots making it more complex and because it was shown in an earlier experiment to be the most difficult. Two possible ways to organize operators were identified as assignment of robots to particular operators or as a shared pool in which operators service robots from the population as needed. The experiment compares two member teams of operators controlling teams of 12 robots each in the assigned robots conditions or sharing control of 24 robots in the shared pool conditions using either waypoint control or autonomous path planning. We identify three self organizing team strategies in the shared pool condition: joint control operators share full authority over robots, mixed control in which one operator takes primary control while the other acts as an assistant, and split control in which operators divide the robots with each controlling a subteam. Automating path planning improved system performance. Effects of team organization favored operator teams who shared authority for the pool of robots.

The present study investigates the effect of the number of controlled robots on performance of an urban search and rescue (USAR) task using a realistic simulation. Participants controlled either 4, 8, or 12 robots. In the fulltask control condition participants both dictated the robots' paths and controlled their cameras to search for victims. In the exploration condition, participants directed the team of robots in order to explore as wide an area as possible. In the perceptual search condition, participants searched for victims by controlling cameras mounted on robots following predetermined paths selected to match characteristics of paths generated under the other two conditions. By decomposing the search and rescue task into exploration and perceptual search subtasks the experiment allows the determination of their scaling characteristics in order to provide a basis for tentative task allocations among humans and automation for controlling larger robot teams. In the fulltask control condition task performance increased in going from four to eight controlled robots but deteriorated in moving from eight to twelve. Workload increased monotonically with number of robots. Performance per robot decreased with increases in team size. Results are consistent with earlier studies suggesting a limit of between 8-12 robots for direct human control.

Camera guided teleoperation has long been the preferred mode for controlling remote robots with other modes such as asynchronous control only used when unavoidable. Because controlling multiple robots places additional demands on the operator we hypothesized that removing the forced pace for reviewing camera video might reduce workload and improve performance. In an earlier experiment participants operated four teams performing a simulated urban search and rescue (USAR) task using a conventional streaming video plus map interface or an experimental interface without streaming video but with the ability to store panoramic images on the map to be viewed at leisure. Operators were more accurate in marking victims on maps using the conventional interface; however, ancillary measures suggested that the asynchronous interface succeeded in reducing temporal demands for switching between robots. This raised the possibility that the asynchronous interface might perform better if teams were larger. In this experiment we evaluate the usefulness of asynchronous video for teams of 4, 8, or 12 robots. Operators in the two conditions were equally successful in finding victims, however, the streaming video maintained its advantage for accuracy in locating victims.

We are developing an architecture for controlling robot teams based on considering how control difficulty for different tasks grows with increases in team size. Our analysis suggests that assignments of persons to commander (single commands to entire robot team), operator (commands to individual robots), and coordinator (control of interdependent robots) roles can lead to the most efficient organization. The ability to assign tasks within or between operators makes scheduling these interactions an important factor in team performance. Two possible ways to organize operators are through Individual Assignments of robots or as a Call Center in which operators service robots from the population as needed. In recent experiments we have found that participants performing an Urban Search And Rescue (USAR) foraging task using waypoint control were at or over their limits when controlling 12 robots. The present study uses the same robots, environment, and level of autonomy but with teams of two operators assigned to control 24 robots. These operators controlled teams of 12 robots in the Individual Assignment condition. In the Call Center condition operators shared control of the 24 robots. For this task and level of robot autonomy Individual Assignment participants performed marginally better searching larger regions but without finding more victims.

Camera guided teleoperation has long been the preferred mode for controlling remote robots, with other modes such as asynchronous control only used when unavoidable. In this experiment we evaluate the usefulness of asynchronous operation for a multirobot search task. Because controlling multiple robots places additional demands on the operator, removing the forced pace for reviewing camera video might reduce workload and improve performance. In the reported experiment participants operated four robot teams performing a simulated urban search and rescue (USAR) task using either conventional streaming video plus a map interface or an experimental interface without streaming video but with the ability to store panoramic images on the map to be viewed at leisure. Search performance was somewhat better using the conventional interface, however, ancillary measures suggest that the asynchronous interface succeeded in reducing temporal demands for switching between robots.

The present study investigates the effect of the number of controlled robots on performance of an urban search and rescue (USAR) task using a realistic simulation. Task performance increased in going from four to eight controlled robots but deteriorated in moving from eight to twelve. Workload increased monotonically with number of robots. Performance per robot decreased with increases in team size. Results are consistent with earlier studies suggesting a limit of between 8-12 robots for direct human control. This study demonstrates that these findings generalize to a more realistic setting and complex task.

Many applications require that a group of agents share a coherent distributed picture of the world given communication constraints. This paper describes an analysis and design methodology for coordination algorithms for extremely large groups of agents maintaining a distributed belief. This design methodology creates a probability distribution which relates global properties of the system to agent interaction dynamics using the tools of statistical mechanics. Using this probability distribution we show that this system undergoes a rapid phase transition between low divergence and high divergence in the distributed belief at a critical value of system temperature. We also show empirically that at the critical system temperature the number of messages passed and belief divergence between agents is optimal. Finally, we use this fact to develop an algorithm using system temperature as a local decision parameter for an agent.

This paper presents a decentralized approach to path planning for large numbers of autonomous vehicles in sparse environments. Unlike existing approaches, which are either computationally expensive or communication intensive, the presented approach allows large numbers of vehicles to plan independently with low communication overhead. The key to the algorithm is to observe that, in sparse environments, collisions are exceptional and that most of the time vehicles will simply not hit each other. Hence, it is reasonable to allow vehicles to plan independently and then resolve the small number of conflicts. We operationalize this by having each vehicle send their planned paths to a small number of their team mates via tokens. Each team member is required to check for conflicting paths that they have been informed about via a token and inform those involved when any conflict is detected. Both analytic and empirical results show that the approach has very high probability of detecting all potential collisions for large numbers of vehicles in both 2D and 3D environments.

A cooperative team's performance strongly depends on the view that the team has of the environment in which it operates. In a team with many autonomous vehicles and many sensors, there is a large volume of information available from which to create that view. However, typically communication bandwidth limitations prevent all sensor readings being shared with all other team members. This paper presents three policies for sharing information in a large team that balance the value of information against communication costs. Analytical and empirical evidence of their effectiveness is provided. The results show that using some easily obtainable probabilistic information about the team dramatically improves overall belief sharing performance. Specifically, by collectively estimating the value of a piece of information, the team can make most efficient use of its communication resources.

Adjustable autonomy refers to entities dynamically varying their own autonomy, transferring decision-making control to other entities (typically agents transferring control to human users) in key situations. Determining whether and when such transfers-of-control should occur is arguably the fundamental research problem in adjustable autonomy. Previous work has investigated various approaches to addressing this problem but has often focused on individual agent-human interactions. Unfortunately, domains requiring collaboration between teams of agents and humans reveal two key shortcomings of these previous approaches. First, these approaches use rigid one-shot transfers of control that can result in unacceptable coordination failures in multiagent settings. Second, they ignore costs (e.g., in terms of time delays or effects on actions) to an agent's team due to such transfers-of-control. To remedy these problems, this article presents a novel approach to adjustable autonomy, based on the notion of a transfer-of-control strategy. A transfer-of-control strategy consists of a conditional sequence of two types of actions: (i) actions to transfer decision-making control (e.g., from an agent to a user or vice versa) and (ii) actions to change an agent's pre-specified coordination constraints with team members, aimed at minimizing miscoordination costs. The goal is for high-quality individual decisions to be made with minimal disruption to the coordination of the team. We present a mathematical model of transfer-of-control strategies. The model guides and informs the operationalization of the strategies using Markov Decision Processes, which select an optimal strategy, given an uncertain environment and costs to the individuals and teams. The approach has been carefully evaluated, including via its use in a real-world, deployed multi-agent system that assists a research group in its daily activities.

As the science of building agents and agent based applications improves, agents are gradually making the transition from research laboratory prototypes to industrial applications. In an industrial setting pragmatic engineering issues related to the development and maintenance of agents come to the fore. In this paper a list of important engineering characteristics of agent architectures is presented. The list has the dual aims of providing evaluation criteria for agent users and design issues for agent architecture designers. Evaluation of an architecture with respect to the presented characteristics will allow industrial project managers to better assess the trade offs between architectures. Designing architectures that better fulfill the characteristics should lead to more industrially relevant architectures and successful agent applications. This list is designed to start discussion and raise awareness of industrial issues. It is hoped that over time good metrics and an evaluation methodology evolve which provide a robust framework for evaluating agent architectures with respect to industrial concerns.