Multi-robot teams can be very useful in a wide variety of search and rescue missions in challenging environments. In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states and update task assignments in human-robot teams as quickly as possible. We have developed an alert generation framework that can perform risk assessment and robot tasking suggestions to assist human supervisors. Our approach for task assignment suggestion generation combines heuristics-based task selection with forward simulation-based probabilistic assessment. As the characteristics of decision aids can largely vary human performance, an alert system may or may not improve decision-making. We aim to configure our framework with a goal to improve human decision-making performance. Towards that, we present some preliminary user studies and design reasoning, which informed our final comprehensive human subject study. We demonstrate in the study that supervisors can improve their decision-making abilities, make faster decisions, and increase mission performance by using our alert generation framework. Our empirical findings also show that our framework does not require significant training and that people with a higher level of trust in automation perform better when provided with alerts. We also find that people with certain personality traits such as high agreeableness and conscientiousness are the most benefited by alerts.

We introduce the Situated Corpus Of Understanding Transactions (SCOUT), a multi-modal collection of human-robot dialogue in the task domain of collaborative exploration. The corpus was constructed from multiple Wizard-of-Oz experiments where human participants gave verbal instructions to a remotely-located robot to move and gather information about its surroundings. SCOUT contains 89,056 utterances and 310,095 words from 278 dialogues averaging 320 utterances per dialogue. The dialogues are aligned with the multi-modal data streams available during the experiments: 5,785 images and 30 maps. The corpus has been annotated with Abstract Meaning Representation and Dialogue-AMR to identify the speaker’s intent and meaning within an utterance, and with Transactional Units and Relations to track relationships between utterances to reveal patterns of the Dialogue Structure. We describe how the corpus and its annotations have been used to develop autonomous human-robot systems and enable research in open questions of how humans speak to robots. We release this corpus to accelerate progress in autonomous, situated, human-robot dialogue, especially in the context of navigation tasks where details about the environment need to be discovered.

Human-guided robotic exploration is a useful approach to gathering information at remote locations, especially those that might be too risky, inhospitable, or inaccessible for humans. Maintaining common ground between the remotely-located partners is a challenge, one that can be facilitated by multi-modal communication. In this paper, we explore how participants utilized multiple modalities to investigate a remote location with the help of a robotic partner. Participants issued spoken natural language instructions and received from the robot: text-based feedback, continuous 2D LIDAR mapping, and upon-request static photographs. We noticed that different strategies were adopted in terms of use of the modalities, and hypothesize that these differences may be correlated with success at several exploration sub-tasks. We found that requesting photos may have improved the identification and counting of some key entities (doorways in particular) and that this strategy did not hinder the amount of overall area exploration. Future work with larger samples may reveal the effects of more nuanced photo and dialogue strategies, which can inform the training of robotic agents. Additionally, we announce the release of our unique multi-modal corpus of human-robot communication in an exploration context: SCOUT, the Situated Corpus on Understanding Transactions.

Information uncertainty is ubiquitous in everyday life, including in domains as diverse as weather forecasts, investments, and health risks. Knowing how to interpret and integrate this uncertain information is vital for making good decisions, but this can be difficult for experts and novices alike. In this study, we examine whether brief, focused practice can improve people's ability to understand and integrate bivariate Gaussian uncertainty visualized via ensemble displays, summary displays, and distributional displays, and we examine whether this is influenced by the complexity of the displayed information. In two experiments (N=118 and 56), decision making was faster and more accurate after practice relative to before practice. Furthermore, the performance improvements transferred to use of display types that were not practiced. This suggests that practice with feedback may improve underlying skills in probabilistic reasoning and provides a promising approach to improve people's decision making under uncertainty.

Scalable systems can increase or decrease their size with costs that are proportionate to the resulting change in performance. These costs can be monetary or related to other factors such as integration effort, operator training, or infrastructure upgrades. The options to increase in size to meet growing demand and decrease in size to minimize costs while servicing low demand make scalable systems attractive for completing tasks under uncertainty. Multirobot systems are used in many tasks characterized by uncertainty, such as search and rescue, mapping, and perimeter defense. When human operators interact with multirobot systems, the scalability can be limited by the human's cognitive abilities, decision making speed, and performance under stress. Human problem solving and creativity can also be beneficial to the system to overcome potential scaling challenges. This paper summarizes the literature on humans' effects on scalability of multirobot systems and presents an illustrative simulation of the challenges of scaling a multioperator, multirobot surveillance system.

This paper identifies stylistic differences in instruction-giving observed in a corpus of human-robot dialogue.Differences in verbosity and structure (i.e., single-intent vs. multi-intent instructions) arose naturally without restrictions or prior guidance on how users should speak with the robot.Different styles were found to produce different rates of miscommunication, and correlations were found between style differences and individual user variation, trust, and interaction experience with the robot.Understanding potential consequences and factors that influence style can inform design of dialogue systems that are robust to natural variation from human users.