As robots evolve into collaborators in human-robot teams, appropriately calibrated trust becomes crucial. This study investigates trust dynamics in an Augmented Reality-based human-robot teaming system, focusing on the interplay between robot performance and robot-to-human feedback. In an experiment with 32 participants, we examined how robot feedback influences user trust, particularly when it is mismatched with robot performance. The results show that while robot-to-human feedback does not significantly affect trust on its own, it positively affects user responses when matched to performance. Robot performance had a stronger influence on trust than feedback, and error severity significantly impacted trust levels. These findings contribute to understanding trust calibration in human-robot interactions and provide insight for designing effective trust-aware robotic systems, addressing critical gaps in existing research, and offering implications for improving human-robot collaboration across various domains.

Multi-robot teams can be very useful in a wide variety of search and rescue missions in challenging environments. In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states and update task assignments in human-robot teams as quickly as possible. We have developed an alert generation framework that can perform risk assessment and robot tasking suggestions to assist human supervisors. Our approach for task assignment suggestion generation combines heuristics-based task selection with forward simulation-based probabilistic assessment. As the characteristics of decision aids can largely vary human performance, an alert system may or may not improve decision-making. We aim to configure our framework with a goal to improve human decision-making performance. Towards that, we present some preliminary user studies and design reasoning, which informed our final comprehensive human subject study. We demonstrate in the study that supervisors can improve their decision-making abilities, make faster decisions, and increase mission performance by using our alert generation framework. Our empirical findings also show that our framework does not require significant training and that people with a higher level of trust in automation perform better when provided with alerts. We also find that people with certain personality traits such as high agreeableness and conscientiousness are the most benefited by alerts.

In a mission with significant uncertainty due to intermittent communications, delayed information flow, and robotic failures, the role of human supervisors is extremely challenging. As and when any new information arrives, humans must infer both the existing and predicted future states, identify potential contingencies, and update task assignments to robots rapidly. We propose methodologies for automated generation of task reallocation suggestions to humans to assist in the decision-making process. Our generated robot retasking plan minimizes a modified makespan of the mission, which incorporates task criticality and penalty for incomplete tasks. The plan considers the effects of potential mission contingencies on the tasks, the robots, and the future performance of the robots operating based on the previous task plans. Our method includes the incorporation of two optional tasks, i.e., relay and robot rescue, for performance improvement. The rescue task has probabilistic outcomes affecting the team size. One or more rescues are incorporated in a way that can minimize the expected value of the modified makespan overall possibilities of rescue outcomes. We have conducted performance evaluation using simulation, demonstrating the value of the optional tasks and performance enhancement using our method of incorporating them. Note to Practitioners—The work reported in this paper will be useful in applications where a team of agents is deployed to carry on a large-scale mission with communication constraints where the number of functional agents can change probabilistically. Typically, such uncertainty is encountered in applications that are challenging or dangerous in nature. Agents can have non-zero probabilities to fail while doing certain risky tasks and to get recovered by other agents. The proposed centralized multi-agent task reallocation method can help in proactively addressing potential contingencies in surveillance, search and rescue, or disaster relief to support resilient operations while having a supervisor in the higher chain of command.

The creation of information transparency solutions to enable humans to understand robot perception is a challenging requirement for autonomous and artificially intelligent robots to impact a multitude of domains. By taking advantage of comprehensive and high-volume data from robot teammates’ advanced perception and reasoning capabilities, humans will be able to make better decisions, with significant impacts from safety to functionality. We present a solution to this challenge by coupling augmented reality (AR) with an intelligent mobile robot that is autonomously detecting novel changes in an environment. We show that the human teammate can understand and make decisions based on information shared via AR by the robot. Sharing of robot-perceived information is enabled by the robot’s online calculation of the human’s relative position, making the system robust to environments without external instrumentation such as global positioning system. Our robotic system performs change detection by comparing current metric sensor readings against a previous reading to identify differences. We experimentally explore the design of change detection visualizations and the aggregation of information, the impact of instruction on communication understanding, the effects of visualization and alignment error, and the relationship between situated 3D visualization in AR and human movement in the operational environment on shared situational awareness in human-robot teams. We demonstrate this novel capability and assess the effectiveness of human-robot teaming in crowdsourced data-driven studies, as well as an in-person study where participants are equipped with a commercial off-the-shelf AR headset and teamed with a small ground robot that maneuvers through the environment. The mobile robot scans for changes, which are visualized via AR to the participant. The effectiveness of this communication is evaluated through accuracy and subjective assessment metrics to provide insight into interpretation and experience.

Estimating terrain traversability in off-road environments requires reasoning about complex interaction dynamics between the robot and these terrains. However, it is challenging to create informative labels to learn a model in a supervised manner for these interactions. We propose a method that learns to predict traversability costmaps by combining exteroceptive environmental information with proprioceptive terrain interaction feedback in a self-supervised manner. Additionally, we propose a novel way of incorporating robot velocity into the costmap prediction pipeline. We validate our method in multiple short and large-scale navigation tasks on challenging off-road terrains using two different large, all-terrain robots. Our short-scale navigation results show that using our learned costmaps leads to overall smoother navigation, and provides the robot with a more fine-grained understanding of the robot-terrain interactions. Our large-scale navigation trials show that we can reduce the number of interventions by up to 57% compared to an occupancy-based navigation baseline in challenging off-road courses ranging from 400 m to 3150 m. Appendix and full experiment videos can be found in our website: https://mateoguaman.github.io/hdif.

Robot data collected in complex real-world scenarios are often biased due to safety concerns, human preferences, and mission or platform constraints. Consequently, robot learning from such observational data poses great challenges for accurate parameter estimation. We propose a principled causal inference framework for robots to learn the parameters of a stochastic motion model using observational data. Specifically, we leverage the de-biasing functionality of the potential-outcome causal inference framework, the Inverse Propensity Weighting (IPW), and the Doubly Robust (DR) methods, to obtain a better parameter estimation of the robot's stochastic motion model. The IPW is a re-weighting approach to ensure unbiased estimation, and the DR approach further combines any two estimators to strengthen the unbiased result even if one of these estimators is biased. We then develop an approximate policy iteration algorithm using the bias-eliminated estimated state transition function. We validate our framework using both simulation and real-world experiments, and the results have revealed that the proposed causal inference-based navigation and control framework can correctly and efficiently learn the parameters from biased observational data.

Mixup-based data augmentation has been validated to be a critical stage in the self-training framework for unsupervised domain adaptive semantic segmentation (UDASS), which aims to transfer knowledge from a well-annotated (source) domain to an unlabeled (target) domain. Existing self-training methods usually adopt the popular region-based mixup techniques with a random sampling strategy, which unfortunately ignores the dynamic evolution of different semantics across various domains as training proceeds. To improve the UDA-SS performance, we propose an Informed Domain Adaptation (IDA) model, a self-training framework that mixes the data based on class-level segmentation performance, which aims to emphasize small-region semantics during mixup. In our IDA model, the class-level performance is tracked by an expected confidence score (ECS). We then use a dynamic schedule to determine the mixing ratio for data in different domains. Extensive experimental results reveal that our proposed method is able to outperform the state-of-the-art UDA-SS method by a margin of 1.1 mIoU in the adaptation of GTA-V to Cityscapes and of 0.9 mIoU in the adaptation of SYNTHIA to Cityscapes. Code link: https://github.com/ArlenCHEN/IDA.git

To safely traverse non-flat terrain, robots must account for the influence of terrain shape in their planned motions. Terrain-aware motion planners use an estimate of the vehicle roll and pitch as a function of pose, vehicle suspension, and ground elevation map to weigh the cost of edges in the search space. Encoding such information in a traditional two-dimensional cost map is limiting because it is unable to capture the influence of orientation on the roll and pitch estimates from sloped terrain. The research presented herein addresses this problem by encoding kinodynamic information in the edges of a recombinant motion planning search space based on the Efficiently Adaptive State Lattice (EASL). This approach, which we describe as a Kinodynamic Efficiently Adaptive State Lattice (KEASL), differs from the prior representation in two ways. First, this method uses a novel encoding of velocity and acceleration constraints and vehicle direction at expanded nodes in the motion planning graph. Second, this approach describes additional steps for evaluating the roll, pitch, constraints, and velocities associated with poses along each edge during search in a manner that still enables the graph to remain recombinant. Velocities are computed using an iterative bidirectional method using Eulerian integration that more accurately estimates the duration of edges that are subject to terrain-dependent velocity limits. Real-world experiments on a Clearpath Robotics Warthog Unmanned Ground Vehicle were performed in a non-flat, unstructured environment. Results from 2093 planning queries from these experiments showed that KEASL provided a more efficient route than EASL in 83.72% of cases when EASL plans were adjusted to satisfy terrain-dependent velocity constraints. An analysis of relative runtimes and differences between planned routes is additionally presented. These results reinforce the importance of considering kinodynamic constraints for motion planning in non-flat environments and illustrate how such information can be encoded in an adaptive recombinant motion planning search space.

Experimental design of autonomous systems involves defining experimental inputs to maximize the experimenter’s information gained, minimize costs, and balance risk. This effectively leads to improved understanding and trustworthiness, which are necessary for deployment in realworld settings. Since experimental design is inherently a human-in-the-loop, sequential decision making problem, and decisions are being made about complex systems, an investigation into decision-making quality and decision-supporting methods is warranted. In this work, we investigate a decision support system (DSS) to augment the human’s experimental design decision making abilities, and conduct an exploratory user study to investigate the potential for decision support. Our findings show that experimenters, including experienced field roboticists, make suboptimal decisions and mistakes during the experimental design process, which suggests robotics research could benefit from DSSs. Our proposed DSS shows promise in some select aspects of experimental design, including helping to reduce suboptimal decisions, and participants in the user study reported favorable opinions of using such a system, including a sense of usefulness and lack of burden. The broader implication of this work is the identification of decision support in experimental design as one way to help bridge the gap between academia and industry by way of accelerated, informative experimentation and increased system explainability.

Testing and evaluation of field robotic systems requires both experimentation in representative conditions and human supervision to effectively assess components, manage risk, and interpret results. Due to the complexity of robotic sys-tems, we argue this experimentation should be done adaptively by using insights gained from previous trials. Furthermore, we envision an advisory system that could assist experimenters with selecting trial configurations by learning and accounting for human preferences and risk tolerances; however, formal methods for human decision making in the context of field robotic experimentation remains an open question. In this work, we present and analyze a case study for how decisions were made during the testing and evaluation of an off-road, autonomous navigation system. From the perspective of active learning, we find that Bayesian Optimization is a promising mathematical framework for modeling human decision making in adaptive experimental design of field robotics and that a combination of the EI, KG, and PES acquisition functions would likely be useful for realizing an advisory system.

This paper presents an approach and introduces new open-source tools that can be used to evaluate robotic mapping algorithms. Also described is an extensive subterranean mine rescue dataset based upon the DARPA Subterranean (SubT) challenge including professionally surveyed ground truth. Finally, some commonly available approaches are evaluated using this metric.

In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states, and update task assignments to robots as quickly as possible. We present a forward simulation-based alert system that proactively notifies the human supervisor of possible, negatively-impactful events, which provides an opportunity for the human to retask agents to avoid undesirable scenarios. We propose methods for speeding up mission simulations and extracting alerts from simulation data in order to enable real-time alert generation suitable for time-critical missions. We present the results from a user trial and verify our hypothesis that the decision making performance of human supervisors can be improved by introducing forward simulation-based alerts.

We propose a scalable, parallelizable policy synthesis framework intended for a robot presented with the decision of exploration or rescue, given some time-varying, stochastic mission conditions, referred to as context. We demonstrate the feasibility of such a solution using physics-based simulations to synthesize a policy in a computationally-efficient manner and exhibit superior performance with regards to the minimization of probability of mission failure when compared to two feasible baseline approaches. Furthermore, we present preliminary results that suggest our approach is robust to errors in the state estimation used to build mission context, which further supports the notion of real-world applicability.

Autonomous platforms are confronted by a diversity of challenges in unstructured environments, which make monitoring performance a non-trivial task. Some of these environments are so complex that they preclude persistent, nearby operator oversight. This absence of oversight motivates the need for an online monitoring system, specifically for robots operating in difficult environments. We develop a test methodology and set of online monitoring metrics by extending methods for characterizing robotic-systems using offline metrics. We implement this test methodology in an unstructured, outdoor environment and show the resulting performance information gained from our online monitoring solution. This online monitoring approach is generalizable such that it characterizes any robotic system that meets our set of hardware and software criteria.