In this article, we investigate using explanations to improve decision-making for robotic scientific data collection missions. We propose the Preference Elicitation with contrasting Feature-based eXplanations (PrEFeX) method, which combines preferences with contrasting explanations focused on a single explanatory feature. We first provide a verification of the contrasting explanations by themselves using a planner prediction user study with 16 expert participants (Phase 1). This study showed that there was no increase in understanding of robot plans using contrasting explanations without preferences. To elucidate what information the autonomous measurement selection system was missing to be useful, we interviewed 4 planetary scientists and 2 oceanographers (Phase 2). We found that scientists focused heavily on understanding the objectives of the system and wanted explanations that (1) grounded the explanations to tradeoffs the system made, (2) connected the explanations to an ability to modify the behavior of the decision making, and (3) attached the explanation system’s features to comparable features the scientists considered. To this end, we propose combining explanations with user preference learning of the reward function in an iterative design process of the measurement plans with scientists in the loop. We tested our proposed preference and explanation system in a field deployment with planetary scientists on Mt. Hood, Oregon, and performed a post-data collection survey on the quality of the plans with 22 experts (Phase 3). We found the experts preferred the measurement plan selected by our proposed PrEFeX method over a baseline without explanations or preferences.

Cooperative manipulation tasks impose various structure-, task-, and robot-specific constraints on mobile manip-ulators. However, current methods struggle to model and solve these myriad constraints simultaneously. We propose a twofold solution: first, we model constraints as a family of manifolds amenable to simultaneous solving. Second, we introduce the constrained nonlinear Kaczmarz (cNKZ) projection technique to produce constraint-satisfying solutions. Experiments show that cNKZ dramatically outperforms baseline approaches, which cannot find solutions at all. We integrate cNKZ with a sampling-based motion planning algorithm to generate complex, coordi-nated motions for 3–6 mobile manipulators (18–36 DoF), with cNKZ solving up to 80 nonlinear constraints simultaneously and achieving up to a 92% success rate in cluttered environments. We also demonstrate our approach on hardware using three Thrtlebot3 Waffle Pi robots with OpenMANIPULATOR-X arms.

We present a hybrid centralized-decentralized planning algorithm for a multi-robot system consisting of a Mothership robot and multiple Passenger robots. In this system, the Passenger robots execute tasks while the Mothership provides support. This paper addresses the challenge of planning Passenger robot movements, framing it as a Stochastic Multi-Agent Orienteering Problem (SMOP) complicated by factors like stochastic operational efforts and disruptive events. We optimize the task completion efficiency of the system by combining centralized solutions from the Mothership with local plans from Passengers to enhance system resilience. Our contributions include defining the SMOP, developing a solution using Decentralized Monte Carlo Tree Search, presenting a hybrid algorithm that integrates centralized plans into the distributed framework, and evaluating the algorithm's performance in a simulated environment. Our results show that our hybrid approaches outperform fully centralized and fully distributed algorithms in highly-dynamic scenarios with up to a 26.6 % increase in task completion efficiency over baseline methods.

In early motor interventions from clinical rehabilitation to physical activity encouragement, one major challenge is maintaining child engagement and motivation. Robots show unique promise for addressing this challenge, but providing robots with new types of autonomous functionality is vital for promoting robot integration and usefulness in the clinic and home spaces. To provide needed autonomy capabilities for GoBot, our assistive robot for child–robot motion interventions, we propose a behavior tree framework. Within our framework, we build two trees: one manually designed based on expert knowledge of the child–robot interaction domain, and a second automatically synthesized and requiring minimal human input and time to construct. We tested each behavior tree with N = 11 children who interacted with GoBot during two behavior tree phases and a stationary-robot control phase. Our results show that both behavior tree phases tended to yield more child motion and significantly higher parent perception of child engagement, compared to the control phase. We showed that GoBot, equipped with our framework, has the potential to encourage movement and interaction in children and that a synthesized tree can be competitive with a manually designed tree. The products of this work can benefit researchers of behavior trees and child–robot interaction.

Developing autonomous intervention capabilities for lightweight underwater vehicle manipulator systems (UVMS) has garnered significant attention within recent years because of the opportunity for these systems to reduce intervention operating costs. Developing autonomous UVMS capabilities is challenging, however, because of the lack of available standardized software frameworks and pipelines. Previous works offer simulation environments and deployment pipelines for underwater vehicles, but fall short of providing a complete UVMS software framework. We address this gap by creating Angler: a software framework for developing localization, control, and decision-making algorithms with support for sim-to-real transfer. We validate this framework by implementing a state-of-the-art control architecture and demonstrate the ability to perform station keeping with a mean error below 0.25 m and waypoint tracking with an average final error of 0.398 m.

Underwater vehicle manipulator systems (UVMS) are increasingly popular platforms for performing subsea operations that require precision manipulation. While there is high demand for fully autonomous or even semi-autonomous systems, most UVMS still require human support teams. Developing new hardware and algorithms for autonomous underwater manipulation is challenging. Simulations do not capture the full complexity of the underwater environment, and deploying a UVMS at sea for testing/validation is resource-intensive and expensive. In this paper, we present a physical testbed for underwater manipulation that bridges the gap between simulation and full field trials. The underWater Arm-Vehicle Emulator (WAVE) is a 10-degree of freedom system designed to replicate an inspection-class UVMS. WAVE includes an underwater perception sensor and has 2 operating modes: rigid or passive-mode. In passive-mode, the ROV body can pitch similar to how a dynamically-coupled underactuated UVMS without pitch control would rotate during manipulation tasks. To validate the overall design and passive pitch concept, we evaluated the testbed during underwater experiments in energetic conditions at a wave basin. To support continued research and development in underwater robotics, we make the design open-access and freely available to the community.

Persistent operation of Autonomous Underwater Vehicles (AUVs) without manual interruption for recharging saves time and total cost for offshore monitoring and data collection applications. In order to facilitate AUVs for long mission durations without ship support, they can be equipped with docking capabilities to recharge in situ at Wave Energy Converter (WEC) with dock recharging stations. However, the power generated at the recharging stations may be constrained depending on the sea conditions. Therefore, a robust mission planning framework is proposed using a centralized Evolutionary Algorithm (EA) and a decentralized Monte Carlo Tree Search (MCTS) method. Both methods incorporate the charge availability constraint at the recharging station in addition to the maximum charge capacity of each AUV. The planner utilizes a time-varying power profile of irregular waves incident at WECs for dock charging and generates efficient mission plans for AUVs by optimizing their time to visit the dock based on the imposed constraint. The effects of increasing the number of AUVs, increasing the number of points of interest in the mission area, and varying sea state on the mission duration are also analyzed.

Real-time sequential decision making under uncertainty is a challenging task for autonomous robots. Such problems are even more challenging when making decisions involving heterogeneous teams of robots completing multiple tasks. Deploying autonomous taxi cabs and utilizing drones for package delivery represent relevant examples of these types of problems. In this paper, we present an effective solution to a multi-robot multi-task sequential stochastic assignment problem using a simulation-based optimization algorithm (MARP). Our algorithm employs a novel approach that uses Monte Carlo simulation to seek the deployment with the highest probability of being optimal. To demonstrate MARP's performance and robustness, we performed more than 2,000 numerical experiments in two different problem domains, evaluating MARP's performance against three different comparison algorithms. These numerical studies show that MARP significantly outperforms the comparison methods, achieving results within 5% of the maximum possible reward.

Robots performing mobile manipulation in unstructured environments must identify grasp affordances quickly and with robustness to perception noise. Yet in domains such as underwater manipulation, where perception noise is severe, computation is constrained, and the environment is dynamic, existing techniques fail. They are too computationally demanding, or too sensitive to noise to allow for closed loop grasping or dynamic replanning, or do not consider 6-DOF grasps. We present a novel grasp synthesis network, TSGrasp, that uses spatio-temporal sparse convolution to process a streaming point cloud in real time. The network generates 6-DOF grasps at greater speed and with less memory than Contact GraspNet, a state-of-the-art algorithm based on Point-Net++. By considering information from multiple successive frames of depth video, TSGrasp boosts robustness to noise or temporary self-occlusion and allows more grasps to be rapidly identified. Our grasp synthesis system was successfully demonstrated in an underwater environment with a Blueprint Labs Bravo robotic arm.

We present a navigation framework to perform autonomous underwater docking to a wave energy converter (WEC) under various ocean conditions by incorporating flow state estimation into the design of model predictive control (MPC). Existing methods lack the ability to perform dynamic rendezvous and autonomously dock in energetic conditions. The use of exteroceptive sensors or high performing acoustic sensors have been previously investigated to obtain or estimate the flow states. However, the use of such sensors increases the overall cost of the system and expects the vehicle to navigate close to the seafloor or other landmarks. To overcome these limitations, our method couples an active perception framework with MPC to estimate the flow states simultaneously while moving towards the dock. Our simulation results demonstrate the robustness and reliability of the proposed framework for autonomous docking under various ocean conditions. Furthermore, we conducted laboratory trials with a BlueROV2 docking with an oscillating dock and achieved a greater than 70% success rate.

Soft manipulators made of deformable materials have great promise in applications that require additional flexibility and compliance; however, these characteristics also make them difficult to simulate accurately and quickly. The lack of a fast and accurate simulator prevents motion planners from generating feasible plans, which would enable soft robots to achieve more complex tasks, such as manipulation. In this work, we propose combining a simplified quasistatic model with a neural network that learns to compensate for unmodeled forces, such as friction and loads, in order to create a fast forward model for soft manipulators with multiple segments. We show that the resulting neural network model reduces average end effector position error by 62% compared to the quasistatic model, while still being fast enough for motion planning. We also incorporate this model into an RRT*-based planner and demonstrate that the plans generated using our model are more likely to be feasible when executed on hardware than plans generated with a simulator using the quasistatic model.

We present a new algorithm for deploying passenger robots in marsupial robot systems. A marsupial robot system consists of a carrier robot (e.g., a ground vehicle), which is highly capable and has a long mission duration, and at least one passenger robot (e.g., a short-duration aerial vehicle) transported by the carrier. We optimize the performance of passenger robot deployment by proposing an algorithm that reasons over uncertainty by exploiting information about the prior probability distribution of features of interest in the environment. Our algorithm is formulated as a solution to a sequential stochastic assignment problem (SSAP). The key feature of the algorithm is a recurrence relationship that defines a set of observation thresholds that are used to decide when to deploy passenger robots. Our algorithm computes the optimal policy in O(NR) time, where N is the number of deployment decision points and R is the number of passenger robots to be deployed. We conducted drone deployment exploration experiments on real-world data from the DARPA Subterranean challenge to test the SSAP algorithm. Our results show that our deployment algorithm outperforms other competing algorithms, such as the classic secretary approach and baseline partitioning methods, and is comparable to an offline oracle algorithm.

This paper presents a framework that uses language instructions to define the constraints and objectives for robots gathering information about their environment. Designing autonomous robotic sampling missions requires deep knowledge of both autonomy systems and scientific domain expertise. Language commands provide an intuitive interface for operators to give complex instructions to robots. The key insight we leverage is using topological constraints to define routing directions from the language instruction such as ‘route to the left of the island.’ This work introduces three main contributions: a framework to map language instructions to constraints and rewards for robot planners, a topology constrained information gathering algorithm, and an automatic semantic feature detection algorithm for upwelling fronts. Our work improves on existing methods by not requiring training data with language instruction to planner constraint pairs, allowing new robotic domains such as marine robotics to use our method. This paper provides results demonstrating our framework producing correct constraints for 84.6% of instructions, from a systematically generated corpus of over 1.1 million instructions We also demonstrate the framework producing robot plans from language instructions for real-world scientific sampling missions with the Slocum underwater glider.

We present an algorithm for learning behavior trees for robotic task planning, which alleviates the need for time-intensive or infeasible manual design of control architectures. Our method involves representing the search space of behavior trees as a formal grammar and searching over this grammar by means of a new generalization of Monte Carlo tree search (MCTS) for directed acyclic graphs (DAGs), named MCDAGS. Additionally, our method employs simulated annealing to expedite the aggregation of the most functional subtrees. We present simulated experiments for a marine target search and response scenario, and an abstract task selection problem. Our results demonstrate that the learned behavior trees compare favorably with a manually-designed tree, and outperform baseline learning methods. Overall, these results show that our method is a viable technique for the automatic design of behavior trees for robotic task planning.

Robots often require a model of their environment to make informed decisions. In unknown environments, the ability to infer the value of a data field from a limited number of samples is essential to many robotics applications. In this work, we propose a neural network architecture to model these spatially correlated data fields based on a limited number of spatially continuous samples. Additionally, we provide a method based on biased loss functions to suggest future areas of exploration to minimize reconstruction error. We run simulated robotic information gathering trials on both the MNIST hand written digits dataset and a Regional Ocean Modeling System (ROMS) ocean dataset for ocean monitoring. Our method outperforms Gaussian process regression in both environments for modeling the data field and action selection.

In this work, we present a novel approach for modeling, and classifying between, the system load states introduced when constructing staged soft arm configurations. Through a two stage approach: (1) an LSTM calibration routine is used to identify the current load state then (2) a control input generation step combines a generalized quasistatic model with the learned load model. Our experiments show that accounting for system load allows us to more accurately control tapered arm configurations. We analyze the performance of our method using soft robotic actuators and show it is capable of classifying between different arm configurations at a rate greater than 95%. Additionally, our method is capable of reducing the end-effector error of quasistatic model only control to within 1 cm of our controller baseline.

This paper presents a new coordination algorithm for decentralised multi-robot information gathering. We consider planning for an online variant of the multi-agent orienteering problem with neighbourhoods. This formulation closely aligns with a number of important tasks in robotics, including inspection, surveillance, and reconnaissance. We propose a decentralised variant of the self-organising map (SOM) learning procedure, named Dec-SOM, which efficiently plans sequences of waypoints for a team of robots. Decentralisation is achieved by performing a distributed allocation scheme jointly with a series of SOM adaptations. We also offer an efficient heuristic to select when to perform negotiations, which reduces communication resource usage. Simulation results in two settings, including an infrastructure inspection scenario with a real-world dataset of oil rigs, demonstrate that Dec-SOM outperforms baseline methods and other SOM variants, is competitive with centralised SOM, and is a viable solution for decentralised information gathering.

In this paper, we present a novel algorithm for constructing a maximally informative path for a robot in an information gathering task. We use a Self-Organizing Map (SOM) framework to discover important topological features in the information function. Using these features, we identify a set of distinct classes of trajectories, each of which has improved convexity compared with the original function. We then leverage a Stochastic Gradient Ascent (SGA) optimization algorithm within each of these classes to optimize promising representative paths. The increased convexity leads to an improved chance of SGA finding the globally optimal path across all homotopy classes. We demonstrate our approach in three different simulated experiments. First, we show that our SOM is able to correctly learn the topological features of a gyre environment with a well-defined topology. Then, in the second set of experiments, we compare the effectiveness of our algorithm in an information gathering task across the gyre world, a set of randomly generated worlds, and a set of worlds drawn from real-world ocean model data. In these experiments our algorithm performs competitively or better than a state-of-the-art Branch and Bound while requiring significantly less computation time. Lastly, the final set of experiments show that our method scales better than the comparison methods across different planning mission sizes in real-world environments.

Robotic exploration requires adaptively selecting navigation goals that result in the rapid discovery and mapping of an unknown world. In many real-world environments, subtle structural cues can provide insight about the unexplored world, which may be exploited by a decision maker to improve the speed of exploration. In sparse subterranean tunnel networks, these cues come in the form of topological features, such as loops or dead-ends, that are often common across similar environments. We propose a method for learning these topological features using techniques borrowed from topological image segmentation and image inpainting to learn from a database of worlds. These world predictions then inform a frontier-based exploration policy. Our simulated experiments with a set of real-world mine environments and a database of procedurally-generated artificial tunnel networks demonstrate a substantial increase in the rate of area explored compared to techniques that do not attempt to predict and exploit topological features of the unexplored world.

In this work we address the challenge of predicting the missing dimension (elevation angle) from 2D underwater sonar images. The high noise levels in these images, from phenomena such as non-diffuse reflections, frequently limits the usefulness of physical models. We thus propose the utilization of Convolutional Neural Networks (CNNs) as a powerful method to extract meaningful information without being misled by noisy data. We also introduce a self-supervised method that uses the physics of the sonar sensor to train the network on real data without ground-truth elevation maps. Our method can produce accurate elevation angle estimates given only a single image. Finally, we demonstrate that our method produces more accurate 3D reconstructions than competing methods, both in simulation and on real data.

In this work we develop a novel framework that enables the real-time 3D reconstruction of underwater environments using features from 2D sonar images. Due to noisy and low-resolution imagery as compared with standard cameras, automatic feature extractors for sonar images are not reliable in many scenarios. Thus, a human often needs to hand-select features in sonar imagery for environment reconstructions. Given the high data capture rates of standard imaging sonars (on the order of 20Hz), hand-annotating the features in every frame cannot be done in real-time. To address this we use a Convolutional Neural Network (CNN) that analyzes incoming imagery in real-time and proposes only a small subset of high-quality frames to the user for feature annotation. We demonstrate that our approach provides real-time reconstruction capability without loss in classification performance on datasets captured onboard our underwater vehicle while operating in a variety of environments.

In this work, we present a novel method for constructing a topological map of biological hotspots in an aquatic environment using a Fast Marching-based Voronoi segmentation. Using this topological map, we develop a closed form solution to the scheduling problem for any single path through the graph. Searching over the space of all paths allows us to compute a maximally informative path that traverses a subset of the hotspots, given some budget. Using a greedy-coverage algorithm we can then compute an informative path. We evaluate our method in a set of simulated trials, both with randomly generated environments and a real-world environment. In these trials, we show that our method produces a topological graph which more accurately captures features in the environment than standard thresholding techniques. Additionally, We show that our method can improve the performance of a greedy-coverage algorithm in the informative path planning problem by guiding it to different informative areas to help it escape from local maxima.

In this work, we present a Stochastic Gradient Ascent (SGA) algorithm for multi-vehicle information gathering that accounts for limitations on a vehicle's control authority caused by external forces. By representing vehicle paths using a novel action space representation, rather than a state space representation, we remove the need to perform feasibility calculations on the vehicle's path. Our algorithm uses a stochastic optimization scheme by sampling perturbed action sequences around the current best known sequence to estimate the gradient of a state space information function with respect to the action sequence. Additionally, we use sequential greedy allocation to plan for multiple vehicles. Results are shown using a Navy Coastal Ocean Model (NCOM) for the Gulf of Mexico (GoM). SGA shows improvement in the amount of information gained over a greedy baseline. Additionally, we compare to Monte Carlo Tree Search (MCTS) Method, which is able to gather competitive amounts of information but is more computationally intensive than our approach.

The self-organizing map (SOM) is an unsupervised learning technique providing a transformation of a high-dimensional input space into a lower dimensional output space. In this paper, we utilize the SOM for the traveling salesman problem (TSP) to develop a solution to autonomous data collection. Autonomous data collection requires gathering data from predeployed sensors by moving within a limited communication radius. We propose a new growing SOM that adapts the number of neurons during learning, which also allows our approach to apply in cases where some sensors can be ignored due to a lower priority. Based on a comparison with available combinatorial heuristic algorithms for relevant variants of the TSP, the proposed approach demonstrates improved results, while also being less computationally demanding. Moreover, the proposed learning procedure can be extended to cases where particular sensors have varying communication radii, and it can also be extended to multivehicle planning.

In this paper, we present a method to improve the navigation of tethered underwater vehicles by computing optimal paths that prevent their tethers from becoming entangled in obstacles. To accomplish this, we define the Non-Entangling Travelling Salesperson Problem (NE-TSP) as an extension of the Travelling Salesperson Problem with a non-entangling constraint. We compute the optimal solution to the NE-TSP by constructing a Mixed Integer Programming model, leveraging homotopy augmented graphs to plan an optimal trajectory through a set of inspection points, while maintaining a non-entangling guarantee. To avoid the computational expense of computing an optimal solution to the NE-TSP, we also introduce several methods to compute near-optimal solutions. In a set of simulated trials, our method was able to plan optimal non-entangling paths through a variety of environments. These results were then validated in a set of pool and field trials using a Seabotix vLBV300 underwater vehicle. The paths generated by our method were then compared to human-generated paths.

In shared autonomy, a robot and human user both have some level of control in order to achieve a shared goal. Choosing the balance of control given to the user and the robot can be a challenging problem since different users have different preferences and vary in skill levels when operating a robot. We propose using a novel formulation of Partially Observable Markov Decision Process (POMDP) to represent a model of the user's expertise in controlling the robot. The POMDP uses observations from the user's actions and from the environment to update the belief of the user's skill and chooses a level of control between the robot and the user. The level of control given between the user and the robot is encapsulated in macro-action controllers. A user study was run to test the performance of our formulation. Users drive a simulated robot through an obstacle-filled map while the POMDP model chooses appropriate macro-action controllers based on the belief state of the user's skill level. The results of the user study show that our model can encapsulate user skill. The results also show that using the controller with greater robot autonomy helped users of low skill avoid obstacles more than it helped users of high skill.

Robots often operate in built environments containing underlying structure that can be exploited to help predict future observations. In this work, we present a deep learning based approach to predict exit locations of buildings. This technique exploits the inherent structure of buildings to create a model. A convolutional neural network is trained using a database of building blueprints and used to guide a search within a building. This technique is compared to standard frontier exploration and a traditional image processing approach of extracting features through histogram of gradients (HOG) and training a support vector machine (SVM). After validation through simulation, we show that the proposed deep learning technique reduces the amount of building exploration required to find the goal by 36%.

We propose a multi-robot exploration algorithm that uses adaptive coordination to provide heterogeneous behavior. The key idea is to maximize the efficiency of exploring and mapping an unknown environment when a team is faced with unreliable communication and limited battery life (e.g., with aerial rotorcraft). The proposed algorithm utilizes four states: explore, meet, sacrifice, and relay. The explore state uses a frontier-based exploration algorithm, the meet state returns to the last known location of communication to share data, the sacrifice state sends the robot out to explore without consideration of remaining battery, and the relay state lands the robot until a meeting occurs. This approach allows robots to take on the role of a relay to improve communication between team members. In addition, the robots can “sacrifice” themselves by continuing to explore even when they do not have sufficient battery to return to the base station. We compare the performance of the proposed approach to state-of-the-art frontier-based exploration, and results show gains in explored area. The feasibility of components of the proposed approach is also demonstrated on a team of two custom-built quadcopters exploring an office environment.

We present a coactive algorithm for learning a human expert's preferences in planning trajectories for information gathering in scientific autonomy domains. The algorithm learns these preferences by iteratively presenting solutions to the expert and updating an estimated utility function based on the expert's improvements. We apply these algorithms, in the context of underwater data collection, using a pair of risk and reward maps. In simulated trials, the algorithm successfully learns the underlying weighting behind a utility map used by a human planning trajectories. We also present experimental trials demonstrating the algorithm using a temperature and depth monitoring task in an inland lake with an autonomous surface vehicle. This work shows it is possible to design algorithms for autonomous navigation with reward functions that capture the essence of a human's preferences.

In this paper we consider the problem of routing autonomously guided robots by manipulating the cost space to induce safe trajectories in the work space. Specifically, we examine the domain of UAV traffic management in urban airspaces. Each robot does not explicitly coordinate with other vehicles in the airspace. Instead, the robots execute their own individual internal cost-based planner to travel between locations. Given this structure, our goal is to develop a high-level UAV traffic management (UTM) system that can dynamically adapt the cost space to reduce the number of conflict incidents in the airspace without knowing the internal planners of each robot. We propose a decentralized and distributed system of high-level traffic controllers that each learn appropriate costing strategies via a neuro-evolutionary algorithm. The policies learned by our algorithm demonstrated a 16.4% reduction in the total number of conflict incidents experienced in the airspace while maintaining throughput performance.

This paper presents novel data fusion methods that enable teams of vehicles to perform target search tasks without guaranteed communication. Techniques are introduced for merging estimates of a target's position from vehicles that regain contact after long periods of time, and a fully distributed team-planning algorithm is proposed, which utilizes limited shared information as it becomes available. The proposed data fusion techniques are shown to avoid overcounting information, which ensures that combining data from different vehicles will not decrease the performance of the search. Motivated by the underwater search domain, a realistic underwater acoustic communication channel is used to determine the probability of successful data transfer between two locations. The channel model is integrated into a simulation of multiple autonomous vehicles in both open water and harbor environments. The results demonstrate that the proposed distributed coordination techniques provide performance competitive with full communication.

We propose an integrated learning and planning framework that leverages knowledge from a human user along with prior information about the environment to generate trajectories for scientific data collection. The proposed framework combines principles from probabilistic planning with uncertainty modeling through nonparametric Bayesian methods to refine trajectories for execution by autonomous vehicles. The resulting techniques allow for trajectories specified by a user to be modified for reduced risk of collision and increased reliability. We test our approach in the underwater ocean monitoring domain, and we show that the proposed framework reduces the risk of collision with ship traffic by as much as 51% for an autonomous underwater vehicle operating in ocean currents. This work provides insight into the tools necessary for combining human-robot interaction with autonomous navigation.

In this paper, we propose a framework for solving variants of the multi-goal path planning problem with applications to autonomous data collection. Autonomous data collection requires optimizing the trajectory of a mobile vehicle to collect data from a number of stationary sensors in a known configuration. The proposed approach utilizes the self-organizing map (SOM) architecture to provide a unified solution to multi-goal path planning problems. Our approach applies to cases where the vehicle must move within a radius of a sensor to collect data and also where some sensors can be ignored due to a lower priority. We compare our proposed approach to state-of-the-art approximate solutions to variants of the Traveling Salesman Problem (TSP) for random deployments and in an underwater monitoring application domain. Our results demonstrate that the SOM approach outperforms combinatorial heuristic algorithms and also provides a unified approach for solving variants of the multi-goal path planning problem.

We discuss the problem of learning uncertainty models of ocean processes to assist in the operation of Autonomous Underwater Vehicles (AUVs) in the ocean. We focus on the prediction of ocean currents, which have significant effect on the navigation of AUVs. Available models provide accurate prediction of ocean currents, but they typically do not provide confidence estimates of these predictions. We propose augmenting existing prediction methods with variance measures based on Gaussian Process (GP) regression. We show that commonly used measures of variance in GPs do not accurately reflect errors in ocean current prediction, and we propose an alternative uncertainty measure based on interpolation variance. We integrate these measures of uncertainty into a probabilistic planner running on an AUV during a field deployment in the Southern California Bight. Our experiments demonstrate that the proposed uncertainty measures improve the safety and reliability of AUVs operating in the coastal ocean.

We discuss the problem of inspecting an underwater structure, such as a submerged ship hull, with an autonomous underwater vehicle (AUV). In such scenarios, the goal is to construct an accurate 3D model of the structure and to detect any anomalies (e.g., foreign objects or deformations). We propose a method for constructing 3D meshes from sonar-derived point clouds that provides watertight surfaces, and we introduce uncertainty modeling through non-parametric Bayesian regression. Uncertainty modeling provides novel cost functions for planning the path of the AUV to minimize a metric of inspection performance. We draw connections between the resulting cost functions and submodular optimization, which provides insight into the formal properties of active perception problems. In addition, we present experimental trials that utilize profiling sonar data from ship hull inspection.

We examine the problem of utilizing an autonomous underwater vehicle (AUV) to collect data from an underwater sensor network. The sensors in the network are equipped with acoustic modems that provide noisy, range-limited communication. The AUV must plan a path that maximizes the information collected while minimizing travel time or fuel expenditure. We propose AUV path planning methods that extend algorithms for variants of the Traveling Salesperson Problem (TSP). While executing a path, the AUV can improve performance by communicating with multiple nodes in the network at once. Such multi-node communication requires a scheduling protocol that is robust to channel variations and interference. To this end, we examine two multiple access protocols for the underwater data collection scenario, one based on deterministic access and another based on random access. We compare the proposed algorithms to baseline strategies through simulated experiments that utilize models derived from experimental test data. Our results demonstrate that properly designed communication models and scheduling protocols are essential for choosing the appropriate path planning algorithms for data collection.

We examine the scenario in which a mobile network of robots must search, survey, or cover an environment and communication is restricted by relative location. While many algorithms choose to maintain a connected network at all times while performing such tasks, we relax this requirement and examine the use of periodic connectivity, where the network must regain connectivity at a fixed interval. We propose an online algorithm that scales linearly in the number of robots and allows for arbitrary periodic connectivity constraints. To complement the proposed algorithm, we provide theoretical inapproximability results for connectivity-constrained planning. Finally, we validate our approach in the coordinated search domain in simulation and in real-world experiments.

This paper presents coordination and data fusion methods for teams of vehicles performing target search tasks without guaranteed communication. A fully distributed team planning algorithm is proposed that utilizes limited shared information as it becomes available, and data fusion techniques are introduced for merging estimates of the target's position from vehicles that regain contact after long periods of time. The proposed data fusion techniques are shown to avoid overcounting information, which ensures that combining data from different vehicles will not decrease the performance of the search. Motivated by the underwater search domain, a realistic underwater acoustic communication channel is used to determine the probability of successful data transfer between two locations. The channel model is integrated into a simulation of multiple autonomous vehicles in both open ocean and harbor search scenarios. The simulated experiments demonstrate that distributed coordination with limited communication significantly improves team performance versus prior techniques that continually maintain connectivity.

We examine the problem of planning paths for an autonomous underwater vehicle (AUV) to collect data from an underwater sensor network. The sensors in the network are equipped with acoustic modems that provide noisy, range-limited communication. The AUV must plan a path that maximizes the information collected while minimizing travel time or fuel expenditure. This problem is closely related to the classical Traveling Salesperson Problem (TSP), but differs in that data from a particular sensor has a probability of being collected depending on the quality of communication. We propose methods for solving this problem by extending approximation algorithms for variants of TSP, and we compare our proposed algorithms to baseline strategies through simulated experiments with varying levels of communication quality. Our simulations utilize a realistic model of acoustic communication to determine the probability of acquiring data from each sensor. The results demonstrate that planning the tour for the entire network while exploiting the communication model during planning improves performance versus myopic methods.

We consider the problem of multi-robot coordination subject to constraints on the configuration. Specifically, we examine the case in which a mobile network of robots must search, survey, or cover an environment while remaining connected. While many algorithms utilize continual connectivity for such tasks, we relax this requirement and introduce the idea of periodic connectivity, where the network must regain connectivity at a fixed interval. We show that, in some cases, this problem reduces to the well-studied NP-hard multi-robot informative path planning (MIPP) problem, and we propose an online algorithm that scales linearly in the number of robots and allows for arbitrary periodic connectivity constraints. We prove theoretical performance guarantees and validate our approach in the coordinated search domain in simulation and in real-world experiments. Our proposed algorithm significantly outperforms a gradient method that requires continual connectivity and performs competitively with a market-based approach, but at a fraction of the computational cost.

We explore the interconnection between search and action in the context of mobile robotics. The task of searching for an object and then performing some action with that object is important in many applications. Of particular interest to us is the idea of a robot assistant capable of performing worthwhile tasks around the home and office (e.g., fetching coffee, washing dirty dishes, etc.). We prove that some tasks allow for search and action to be completely decoupled and solved separately, while other tasks require the problems to be analyzed together. We complement our theoretical results with the design of a combined search/action approximation algorithm that draws on prior work in search. We show the effectiveness of our algorithm by comparing it to state-of-the-art solvers, and we give empirical evidence showing that search and action can be decoupled for some useful tasks. Finally, we demonstrate our algorithm on an autonomous mobile robot performing object search and delivery in an office environment.

In this paper, we propose a framework for utilizing fixed, ultra-wideband ranging radio nodes to track a moving target node through walls in a cluttered environment. We examine both the case where the locations of the fixed nodes are known as well as the case where they are unknown. For the case when the fixed node locations are known, we derive a Bayesian room-level tracking method that takes advantage of the structural characteristics of the environment to ensure robustness to noise. We also develop a method using mixtures of Gaussians to model the noise characteristics of the radios. For the case of unknown fixed node locations, we present a two-step approach that first reconstructs the target node's path and then uses that path to determine the locations of the fixed nodes. We reconstruct the path by projecting down from a higher-dimensional measurement space to the 2D environment space using non-linear dimensionality reduction with Gaussian Process Latent Variable Models(GPLVMs). We then utilize the reconstructed path to map the locations of the fixed nodes using a Bayesian occupancy grid. We present experimental results verifying our methods in an office environment. Our methods are successful at tracking a moving target node and mapping the locations of fixed nodes using radio ranging data that are both noisy and intermittent.

In this paper, we describe a method for coordinating multiple robots in a pursuit-evasion domain. We examine the problem of multiple robotic pursuers attempting to locate a non-adversarial mobile evader in an indoor environment. Unlike many other approaches to this problem, our method seeks to minimize expected time of capture rather than guaranteeing capture. This allows us to examine the performance of our algorithm in complex and cluttered environments where guaranteed capture is difficult or impossible with limited pursuers. We present a probabilistic formulation of the problem, discretize the environment, and define cost heuristics for use in planning. We then propose a scalable algorithm using an entropy cost heuristic that searches possible movement paths to determine coordination strategies for the robotic pursuers. We present simulated results describing the performance of our algorithm against state of the art alternatives in a complex office environment. Our algorithm successfully reduces capture time with limited pursuers in an environment beyond the scope of many other approaches.

In this paper, a genetic algorithm (GA) is used to design fault-tolerant analog controllers for a piezoelectric micro-robot. First-order and second-order functions are developed to model the robot's piezoelectric actuators, and the GA is used to evolve closed-loop controllers for both models. The GA is first used to assist in traditional PID design and is later used to synthesize variable topology analog controllers. Through the use of a compact circuit representation, runtimes are minimized and controllers are synthesized with minimum population sizes and components. Fault-tolerance is built into the fitness function to facilitate the design of controllers robust to both actuator failure and component failure. The GA is successfully used to design synthetic controllers and to optimize a traditional PID design. This research shows the advantages of GA assisted design when applied to robot-control problems.

In this paper, we describe a robot that interacts with humans in a crowded conference environment. The robot detects faces, determines the shirt color of onlooking conference attendants, and reacts with a combination of speech, musical, and movement responses. It continuously updates an internal emotional state, modeled realistically after human psychology research. Using empirically-determined mapping functions, the robot's state in the emotion space is translated to a particular set of sound and movement responses. We successfully demonstrate this system at the AAAI '05 Open Interaction Event, showing the potential for emotional modeling to improve human-robot interaction.

Using the Darwin2k development software, a genetic algorithm (GA) was used to design and optimize a pipe-crawling robot for parameters such as mass, power consumption, and joint extension to further the research of the Miniature Inspection Systems Technology (MIST) team. In an attempt to improve on existing designs, a new robot was developed, the piezo robot. The final proposed design uses piezoelectric expansion actuators to move the robot with a ‘ chimneying’ method employed by mountain climbers and greatly improves on previous designs in load bearing ability, pipe traversing specifications, and field usability. This research shows the advantages of GA assisted design in the field of robotics.