Utilizing heterogeneous mobile sensors to actively gather information improves adaptability and reliability in extended environments. This article presents a cooperative multirobot multitarget search and tracking framework aimed at enhancing the efficiency of the heterogeneous sensor network, and consequently, improving the overall target tracking accuracy. The concept ofnormalized unused sensing capacityis introduced to quantify the information a sensor is currently gathering relative to its theoretical maximum. This measurement can be computed using entirely local information and is applicable to various sensor models, distinguishing it from previous literature on the subject. It is then utilized to develop a heuristics distributed coverage control strategy for a heterogeneous sensor network, adaptively balancing the workload based on each sensor's current unused capacity. The algorithm is validated through a series of robot operating system (ROS) andMATLABsimulations, demonstrating superior results compared to standard approaches that do not account for heterogeneity or current usage rates.

We present Splat-Nav, a real-time robot navigation pipeline for Gaussian splatting (GSplat) scenes, a powerful new 3-D scene representation. Splat-Nav consists of two components: first, Splat-Plan, a safe planning module, and second, Splat-Loc, a robust vision-based pose estimation module. Splat-Plan builds a safe-by-construction polytope corridor through the map based on mathematically rigorous collision constraints and then constructs a Bézier curve trajectory through this corridor. Splat-Loc provides real-time recursive state estimates given only an RGB feed from an on-board camera, leveraging the point-cloud representation inherent in GSplat scenes. Working together, these modules give robots the ability to recursively replan smooth and safe trajectories to goal locations. Goals can be specified with position coordinates, or with language commands by using a semantic GSplat. We demonstrate improved safety compared to point cloud-based methods in extensive simulation experiments. In a total of 126 hardware flights, we demonstrate equivalent safety and speed compared to motion capture and visual odometry, but without a manual frame alignment required by those methods. We show online replanning at more than 2 Hz and pose estimation at about 25 Hz, an order of magnitude faster than neural radiance field-based navigation methods, thereby enabling real-time navigation.

This article presents a family of Stochastic Cartographic Occupancy Prediction Engines (SCOPEs) that enable mobile robots to predict the future states of complex dynamic environments. They do this by accounting for the motion of the robot itself, the motion of dynamic objects, and the geometry of static objects in the scene, and they generate a range of possible future states of the environment. These prediction engines are software-optimized for real-time performance for navigation in crowded dynamic scenes, achieving up to 89 times faster inference speed and 8 times less memory usage than other state-of-the-art engines. Three simulated and real-world datasets collected by different robot models are used to demonstrate that these proposed prediction algorithms are able to achieve more accurate and robust stochastic prediction performance than other algorithms. Furthermore, a series of simulation and hardware navigation experiments demonstrate that the proposed predictive uncertainty-aware navigation framework with these stochastic prediction engines is able to improve the safe navigation performance of current state-of-the-art model- and learning-based control policies.

The increased deployment of multi-robot systems (MRS) in various fields has led to the need to analyze systemlevel performance. However, creating consistent metrics for MRS is challenging due to the wide range of team and task parameters, such as the number of robots and the size of the environment. This paper presents a new analytical framework for MRS based on dimensionless variable analysis that effectively condenses the complex relationships between the team and task parameters that influence MRS performance into a manageable set of dimensionless variables. Then we use these dimensionless variables to fit a predictive parametric model of team performance. We apply our methodology to two MRS applications: Multi-Robot Multi-Target Tracking (MR-MTT) and Multi-Agent Path Finding (MAPF). The application of dimensionless variable analysis to MRS offers a promising method for MRS analysis that effectively reduces complexity, improves understanding of system behavior, and can inform the design and management of future MRS deployments.

This article proposes a novel learning-based control policy with strong generalizability to new environments that enables a mobile robot to navigate autonomously through spaces filled with both static obstacles and dense crowds of pedestrians. The policy uses a unique combination of input data to generate the desired steering angle and forward velocity: a short history of lidar data, kinematic data about nearby pedestrians, and a subgoal point. The policy is trained in a reinforcement learning setting using a reward function that contains a novel term based on velocity obstacles to guide the robot to actively avoid pedestrians and move toward the goal. Through a series of 3-D simulated experiments with up to 55 pedestrians, this control policy is able to achieve a better balance between collision avoidance and speed (i.e., higher success rate and faster average speed) than state-of-the-art model-based and learning-based policies, and it also generalizes better to different crowd sizes and unseen environments. An extensive series of hardware experiments demonstrate the ability of this policy to directly work in different real-world environments with different crowd sizes with zero retraining. Furthermore, a series of simulated and hardware experiments show that the control policy also works in highly constrained static environments on a different robot platform without any additional training. Lastly, several important lessons that can be applied to other robot learning systems are summarized.

This paper introduces the normalized unused sensing capacity to measure the amount of information that a sensor is currently gathering relative to its theoretical maximum. This quantity can be computed using entirely local information and works for arbitrary sensor models, unlike previous literature on the subject. This is then used to develop a distributed coverage control strategy for a team of heterogeneous sensors that automatically balances the load based on the current unused capacity of each team member. This algorithm is validated in a multi-target tracking scenario, yielding superior results to standard approaches that do not account for heterogeneity or current usage rates.

Ceramics are one of the major sources of information about the past for archaeologists, with a typical archaeological dig unearthing 1000’s of pottery fragments (sherds) each day. However, archaeologists often are not allowed to remove these sherds from their home countries. Therefore, logging data (e.g., mass, color, decoration) in the field is the only way to record valuable information about these sherds. Currently, this laborious process is done manually, using up much of the valuable time at a dig site. This project aims to automate the data collection process, freeing up archaeologists to spend their limited time in the field on other tasks, and to create a large-scale digital database of sherd information, allowing archaeologists to take advantage of new computational tools to make discoveries. The contribution of this paper is an automated system, consisting of several reconfigurable data collection stations and a robotic arm to transport objects between stations, that can rapidly generate a large database of archaeological artifacts. We validate our system in simulation, using high-resolution models of sherds. In other contexts, our system may help to expand the use of automation in materials handling, parts sorting, and more.

This paper proposes a novel neural network-based control policy to enable a mobile robot to navigate safety through environments filled with both static obstacles, such as tables and chairs, and dense crowds of pedestrians. The network architecture uses early fusion to combine a short history of lidar data with kinematic data about nearby pedestrians. This kinematic data is key to enable safe robot navigation in these uncontrolled, human-filled environments. The network is trained in a supervised setting, using expert demonstrations to learn safe navigation behaviors. A series of experiments in detailed simulated environments demonstrate the efficacy of this policy, which is able to achieve a higher success rate than either standard model-based planners or state-of-the-art neural network control policies that use only raw sensor data.

Accurately tracking dynamic targets relies on robots accounting for uncertainties in their own states to share information and maintain safety. The problem becomes even more challenging when there is an unknown and time-varying number of targets in the environment. In this paper we address this problem by introducing four new distributed algorithms that allow large teams of robots to: i) run the prediction and ii) update steps of a distributed recursive Bayesian multi- target tracker, iii) determine the set of local neighbors that must exchange data, and iv) exchange data in a consistent manner. All of these algorithms account for a bounded level of localization uncertainty in the robots by leveraging our recent introduction of the convex uncertainty Voronoi (CUV) diagram, which extends the traditional Voronoi diagram to account for localization uncertainty. The CUV diagram introduces a tessellation over the environment, which we use in this work both to distribute the multi-target tracker and to make control decisions about where to search next. We examine the efficacy of our method via a series of simulations and compare them to our previous work which assumed perfect localization.

The papers in this special section present recent advances in aerial swarm robotics, and aims to put together a cohesive set of research goals and visions toward realizing fully autonomous aerial swarm systems. One objective is to emphasize the three-way tradeoff among computational efficiency for large-scale swarms, stability, and robustness under uncertainty, and the optimal system performance. Aerial robotics has been one of the most active areas of research within the robotics community, and recently there have been many reports of promising results in aerial swarm systems. This is partly due to the commoditization of multicopter platforms, and communication, sensing, and processing hardware that has substantially lowered the barriers to entry to the field of aerial swarm robotics. Aerial swarms differ from swarms of ground-based vehicles in two major respects: Aerial robots or unmanned aerial vehicles (UAVs) operate in a three-dimensional space, and the dynamics of individual vehicles add an extra layer of complexity to the problems of path planning and trajectory design. Furthermore, the success of aerial swarms is predicated on the distributed and synergistic capabilities of individual and cooperative control, estimation, and decision making of aerial robots with limited resources, such as modest onboard computation and sensing capabilities and size, weight, and power constraints.

The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional space and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of this paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas.

This paper presents an automated robotic system for generating semantic maps of inventory in retail environments. In retail settings, semantic maps are labeled maps of stores where each discrete section of shelving is assigned a department label describing the types of products on that shelf. Starting from a metric map of the store, the robot autonomously extracts the shelf boundaries, generates a distance-optimal tour of the store to view every shelf, and follows the tour while avoiding unmapped clutter and moving people. The robot creates a point cloud of the store using the data collected from this tour. We introduce a novel soft-object assignment algorithm to create a virtual map and a dynamic programming algorithm to segment this map. These algorithms use a priori information about the products to boost data from laser and camera sensors in order to recognize and semantically label objects. The primary contribution of this paper is the integration of multiple systems for automated path planning, navigation, object recognition, and semantic mapping. This paper represents an important contribution toward deploying mobile robots in dynamic human environments.

In this paper we examine the problem of autonomously exploring and mapping an environment using a mobile robot. The robot uses a graph-based SLAM system to perform mapping and represents the map as an occupancy grid. In this setting, the robot must trade-off between exploring new area to complete the task and exploiting the existing information to maintain good localization. Selecting actions that decrease the map uncertainty while not significantly increasing the robot's localization uncertainty is challenging. We present a novel information-theoretic utility function that uses both Shannon's and Rényi's definitions of entropy to jointly consider the uncertainty of the robot and the map. This allows us to fuse both uncertainties without the use of manual tuning. We present simulations and experiments comparing the proposed utility function to state-of-the-art utility functions, which only use Shannon's entropy. We show that by using the proposed utility function, the robot and map uncertainties are smaller than using other existing methods.

This paper considers situations in which a team of mobile sensor platforms autonomously explores an environment to detect and localize an unknown number of targets. Individual sensors may be unreliable, failing to detect objects within the field-of-view, returning false positive measurements to clutter objects, and being unable to disambiguate true targets. In this setting, data association is difficult. We utilize the PHD filter for multitarget localization, simultaneously estimating the number of objects and their locations within the environment without the need to explicitly consider data association. Using sets of potential actions generated at multiple length scales for each robot, the team selects the joint action that maximizes the expected information gain over a finite time horizon. This is computed as the mutual information between the set of targets and the binary events of receiving no detections, effectively hedging against uninformative actions in a computationally tractable manner. We frame the controller as a receding-horizon problem. We demonstrate the real-world applicability of the proposed autonomous exploration strategy through hardware experiments, exploring an office environment with a team of ground robots. We also conduct a series of simulated experiments, varying the planning method, target cardinality, environment, and sensor modality. Note to Practitioners-Teams of small robots have the potential to automate many information gathering tasks, relaying data back to a base station or human operator from multiple vantage points within an environment. The information gathering tasks we consider in this work are those in which the number of objects being sought is not known at the onset of exploration. Such tasks are common in security and surveillance, where the number of such objects is often zero; search and rescue, where, for example, the number of people trapped due to a natural disaster can be large; or smart building/smart city applications, where the data collection needs may be on an even larger scale. This paper seeks to address the problem of automating this data collection process, so that a team of mobile sensor platforms are able to autonomously explore a given environment in order to determine the number of objects of interest and their locations, while avoiding any explicit data association, i.e., matching individual measurements to targets.

This paper addresses the task of searching for an unknown number of static targets within a known obstacle map using a team of mobile robots equipped with noisy, limited field-of-view sensors. Such sensors may fail to detect a subset of the visible targets or return false positive detections. These measurement sets are used to localize the targets using the Probability Hypothesis Density, or PHD, filter. Robots communicate with each other on a local peer-to-peer basis and with a server or the cloud via access points, exchanging measurements and poses to update their belief about the targets and plan future actions. The server provides a mechanism to collect and synthesize information from all robots and to share the global, albeit time-delayed, belief state to robots near access points. We design a decentralized control scheme that exploits this communication architecture and the PHD representation of the belief state. Specifically, robots move to maximize mutual information between the target set and measurements, both self-collected and those available by accessing the server, balancing local exploration with sharing knowledge across the team. Furthermore, robots coordinate their actions with other robots exploring the same local region of the environment.