Global localization is essential in enabling robot autonomy, and collaborative localization is key for multi-robot systems, allowing for more efficient planning and execution of tasks. In this paper, we address the task of collaborative global localization under computational and communication constraints. We propose a method which reduces the amount of information exchanged and the computational cost. We also analyze, implement and open-source seminal approaches, which we believe to be a valuable contribution to the community. We exploit techniques for distribution compression in near-linear time, with error guarantees. We evaluate our approach and the implemented baselines on multiple challenging scenarios, simulated and real-world. Our approach can run online on an onboard computer. We release an open-source C++/ROS2 implementation of our approach, as well as the baselines.1

We introduce PointIt, a toolkit for the Robot Operating System (ROS2) to build human-robot interfaces based on pointing gestures sensed by a wrist-worn Inertial Measurement Unit, such as a smartwatch. We release the software as open-source with MIT license; docker images and exhaustive instructions simplify its usage in simulated and real-world deployments.

We present an interactive demonstration where users are immersed in a virtual reality simulation of a logistic automation system. Using pointing gestures sensed by wrist-worn inertial measurement unit, users select defective packages transported on conveyor belts. The demonstration allows users to experience a novel way to interact with automation systems, and shows an effective application of virtual reality for human-robot interaction studies.

Many emerging applications of nano-sized unmanned aerial vehicles (UAVs), with a few cm2form-factor, revolve around safely interacting with humans in complex scenarios, for example, monitoring their activities or looking after people needing care. Such sophisticated autonomous functionality must be achieved while dealing with severe constraints in payload, battery, and power budget (~100mW). In this work, we attack a complex task going from perception to control: to estimate and maintain the nano-UAV’s relative 3-D pose with respect to a person while they freely move in the environment—a task that, to the best of our knowledge, has never previously been targeted with fully onboard computation on a nano-sized UAV. Our approach is centered around a novel vision-based deep neural network (DNN), called Frontnet, designed for deployment on top of a parallel ultra-low power (PULP) processor aboard a nano-UAV. We present a vertically integrated approach starting from the DNN model design, training, and dataset augmentation down to 8-bit quantization and deployment in-field. PULP-Frontnet can operate in real-time (up to135 frame/s), consuming less than87 mWfor processing at peak throughput and down to0.43 mJ/framein the most energy-efficient operating point. Field experiments demonstrate a closed-loop top-notch autonomous navigation capability, with a tiny 27-g Crazyflie 2.1 nano-UAV. Compared against an ideal sensing setup, onboard pose inference yields excellent drone behavior in terms of median absolute errors, such as positional (onboard:41cm, ideal:26 cm) and angular (onboard:3.7°, ideal:4.1°). We publicly release videos and the source code of our work.

Deep neural networks (DNNs) are becoming the first-class solution for autonomous unmanned aerial vehicles (UAVs) applications, especially for tiny, resource-constrained, nano-UAVs, with a few tens of grams in weight and subten centimeters in diameter. DNN visual pipelines have been proven capable of delivering high intelligence aboard nanoUAVs, efficiently exploiting novel multi-core microcontroller units. However, one severe limitation of this class of solutions is the generalization challenge, i.e., the visual cues learned on the specific training domain hardly predict with the same accuracy on different ones. Ultimately, it results in very limited applicability of State-of-the-Art (SoA) autonomous navigation DNNs outside controlled environments. In this work, we tackle this problem in the context of the human pose estimation task with a SoA vision-based DNN [1]. We propose a novel methodology that leverages synthetic domain randomization by applying a simple but effective image background replacement technique to augment our training dataset. Our results demonstrate how the augmentation forces the learning process to focus on what matters most: the pose of the human subject. Our approach reduces the DNN’s mean square error — vs. a non-augmented baseline — by up to 40%, on a never-seen-before testing environment. Since our methodology tackles the DNN’s training stage, the improved generalization capabilities come at zero-cost for the computational/memory burdens aboard the nano-UAV.

We present an approach for controlling the position of a quadrotor in 3D space using pointing gestures; the task is difficult because it is in general ambiguous to infer where, along the pointing ray, the robot should go. We propose and validate a pragmatic solution based on a push button acting as a simple additional input device which switches between different virtual workspace surfaces. Results of a study involving ten subjects show that the approach performs well on a challenging 3D piloting task, where it compares favorably with joystick control.

We demonstrate a self-supervised approach which learns to detect long-range obstacles from video: it automatically obtains training labels by associating the camera frames acquired at a given pose to short-range sensor readings acquired at a different pose.

We propose a system to control robots in the users proximity with pointing gestures-a natural device that people use all the time to communicate with each other. Our system has two requirements: first, the robot must be able to reconstruct its own motion, e.g. by means of visual odometry; second, the user must wear a wristband or smartwatch with an inertial measurement unit. Crucially, the robot does not need to perceive the user in any way. The resulting system is widely applicable, robust, and intuitive to use.

We demonstrate a system to control robots in the users proximity with pointing gestures-a natural device that people use all the time to communicate with each other. Our setup consists of a miniature quadrotor Crazyflie 2.0, a wearable inertial measurement unit MetaWearR+ mounted on the user's wrist, and a laptop as the ground control station. The video of this demo is available at https://youtu.be/yafy-HZMk_U [1].

We consider a quadrotor equipped with a forward-facing camera, and an user freely moving in its proximity; we control the quadrotor in order to stay in front of the user, using only camera frames. To do so, we train a deep neural network to predict the drone controls given the camera image. Training data is acquired by running a simple hand-designed controller which relies on optical motion tracking data.

We consider the problem of planning paths on graphs with some edges whose traversability is uncertain; for each uncertain edge, we are given a probability of being traversable (e.g., by a learned classifier). We categorize different interpretations of the problem that are meaningful for mobile robots navigating partially-known environments, each of which yields a different formalization; we then focus on the case in which the true traversability of an edge is revealed only when the agent visits one of its endpoints (Canadian Traveller Problem). In this context, we design a large simulation campaign on synthetic and real-world maps to study the impact of two different factors: the planning strategy, and the amount of uncertainty (which could depend on the quality of the classifier producing traversability estimates).

We consider the task of controlling a quadrotor to hover in front of a freely moving user, using input data from an onboard camera. On this specific task we compare two widespread learning paradigms: a mediated approach, which learns a high-level state from the input and then uses it for deriving control signals; and an end-to-end approach, which skips high-level state estimation altogether. We show that despite their fundamental difference, both approaches yield equivalent performance on this task. We finally qualitatively analyze the behavior of a quadrotor implementing such approaches.

Thymio is a small, inexpensive, mass-produced mobile robot with widespread use in primary and secondary education. In order to make it more versatile and effectively use it in later educational stages, including university levels, we have expanded Thymio's capabilities by adding off-the-shelf hardware and open software components. The resulting robot, that we call Mighty Thymio, provides additional sensing functionalities, increased computing power, networking, and full ROS integration. We present the architecture of Mighty Thymio and show its application in advanced educational activities.

We detect and localize obstacles in front of a mobile robot by means of a deep neural network that maps images acquired from a forward-looking camera to the outputs of five proximity sensors. The robot autonomously acquires training data in multiple environments; once trained, the network can detect obstacles and their position also in unseen scenarios, and can be used on different robots, not equipped with proximity sensors. We demonstrate both the training and deployment phases on a small modified Thymio robot.

We propose a model of artificial emotions for adaptation and implicit coordination in multi-robot systems. Artificial emotions play two roles, which resemble their function in animals and humans: modulators of individual behavior, and means of communication for social coordination. Emotions are modeled as compressed representations of the internal state, and are subject to a dynamics depending on internal and external conditions. Being a compressed representation, they can be efficiently exposed to nearby robots, allowing to achieve local group-level communication. The model is instantiated for a navigation task, with the aim of showing how coordination can effectively emerge by adding artificial emotions on top of an existing navigation framework. We show the positive effects of emotion-mediated group behaviors in a few challenging scenarios that would otherwise require ad hoc strategies: preventing deadlocks in crowded conditions; enabling efficient navigation of agents with time-critical tasks; assisting robots with faulty sensors. Two performance measures, throughput and number of collisions, are used to quantify the contribution of emotions for modulation and coordination.

This work focuses on how to compute trajectories for an autonomous wheelchair based on indoor GIS maps, in particular on IndoorGML maps, which set the standard in this context. Good wheelchair trajectories are safe and comfortable for the user and the people sharing the space with him, turn gently, are high legible, and smooth (at least G2continuos). We derive a navigation graph from a given IndoorGML map. We define and solve an optimization problem to find the desired path: given a succession of cells to traverse, the path corresponds to the best composite Bézier trajectory for the wheelchair. We discuss a related multi-objective path planning problem. Experimental results and an implementation on real robots show the planner performance.

Cooperative Multi-Robot Observation of Multiple Moving Targets (CMOMMT) denotes a class of problems in which a set of autonomous mobile robots equipped with limited-range sensors are used to keep under observation a (possibly larger) set of mobile targets. Robots cooperatively plan their motion in order to maximize the time during which each target lies within the sensing range of at least one robot.

No abstract available.

Once a multi-robot system is implemented on real hardware and tested in the real world, analyzing its evolution and debugging unexpected behaviors is often a very difficult task. We present a tool for aiding this activity, by visualizing an Augmented Reality overlay on a live video feed acquired by a fixed camera overlooking the robot environment. Such overlay displays live information exposed by each robot, which may be textual (state messages), symbolic (graphs, charts), or, most importantly, spatially-situated; spatially-situated information is related to the environment surrounding the robot itself, such as for example the perceived position of neighboring robots, the perceived extent of obstacles, the path the robot plans to follow. We show that, by directly representing such information on the environment it refers to, our proposal removes a layer of indirection and significantly eases the process of understanding complex multi-robot systems. We describe how the system is implemented, discuss application examples in different scenarios, and provide supplementary material including demonstration videos and a functional implementation.

Small-footprint mobile ground robots, such as the popular Turtlebot and Kobuki platforms, are by necessity equipped with sensors which lie close to the ground. Reliably detecting and tracking people from this viewpoint is a challenging problem, whose solution is a key requirement for many applications involving sharing of common spaces and close human-robot interaction. We present a robust solution for cluttered indoor environments, using an inexpensive RGB-D sensor such as the Microsoft Kinect or Asus Xtion. Even in challenging scenarios with multiple people in view at once and occluding each other, our system solves the person detection problem significantly better than alternative approaches, reaching a precision, recall and F1-score of 0.85, 0.81 and 0.83, respectively. Evaluation datasets, a real-time ROS-enabled implementation and demonstration videos are provided as supplementary material.

The vision-based mechanisms that pedestrians in social groups use to navigate in dynamic environments, avoiding obstacles and each others, have been subject to a large amount of research in social anthropology and biological sciences. We build on recent results in these fields to develop a novel fully-distributed algorithm for robot local navigation, which implements the same heuristics for mutual avoidance adopted by humans. The resulting trajectories are human-friendly, because they can intuitively be predicted and interpreted by humans, making the algorithm suitable for the use on robots sharing navigation spaces with humans. The algorithm is computationally light and simple to implement. We study its efficiency and safety in presence of sensing uncertainty, and demonstrate its implementation on real robots. Through extensive quantitative simulations we explore various parameters of the system and demonstrate its good properties in scenarios of different complexity. When the algorithm is implemented on robot swarms, we could observe emergent collective behaviors similar to those observed in human crowds.

Most local robot navigation algorithms are based on the concept of velocity obstacle, a mechanistic approach to the navigation problem in which a solution is engineered from scratch. Over the years, a number of different velocity obstacle variants have been developed to effectively handle multi-robot systems. In parallel, an alternative, human-inspired approach for robot navigation has been recently proposed, which derives from the observation and modeling of crowds of pedestrians. We discuss similarities and differences among two broadly used obstacle-velocity variants, namely Hybrid Reciprocal Velocity Obstacle and Optimal Reciprocal Collision Avoidance, and the human-inspired approach. How do these differences (which are often subtle) impact performance, and why? We answer these questions through extensive simulation experiments, wherein we evaluate the the algorithms for safety, trajectory efficiency, and emergence of collective behaviors, in different challenging multi-robot scenarios using both ideal and realistic models for robots and sensing.