This paper presents a 3D safe heuristic planner for aerial robot indoor navigation incorporating semantic information of the environment. Our interest lies in how the inclusion of semantic information in industrial environments or buildings can positively affect planning to ensure 3D safe navigation within the environment. The planner is based on an any-angle path planning algorithm, particularly Lazy Theta* algorithm. There are many works in the literature that include non-uniform costs related to distance to the closest obstacles in order to achieve a safe planner, but they generally do not consider the semantic information which is relevant in factories. Semantic-aware planning opens the door to safer plans, adapting the distance to obstacles depending on the category of the objects. Our Semantic Heuristic planner considers the semantic information of all the nodes along the line of sight between an initial and final node. Tests in a 3D environment with corridors, stairs, doors, columns and walls are performed to evaluate the proposed planner with respect to Lazy Theta* algorithms and a cost-aware Lazy-Theta* over Euclidean distance functions. The results show advantages of the Semantic Heuristic planner in terms of safety depending on the type of obstacle, the length of the path and efficiency with respect to the cost-aware Lazy-Theta* which only considers the distance to obstacles in its cost function.

Human-aware navigation is a complex task for mobile robots, requiring an autonomous navigation system capable of achieving efficient path planning together with socially compliant behaviors. Social planners usually add costs or constraints to the objective function, leading to intricate tuning processes or tailoring the solution to the specific social scenario. Machine Learning can enhance planners’ versatility and help them learn complex social behaviors from data. This work proposes an adaptive social planner, using a Deep Reinforcement Learning agent to dynamically adjust the weighting parameters of the cost function used to evaluate trajectories. The resulting planner combines the robustness of the classic Dynamic Window Approach, integrated with a social cost based on the Social Force Model, and the flexibility of learning methods to boost the overall performance on social navigation tasks. Our extensive experimentation on different environments demonstrates the general advantage of the proposed method over static cost planners.

This paper presents a fast cost-aware any-angle path planning algorithm for aerial robots in 3D building-like environments. The approach integrates Euclidean Distance Fields (EDF) and Lazy Theta* algorithm to compute safe and smooth paths. We show how to consider the analytical proper-ties of EDFs for polygonal obstacles to get an approximation of the cost along the line of sight segments of the planner, reducing the computational requirements. Numerous tests in a realistic building-like environment are performed to evaluate the proposed algorithm with respect to other heuristic search algorithms considering the distance cost by using an EDF. The results show that the proposed algorithm considerably reduces the computation time in indoor and outdoor environments enabling fast, safe and smooth paths.

Human collaboration is more likely to lead to cognitive growth when all group-members are actively involved in the collaborative process. However, there are cases that intragroup relationships need support. In this paper, we present an autonomous robotic system designed to interact with a pair of children in a problem-solving setting, aiming to understand how the robot behaviour impacts the group-members’ social dynamics. We developed an autonomous system with the Haru robot which we evaluated with an experimental study with 5-8yo children (N =84) to test the impact of the robot’s cognitive reliability and social positioning on human-to-human social dynamics, task performance and help-seeking behaviour. All participants took part in a baseline session (without the robot), an intervention (with the robot in a turn-taking setting) and an evaluation session (with a robot in a voluntary interaction setting). Results indicate that children who interacted with the reliable robot had a better task performance but children who interacted with the unreliable robot exhibited more task-related social interactions. Based on the results, we propose an interaction design concept which combines the set of the evaluated robot behaviours for an adaptive targeted support of child-robot teaming.

This paper presents a non-linear optimization method for trajectory planning of tethered aerial robots. Particularly, the paper addresses the planning problem of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) by means of a tether. The result is a collision-free trajectory for UAV and tether, assuming the UGV position is static. The optimizer takes into account constraints related to the UAV, UGV and tether positions, obstacles and temporal aspects of the motion such as limited robot velocities and accelerations, and finally the tether state, which is not required to be tense. The problem is formulated in a weighted multi-objective optimization framework. Results from simulated scenarios demonstrate that the approach is able to generate obstacle-free and smooth trajectories for the UAV and tether.

This paper presents DLL, a fast direct map-based localization technique using 3D LIDAR for its application to aerial robots. DLL implements a point cloud to map registration based on non-linear optimization of the distance of the points and the map, thus not requiring features, neither point correspondences. Given an initial pose, the method is able to track the pose of the robot by refining the predicted pose from odometry. Through benchmarks using real datasets and simulations, we show how the method performs much better than Monte-Carlo localization methods and achieves comparable precision to other optimization-based approaches but running one order of magnitude faster. The method is also robust under odometric errors. The approach has been implemented under the Robot Operating System (ROS), and it is publicly available.

This paper addresses the material handling problem (MHS) in warehouse automation by proposing a system that uses an automated guided vehicle (AGV) in industrial environments. The aim is to optimize the picking task with respect to manual operation in a paint factory. The work describes the whole system to perform all the automatic tasks. The process is controlled by the Manufacturing Process Management System (MPMS) and an autonomous co-worker robot execute the mission in partially known environments. The navigation system implemented is safe and robust. It considers the people detection and unknown static obstacles. Also, an ultra-wide-band localization system is implemented by offering new capabilities for situation awareness in factories. Experiments with a real holonomic platform called ARCO are performed to validate the approach.

In this work we present a bioinspired visual system sensor to estimate angular rates in unmanned aerial vehicles (UAV) using Neural Networks. We have conceived a hardware setup to emulate Drosophila's ocellar system, three simple eyes related to stabilization. This device is composed of three low resolution cameras with a similar spatial configuration as the ocelli. There have been previous approaches based on this ocellar system, most of them considering assumptions such as known light source direction or a punctual light source. In contrast, here we present a learning approach using Artificial Neural Networks in order to recover the system's angular rates indoors and outdoors without previous knowledge. A classical computer vision based method is also derived to be used as a benchmark for the learning approach. The method is validated with a large dataset of images (more than half a million samples) including synthetic and real data. The source code of the algorithms and the datasets used in this paper have been released in an open repository.

Motion is an important modality for human-robot interaction. Besides a fundamental component to carry out tasks, through motion a robot can express intentions and expressions as well. In this paper, we focus on a tabletop robot in which motion, among other modalities, is used to convey expressions. The robot incorporates a set of pre-programmed motion animations that show different expressions with various intensities. These have been created by designers with expertise in animation. The objective in the paper is to analyze if these examples can be used as demonstrations, and combined by the robot to generate additional richer expressions. Challenges are the representation space used, and the scarce number of examples. The paper compares three different learning from demonstration approaches for the task at hand. A user study is presented to evaluate the resultant new expressive motions automatically generated by combining previous demonstrations.

This work presents an approach to learn path planning for robot social navigation by demonstration. We make use of Fully Convolutional Neural Networks (FCNs) to learn from expert's path demonstrations a map that marks a feasible path to the goal as a classification problem. The use of FCNs allows us to overcome the problem of manually designing/identifying the cost-map and relevant features for the task of robot navigation. The method makes use of optimal Rapidly-exploring Random Tree planner (RRT*) to overcome eventual errors in the path prediction; the FCNs prediction is used as cost-map and also to partially bias the sampling of the configuration space, leading the planner to behave similarly to the learned expert behavior. The approach is evaluated in experiments with real trajectories and compared with Inverse Reinforcement Learning algorithms that use RRT* as underlying planner.

This paper presents a vision-based localization system for global pose estimation of a sewer inspection robot given prior information of the sewer network from local institutions. The system is based on a Monte-Carlo Localization system that uses RGBD odometry for the prediction stage. The update step takes into account the sewer network topology for discarding wrong hypotheses. Moreover, this step is further refined whenever a discrete element of the network (i.e. manhole) is detected. To this end, another RGBD camera pointing upwards is used for precise manhole detection. A Convolutional Neural Network has been successfully trained for classifying images with and without manholes with 96% accuracy over the tested dataset. The complete system has been validated with real data obtained from the sewers of Barcelona yielding accurate localization results. All the logs and code used in the context of this paper are publicly available.

This paper presents a methodology for mapping and localization of Unmanned Aerial Vehicles (UAVs) based on the integration of sensors from different modalities. Particularly, we integrate distance estimations to Ultra-Wideband (UWB) sensors and 3D point-clouds from RGB-D sensors. First, a novel approach for environment mapping is introduced, exploiting the synergies between UWB sensors and point-clouds to produce a multi-modal 3D map that integrates the estimated UWB sensors position. This map is further integrated into a Monte Carlo Localization method to robustly estimate the UAV pose. Finally, the full approach is tested with real indoor flights and validated with a motion tracking system.

This paper presents an approach for decentralized range-only simultaneous localization and mapping (RO-SLAM) of a network of aerial vehicles and a set of static range-only sensors deployed in the environment. The paper makes use of a multi-hypothesis framework developed by the authors [1] in order to deal with the multiple hypotheses that are present in the early stages of the undelayed RO-SLAM, this paper extends the approach to consider the integration of landmark estimations provided by other aerial vehicles nearby the robot. The method will enable a significant reduction in the convergence time needed to remove wrong localization hypotheses for every range-only landmark and, as a results, a map with improved accuracy. The proposed approach is validated first in simulations and later on with real experiments involving real range-only sensors and two unmanned aerial vehicles.

Robot navigation in human environments is an active research area that poses serious challenges. Among them, human-awareness has gain lot of attention in the last years due to its important role in human safety and robot acceptance. The proposed robot navigation system extends state of the navigation schemes with some social skills in order to naturally integrate the robot motion in crowded areas. Learning has been proposed as a more principled way of estimating the insights of human social interactions. To do this, inverse reinforcement learning is used to derive social cost functions by observing persons walking through the streets. Our objective is to incorporate such costs into the robot navigation stack in order to “emulate” these human interactions. In order to alleviate the complexity, the system is focused on learning an adequate cost function to be applied at the local navigation level, thus providing direct low-level controls to the robot. The paper presents an analysis of the results in a robot navigating in challenging real scenarios, analyzing and comparing this approach with other algorithms.

This paper presents a new approach for aerial robots simultaneous localization and mapping (SLAM) oriented to aerial manipulation applications. The approach is based on the integration of range-only measurements and visual markers detected with the on-board camera. A multiple hypotheses framework is applied for range-only together with visual markers SLAM. This approach allows integrating two different types of sensors that are complementary for localization, mixing the stable and continuous estimation provided by range sensors with the precise but infrequent measurements from the visual markers. Real experiments involving an aerial robot and radio/visual markers have been used to validate the approach.

Robot navigation in human environments is an active research area that poses serious challenges. Among them, social navigation and human-awareness has gain lot of attention in the last years due to its important role in human safety and robot acceptance. Learning has been proposed as a more principled way of estimating the insights of human social interactions. In this paper, inverse reinforcement learning is analyzed as a tool to transfer the typical human navigation behavior to the robot local navigation planner. Observations of real human motion interactions found in one publicly available datasets are employed to learn a cost function, which is then used to determine a navigation controller. The paper presents an analysis of the performance of the controller behavior in two different scenarios interacting with persons, and a comparison of this approach with a Proxemics-based method.

This paper presents an undelayed range-only simultaneous localization and mapping (RO-SLAM) based on the Extended Kalman filter. The approach is optimized for working in 3D scenarios, reducing the required computational payload at two levels: first, using a reduced spherical state vector parametrization and, second, proposing a new EKF update scheme. The paper proposes a state vector parametrization based on Gaussian-Mixture to cope with the multi-modal nature of range-only measurements and a reduced spherical parametrization of the range sensor positions that allows to shorten the length of the state vector for a given number of hypotheses. The approach is firstly tested and discussed in simulation, followed by experimental results involving a real robot and radio-based range sensors.

Radio signal-based localization and mapping is becoming more interesting as applications involving the collaboration between robots and static wireless devices are more common. Under certain assumptions, the problem is basically equivalent to the range-only localization and mapping problem. The paper presents a method for mapping with a mobile robot the position of a set of nodes using radio signal measurements. It uses Gaussian Mixtures for undelayed initialization of the position of the wireless nodes. The paper shows how the approach can be integrated within a Kalman Filter. This way, information can be used in the filter since the first measurement. The paper describes simulations to verify the feasibility of the approach, and presents results obtained with experimental data involving one mobile robot and a wireless sensor network.

This paper and video are devoted to the last experiments and demonstration of the AWARE project (European Commission, FP6) carried out in Utrera, near Seville (Spain) May, 2009. The project has developed and validated in field experiments a platform providing the functionalities required for the cooperation of aerial robots with ground sensor-actuator wireless networks, including static and mobile nodes carried by people and vehicles. The project demonstrated the self-deployment, self-configuration and self-repairing of the network by using autonomous helicopters that transported and deployed sensor nodes and loads. These features are highly relevant in natural and urban environments without pre-existing infrastructure or where the infrastructure has been damaged or destroyed. Two validation scenarios have been considered: Disaster Management/Civil Security and Filming.

Radio signal-based localization and mapping is becoming more interesting in robotics as applications involving the collaboration between robots and static wireless devices are more common. This paper describes a method for mapping with a mobile robot the position of a set of nodes using radio signal measurements. The method employs Gaussian Mixtures Models (GMM) for undelayed initialization of the position of the wireless nodes within a Kalman filter. Moreover, the paper extends the method to consider active sensing strategies in order to map the nodes. Entropy variation is used as a measurement of information gain, and allows to prioritize control actions of the robot. However, as there is no analytical expression for the entropy of a GMM, upper bounds of the entropy, for which close form computation is possible, are used instead. The paper describes simulations that show the feasibility of the approach.

This paper presents a decentralized data fusion approach to perform cooperative perception with data gathered from heterogeneous sensors, which can be static or carried by robots. Particularly, a Decentralized Delayed-State Extended Information Filter (DDSEIF) is described, where full state trajectories are considered to fuse the information. This permits to obtain an estimation equal to that obtained by a centralized system, and allows delays and latency in the communications. The sparseness of the information matrix maintains the communications overhead at a reasonable level. The method is applied to cooperative tracking and some results in disaster management scenarios are shown. In this kind of scenarios the target might move in both open field and indoor areas, so fusion of data provided by heterogeneous sensors is beneficial.

This paper presents a new method for the 3D localization of an outdoor wireless sensor network (WSN) by using a single flying beacon-node on-board an autonomous helicopter, which is aware of its position thanks to a GPS device. The technique is based on particle filtering and does not require any prior information about the position of the nodes to be estimated. Its structure and stochastic nature allows a distributed computation of the position of the nodes. The paper shows how the method is very suitable for outdoor applications with robotic data-mule systems. The paper includes a section with experiments.

This paper presents a probabilistic framework where uncertainties can be considered in the mosaic building process. It is shown how can be used as an environment representation for an aerial robot. The inter-image relations are modeled by homographies. The paper shows a robust method to compute them in the case of quasi-planar scenes, and also how to estimate the uncertainties in these local image relations. Moreover, the paper describes how, when a loop is present in the sequence of images, the accumulated drift can be compensated and propagated to the rest of the mosaic. In addition, the relations among images in the mosaic can be used, under certain assumptions, to localize the robot.

The paper presents a vision-based position estimation method for UAVs. It assumes a planar scene, approximation that usually holds when a vehicle is flying at a relatively high altitude. Monocular image sequences gathered by the UAV are used to estimate the vehicle motion, but accumulative errors can make diverge the estimated position. The proposed method uses an online-built mosaic to correct the drift associated to the planar motion estimation algorithm. The mosaic allows to use not only the current image but also previously recorded information for localization. Results from actual field experiments are presented

This paper presents an approach to detect safe landing areas for a flying robot, on the basis of a sequence of monocular images. The approach does not require precise position and attitude sensors: it exploits the relations between 2D image homographies and 3D planes. The combination of a robust homography estimation and of an adaptive thresholding of correlation scores between registered images yields the update of a stochastic grid, that exhibits the horizontal planar areas perceived. This grid allows the integration of data gathered at various altitudes. Results are presented

This paper presents a vision-based method to estimate the * real motion of a single camera from views of a planar patch. Projective techniques allow to estimate camera motion from pixel space apparent motion without explicit 3-D reconstruction. In addition, the paper will present the HELINSPEC project, the framework where the proposed method has been tested, and will detail some applications in external building inspection that make use of the proposed techniques.

The paper presents a framework for cooperative fire detection by means of a fleet of heterogeneous UAVs. Computer vision techniques are used to detect and localize fires from infrared and visual images and other data provided by the cameras and other sensors on-board the UAVs. The paper deals with the techniques used to decrease the uncertainty in fire detection and increase the accuracy in fire localisation by means of the cooperation of the information provided by several UAVs. The presented methods have been developed in the COMETS multi-UAV project.

This work presents real time computer vision techniques for autonomous navigation and operation of unmanned aerial vehicles. The proposed techniques are based on image feature matching and projective methods. Particularly, the paper presents the application to helicopter motion compensation and object detection. These techniques have been implemented in the framework of the COMETS multi-UAV systems. Furthermore, The work presents the application of the proposed techniques in a forest fire scenario in which the COMETS system can be demonstrated.