This paper presents a tele-operation system that enables a MAV to be controlled on virtual surfaces by an unskilled operator using high-level inputs. These virtual surfaces can be placed relatively to the infrastructure to be inspected, in order to ensure safety of the flight and repeatability of the acquisition conditions of the inspection data (e.g. at a constant distance from the infrastructure). The architecture, interface and embedded controller of the tele-operation system are described, and results from flight experiments in an industrial warehouse are provided for three typical inspection scenarios of infrastructures.

We explore a new sensor suite to provide a precise and robust navigation information, primarily intended for pedestrian localisation. We use an IMU sensor augmented with an array of magnetometers, called MIMU (for Magneto-Inertial measurement Unit) hereafter, and a single central camera as the vision sensor. The MIMU sensor has been shown in previous work to significantly improve the inertial dead-reckoning for pedestrian, provided we can assume a stationary and non-uniform magnetic environment. Such conditions are encountered in particular in indoor environment, where visual methods can be challenged by textureless or low-light areas. Alternatively, visual methods are useful in outdoor trajectories, where MIMU often fails because of a uniform magnetic field. We build a fusion estimator based on an inverse square-root filter fed with the raw measurement from magnetometers, accelerometers, gyrometers and visual features extracted and tracked in the video. Real-data experiments show the benefits of the fusion : on the one hand, the added information from the MIMU allows to complement the vision/inertial system in case where the vision does not provide useful information during an extended period of time; on the other hand, vision does extend the operational domain of the navigation system compared to a pure MIMU solution, in particular for outdoor sections of the trajectory.

This paper aims to leverage magnetic information from a Magneto-Inertial Measurement Unit - an IMU sensor augmented with an array of magnetometers, called MIMU hereafter - in a vision/inertial navigation system (VINS). This ego-motion estimation problem is formulated as an optimization over a sliding window fusing data from the MIMU with features tracked in a monocular camera image stream. The novelty of our approach lies in the formulation of preintegrated magnetic measurements that are computed from successive measurements of the local variations of the magnetic field, in the line of the preintegration of IMU data introduced in [1]. The resulting magnetic error terms participate to the minimized cost function along with the classical reprojection and IMU error terms. Our experiments show the benefits of this fusion. On the one hand, the added magnetic information from the MIMU allows to complement the VINS in cases where vision does not provide useful information during an extended period of time; on the other hand, vision does extend the operational domain of the navigation system compared to a pure MIMU solution, in particular for the outdoor portions of the trajectory.

This paper considers collaborative stereo-vision as a mean of localization for a fleet of micro-air vehicles (MAV) equipped with monocular cameras, inertial measurement units and sonar sensors. A sensor fusion scheme using an extended Kalman filter is designed to estimate the positions and orientations of all the vehicles from these distributed measurements. The estimation is completed by a formation control to maximize the overlapping fields of view of the vehicles. Experimental tests for the complete perception and control loop have been performed on multiple MAVs with centralized processing on a ROS ground station.

We present a novel visual-magneto-inertial system for pedestrian indoor navigation. It includes magnetic, inertial and depth sensors integrated into a device that can be hand held by a pedestrian. Our method builds upon a magneto-inertial tachymeter that is able to accurately reconstruct body speed in presence of magnetic gradient and a depth registration algorithm for computing inter-image movement. We propose a simple fusion strategy using the magneto-inertial tachymeter in the predition step and the depth image registration in the correction step. We demonstrate accurate reconstruction of indoor trajectories in infrastructureless environments, with a drift often below one percent of the length of the trajectory. We emphasize the benefit of the proposed hybrid system in situations where the 3D environment is too simple, making depth registration useless, or situations where the magnetic environment lead to failure of the magneto-inertial tachymetry.

This paper presents an exemplar based metric learning framework dedicated to robust visual localization in complex scenes, e.g. street images. The proposed framework learns off-line a specific (local) metric for each image of the database, so that the distance between a database image and a query image representing the same scene is smaller than the distance between the current image and other images of the database. To achieve this goal, we generate geometric and photometric transformations as proxies for query images. From the generated constraints, the learning problem is cast as a convex optimization problem over the cone of positive semi-definite matrices, which is efficiently solved using a projected gradient descent scheme. Successful experiments, conducted using a freely available geo-referenced image database, reveal that the proposed method significantly improves results over the metric in the input space, while being as efficient at test time. In addition, we show that the model learns discriminating features for the localization task, and is able to gain invariance to meaningful transformations.

This paper considers passive vision for robotics and focuses on devising a real-time process for moving object detection using a stereo rig. As several previous works, our method relies on the use of dense stereo and of optical flow. Observing that the main computational load of existing methods is related to the estimation of the optical flow, we propose to use a fast algorithm based on Lucas-Kanade's paradigm. We derive a new uncertainty model which explicitly takes into account all errors originating from each estimation step of the process. In contrast with most previous works, we describe a rigorous expansion of the error related to vision based ego-motion estimation. Finally, we present a comparative study of performance on the KITTI dataset, which demonstrates the effectiveness of the proposed approach.

The navigation of a miniature aerial vehicle (MAV) in GPS-denied environments requires a robust embedded visual localization method. In this paper, we describe a simple but efficient stereo visual odometry algorithm, called eVO, running onboard our quadricopter MAV at video-rate. The proposed eVO algorithm relies on a keyframe scheme which allows to decrease the estimation drift and to reduce the computational cost. We study quantitatively the influence of the main parameters of the algorithm and tune them for optimal performance on various datasets. The eVO algorithm has been submitted to the KITTI odometry benchmark [1] where it ranks first at the date of submission, with an average translational drift of 1.93% and an average angular drift of less than 0.076 degres/m. Besides, we have made several experiments with our MAV with egolocalization given by eVO, for instance for autonomous 3D environment modeling.

We propose solutions to provide unmanned aerial vehicles (UAV) with features to understand the scene below and help the operational planning. First, using a visual mapping of the environnement, interactive learning of specific targets of interest is performed on the ground control station to build semantic maps useful for planning. Then, the learned target detectors are transformed to be applied to new images captured by the UAV. On the technical side, we present: (i) an online gradient boost algorithm to interactively design context-dependent detectors; (ii) a video-domain adaptation method to use object detectors on on-board-camera images. We verify our approach on challenging data captured in real-world conditions.