In recent years, robots have been surfing on a trendy wave as standard devices for teaching programming. The tangibility of robotics platforms allows for collaborative and interactive learning. Moreover, with these robot platforms, we also observe the occurrence of a shift of visual attention from the screen (on which the programming is done) to the physical environments (i.e. the robot). In this paper, we describe an experiment aiming at studying the effect of using augmented reality (AR) representations of sensor data in a robotic learning activity. We designed an AR system able to display in real-time the data of the Infra-Red sensors of the Thymio robot. In order to evaluate the impact of AR on the learner's understanding on how these sensors worked, we designed a pedagogical lesson that can run with or without the AR rendering. Two different age groups of students participated in this between-subject experiment, counting a total of 74 children. The tests were the same for the experimental (AR) and control group (no AR). The exercises differed only through the use of AR. Our results show that AR was worth being used for younger groups dealing with difficult concepts. We discuss our findings and propose future works to establish guidelines for designing AR robotic learning sessions.

This paper presents a method for localizing an Unmanned Aerial Vehicle (UAV) using georeferenced aerial images. Easily maneuverable and more and more affordable, UAVs have become a real center of interest. In the last few years, their utilization has significantly increased. Today, they are used for multiple tasks such as navigation, transportation or vigilance. Nevertheless, the success of these tasks could not be possible without a highly accurate localization which can, unfortunately be often laborious. Here we provide a multiple usage localization algorithm based on vision only. However, a major drawback with vision-based algorithms is the lack of robustness. Most of the approaches are sensitive to scene variations (like season or environment changes) due to the fact that they use the Sum of Squared Differences (SSD). To prevent that, we choose to use the Mutual Information (MI) which is very robust toward local and global scene variations. However, dense approaches are often related to drift disadvantages. Here, we solve this problem by using georeferenced images. The localization algorithm has been implemented and experimental results are presented demonstrating the localization of a hexarotor UAV fitted with a downward looking camera during real flight tests.

We propose a formulation of monocular SLAM which combines live dense reconstruction with shape priors-based 3D tracking and reconstruction. Current live dense SLAM approaches are limited to the reconstruction of visible surfaces. Moreover, most of them are based on the minimisation of a photo-consistency error, which usually makes them sensitive to specularities. In the 3D pose recovery literature, problems caused by imperfect and ambiguous image information have been dealt with by using prior shape knowledge. At the same time, the success of depth sensors has shown that combining joint image and depth information drastically increases the robustness of the classical monocular 3D tracking and 3D reconstruction approaches. In this work we link dense SLAM to 3D object pose and shape recovery. More specifically, we automatically augment our SLAM system with object specific identity, together with 6D pose and additional shape degrees of freedom for the object(s) of known class in the scene, combining image data and depth information for the pose and shape recovery. This leads to a system that allows for full scaled 3D reconstruction with the known object(s) segmented from the scene. The segmentation enhances the clarity, accuracy and completeness of the maps built by the dense SLAM system, while the dense 3D data aids the segmentation process, yielding faster and more reliable convergence than when using 2D image data alone.

In this paper, we present a direct image registration approach that uses mutual information (MI) as a metric for alignment. The proposed approach is robust and gives an accurate estimation of a set of 2-D motion parameters in real time. MI is a measure of the quantity of information shared by signals. Although it has the ability to perform robust alignment with illumination changes, multimodality, and partial occlusions, few works have proposed MI-based applications related to spatiotemporal image registration or object tracking in image sequences because of some optimization problems, which we will explain. In this paper, we propose a new optimization method that is adapted to the MI cost function and gives a practical solution for real-time tracking. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the registration results are far more robust and accurate than the existing solutions, with the computation also being cheaper. A new approach is also proposed to speed up the computation of the derivatives and keep the same optimization efficiency. To validate the advantages of the proposed approach, several experiments are performed.

This paper proposes a method to achieve parametric motion estimation based on mutual information and its use in video mosaicing applications. Sum of Squared Differences (SSD) is widely considered in motion estimation. Here, we consider another metric, Mutual Information (MI), which is far less sensitive to changes in the lighting condition, to occlusions, and to a wide class of non-linear image transformation. Results under various complex conditions are presented.

In this paper we propose a new way to achieve a navigation task for a non-holonomic vehicle. We consider an image-based navigation process. We show that it is possible to navigate along a visual path without relying on the extraction, matching and tracking of geometric visual features such as keypoint. The new proposed approach relies directly on the information (entropy) contained in the image signal. We show that it is possible to build a control law directly from the maximisation of the shared information between the current image and the next key image in the visual path. The shared information between those two images are obtained using mutual information that is known to be robust to illumination variations and occlusions. Moreover the generally complex task of features extraction and matching is avoided. Both simulations and experiments on a real vehicle are presented and show the possibilities and advantages offered by the proposed method.

In this paper, we propose a new information theoretic approach to achieve visual servoing directly utilizing the information (as defined by Shannon) contained in the images. A metric derived from information theory, i.e., mutual information, is considered. Mutual information is widely used in multimodal image registration since it is insensitive to changes in the lighting condition and to a wide class of nonlinear image transformations. In this paper, mutual information is used as a new visual feature for visual servoing, which allows us to build a new control law that can control the six degrees of freedom (DOF) of a robot. Among various advantages, this approach requires no matching or tracking step, is robust to large illumination variations, and allows the consideration of different image modalities within the same task. Experiments on a real robot demonstrate the efficiency of the proposed visual-servoing approach.

In a previous paper, we proposed a new way to achieve visual servoing. Rather than minimizing the error between the position of two set of geometric features, we proposed to maximize the mutual information shared by the current and desired images. This leads to a new information theoretic approach to visual servoing. Mutual information is a well known alignment function. Thanks to its robustness toward illumination variations, occlusions and multi modality, it has been widely used in medical applications for alignment as well as in general tracking problems. Despite those previous works, no highlight has been given on the problem of Hessian computation that yields, in the case of common approximations, to divergence of the optimization process. In this paper we focus on the need of computing the second order derivative of the mutual information in visual servoing. Experiments on a 6 dof robot demonstrates the significance of this work on visual servoing tasks.

In this paper we present a direct tracking approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of the displacement that makes it adapted to augmented reality applications. MI is a measure of the quantity of information shared by signals that has been widely used in medical applications. Since then, and although MI has the ability to perform robust alignment with illumination changes, multi-modality and partial occlusions, few works propose MI-based applications related to object tracking in image sequences due to some optimization problems. In this work, we propose an optimization method that is adapted to the MI cost function and gives a practical solution for augmented reality application. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the tracking results are far more robust and accurate than the existing solutions. A new approach is also proposed to speed up the computation of the derivatives and keep the same optimization efficiency. To validate the advantages of the proposed approach, several experiments are performed. The ESM and the proposed MI tracking approaches are compared on a standard dataset. We also show the robustness of the proposed approach on registration applications with different sensor modalities: map versus satellite images and satellite images versus airborne infrared images within different AR applications.

This paper proposes a new way to achieve feature point tracking using the entropy of the image. Sum of Squared Differences (SSD) is widely considered in differential trackers such as the KLT. Here, we consider another metric called Mutual Information (MI), which is far less sensitive to changes in the lighting condition and to a wide class of non-linear image transformation. Since mutual-information is used as an energy function to be maximized to track each points, a new feature selection, which is optimal for this metric, is proposed. Results under various complex conditions are presented. Comparison with the classical KLT tracker are proposed.

In this work we propose a new way to achieve visual servoing using directly the information (as defined by Shannon) of the image. A metric derived from information theory, mutual information, is considered. Mutual information is widely used in multi-modal image registration (medical applications) since it is insensitive to changes in the lighting condition and to a wide class of non-linear image transformation. In this paper mutual-information is used as a new visual feature for visual servoing and allows us to build a new control law to control the 6 dof of the robot. Among various advantages, this approach does not require any matching nor tracking step, is robust to large illumination variation and allows to consider, within the same task, different image modalities. Experiments that demonstrate these advantages conclude the paper.