This paper addresses the problem of image based localization. The goal is to find quickly and accurately the relative pose from a query taken from a stereo camera and a map obtained using visual SLAM which contains poses and 3D points associated to descriptors. In this paper we introduce a new method that leverages the stereo vision by adding geometric information to visual descriptors. This method can be used when the vertical direction of the camera is known (for example on a wheeled robot). This new geometric visual descriptor can be used with several image based localization algorithms based on visual words. We test the approach with different datasets (indoor, outdoor) and we show experimentally that the new geometric-visual descriptor improves standard image based localization approaches.

Summary This paper presents an open‐source, generic and efficient implementation of a very popular nonlinear optimization method: the Levenberg–Marquardt algorithm (LMA). This minimization algorithm is well known and hundreds of implementations have already been released. However, none of them offer at the same time a high level of genericity, a friendly syntax and a high computational performance. In this paper, we propose a solution to gather all those advantages in one library named LMA. The main challenge is to implement an efficient solver for every encounter problem. To overcome this difficulty, LMA uses compile time algorithms to design a code specific to the given optimization problem. The features of LMA are presented and the performances are compared with the state‐of‐the‐art best alternatives through extensive benchmarks on different kind of problems. Copyright © 2017 John Wiley & Sons, Ltd.

In this paper, we propose a new algorithm, named MCSLAM (Multiple Constrained SLAM ), designed to dynamically adapt each optimization to the variable number of parameters families (sensor pose, roughly known objects poses and dimensions, delay between sensors, ...) and heterogeneous constraints (reprojection error, distance from points or edges to object surface, acceleration...). The proposed algorithm is based on three contributions: 1) a new Levenberg-Marquardt C++ library named LMA and freely available 1 2) an architecture allowing a high level of flexibility and performances 3) a real-time usage of a temporal spline curve as the parametric trajectory model that provides an efficient way to add heterogeneous constraints within the optimization. The main idea of LMA is to provide a simple interface with a nonintrusive mechanism of adaptation to a problem while maintaining good performances. LMA works as a meta-program using C++ template to analyse at compile-time (CT) the problem to optimize from a list of C++ functors. The parameters are deduced from the functors arguments and the degree of freedom (dof ) of each parameter is defined by the user. Then LMA generates a data structure to store the functors and the parameters in tuple according to the number of parameters families and constraints. The resolution of the normal equations is written efficiently using a sparse representation constructed with a set of small matrices of static sizes. The LMA library solves the normal equations using dense Cholesky or a sparse PCG specially designed to manage little matrices of static size. It also implements the classical optimization tricks to be effective on little, medium, and big size problems. LMA also implements common features as automatic differentiation and robust cost functions. The continuous-time representation of the trajectory is used to deal with constraints on the motion. This allows to mix data from many unsynchronised sensors and evolution model. To apply constraints on the 3D structure of the environment, 3D models of coarsely knowns shapes are used. To represent the motion, we use the uniform cumulative b-spline described by Lovegrove et al. [2] but we separate position and orientation in two different splines and we use the Rodriguez formula to compute the exponential and the logarithm of SO3 group. Moreover, we adapt the keyframe based SLAM to deal with the spline: key-frames are constrained to be on the spline. We also use every inter-key-frame poses computed by the localization process to apply a weak constraint on the spline. A temporal sliding window of 3 seconds is used to select, from the SLAM and the IMU, the more recent data used to constrain the spline. The MCSLAM algorithm is based on a graph composed by constraints, parameters and dependencies. First, the problem configuration is analysed at compile-time to generate a specified LMA solver, whose cost function to minimize is the sum of the constraints C dynamically

We propose a new real-time CSLAM algorithm (Constrained Simultaneous Localization And Mapping), named DCSLAM (dynamically constrained SLAM), designed to dynamically adapt each optimization to the variable number of parameters families and heterogeneous constraints. An automatic method is used to generate, from an exhaustive list of constraints, a dedicated optimization algorithm. This is, to our knowledge, the only implementation that combines flexibility and performance. The proposed experiments show the effectiveness of our approach in terms of accuracy and execution time compared to the state of the art on several public benchmarks of varying complexity. An augmented reality application mixing heterogeneous objects and constraints is also available.

This paper presents a new approach for multi-objects tracking from a video camera moving in an unknown environment. The tracking involves static objects of different known shapes, whose poses and sizes are determined online. For augmented reality applications, objects must be precisely tracked even if they are far from the camera or if they are hidden. Camera poses are computed using simultaneous localization and mapping (SLAM) based on bundle adjustment process to optimize problem parameters. We propose to include in an incremental bundle adjustment the parameters of the observed objects as well as the camera poses and 3D points. We show, through the example of 3D models of basics shapes (planes, parallelepipeds, cylinders and spheres) coarsely initialized online using a manual selection, that the joint optimization of parameters constrains the 3D points to approach the objects, and also constrains the objects to fit the 3D points. Moreover, we developed a generic and optimized library to solve this modified bundle adjustment and demonstrate the high performance of our solution compared to the state of the art alternative. Augmented reality experiments in realtime demonstrate the accuracy and the robustness of our method.

Aircraft cockpits are advanced interfaces dedicated to the interaction and exchange of observations and commands between the pilot and the flying system. The design process of cockpits is benefiting from the use of Virtual Reality technologies: early ergonomics and layout analysis through the exploration of numerous alternatives, availability all along the cockpit life cycle of a virtual product ready for experimentation, reduced usage of costly physical mock-ups. Nevertheless, the construction of a virtual cockpit with the adequate performances is very complex. Due to the fact that the CAD based digital mock-up used for setting up the virtual cockpit is very large, one challenge is to achieve interactivity while maintaining the quality of rendering. The reduction of the information contained in the CAD database shall achieve a sufficient frame rate without degradation of the geometrical visual quality of the virtual cockpit which would alleviate the relevance of ergonomics and layout studies. This paper proposes to control the simplification process by using objective criteria based on considerations about the cockpit application and the visual performances of human beings. First, it presents the results of studies on the characteristics of the Human Visual System linked to virtual reality and visualization applications. Illustrated by first results, it establishes how to control simplifications in a rational and automatic way.