Semantic segmentation of 3D LiDAR point clouds, essential for autonomous driving and infrastructure management, is best achieved by supervised learning, which demands extensive annotated datasets and faces the problem of domain shifts. We introduce a new 3D semantic segmentation pipeline that leverages aligned scenes and state-of-the-art 2D segmentation methods, avoiding the need for direct 3D annotation or reliance on additional modalities such as camera images at inference time. Our approach generates 2D views from LiDAR scans colored by sensor intensity and applies 2D semantic segmentation to these views using a camera-domain pretrained model. The segmented 2D outputs are then back-projected onto the 3D points, with a simple voting-based estimator that merges the labels associated to each 3D point. Our main contribution is a global pipeline for 3D semantic segmentation requiring no prior 3D annotation and not other modality for inference, which can be used for pseudo-label generation. We conduct a thorough ablation study and demonstrate the potential of the generated pseudo-labels for the Unsupervised Domain Adaptation task.

In this paper, we investigate continual learning performance metrics used in class incremental learning strategies for continual learning (CL) using some high performing methods. We investigate especially mean task accuracy. First, we show that it lacks of expressiveness through some simple experiments to capture performance. We show that monitoring average tasks performance is over optimistic and can lead to misleading conclusions for future real life industrial uses. Then, we propose first a simple metric, Minimal Incremental Class Accuracy (MICA) which gives a fair and more useful evaluation of different continual learning methods. Moreover, in order to provide a simple way to easily compare different methods performance in continual learning, we derive another single scalar metric that take into account the learning performance variation as well as our newly introduced metric.

Gestures classification is a widespread solution for prosthetic hand control using electromyographic signals (EMG). It can be defined as the task of estimating the class associated to a time-series data provided by the sensor. Nevertheless, it remains unsatisfactory as the number of motions (classes) is limited. In this article, we investigate the problem of estimating the joint coordinates of the hand from the same input data. The main contributions of the article are: (1) a new dataset for joint coordinates estimation from EMG data, (2) a set of neural networks models for this regression task, and (3) an experimental study to compare the performances of the proposed methods. The database is built with a six channels EMG armband and hand motions measured by computer vision. First results are hopeful and provide both a baseline for new methods and a proof of concept for future prosthetic control solutions.

Existing neural network-based object detection approaches process LiDAR point clouds trained from one kind of LiDAR sensor. In the case of a different point cloud input, the trained network performs with less efficiency, especially when the given point cloud has low resolution. In this paper, we propose a new object detection approach, which is more resilient to variations in point cloud resolution. Firstly, layers from the point cloud are randomly discarded during the training phase in order to increase the variability of the data processed by the network. Secondly, the obstacles are described as Gaussian functions, grouping multiple parameters into a single representation. A Bhattacharyya distance is used as a loss function. This approach is tested on a LiDAR-based network and on an architecture using camera and LiDAR sensors. The networks are trained exclusively on the KITTI dataset and tested on Pandaset and the nuScenes Mini dataset. Experiments show that our method improves the performance of the tested networks on low-resolution point clouds without decreasing the ability to process high-resolution data.

Deep neural networks are important for a wide range of scientific and industrial processes. However, a classical discriminative model always makes a classification with respect to the probabilities allocated to the training labels, even when the sample is out of the domain. Thus, it is of interest to assign uncertainty to a model prediction to avoid such a situation. Fortunately, there are many existing methods for dealing with this kind of problem, one branch of which involves combining neural networks with subjective logic (SL). Based on previous works, we propose a new method called subjective-logic-based uncertainty estimation (SLUE) that can take the base rate distribution explicitly into account to refine the Dirichlet distribution parameters and guide the model training. Experiments were performed on several public datasets and additional adversarial datasets. Compared with existed methods, SLUE reached better uncertainty assessment performance (15% improvement in terms of % max entropy) as well as comparable prediction accuracy performance.

We present a temporal Bayesian filter for semantic segmentation of a video sequence. Each pixel is a random variable following a discrete probabilistic distribution function representing possible semantic classes. Bayesian filtering consists in two main steps: 1) a prediction model and 2) an observation model (likelihood). We propose to use a datadriven prediction function derived from a dense optical flow between images t and t + 1 achieved by a deep neural network [1]. Moreover, the observation function uses a semantic segmentation network. The resulting approach is evaluated on the public dataset Cityscapes. We show that using the temporal filtering increases the accuracy of the semantic segmentation.

Tracking-by-detection have become a hot topic for intelligent vehicle applications in recent year. Generally, the existing tracking-by-detection frameworks have difficulties with congestion, occlusion, and inaccurate detection in crowded scenes. In this paper, we propose a new framework for Multi-Object Tracking-by-detection (MOT-TbD) based on an temporal interlaced encoding video model and a specialized Deep Convolutional Neural Network (DCNN) detector. Spatio-temporal variation of objects between images are encoded into “interlaced images”. A specialized “interlaced object” deep detector is trained on a interlaced dataset. The detected objects are associated with a classical data association algorithm. Since interlaced objects are built to increase overlap during the association step, the performance of the MOT-TbD increases related to the same detector/association algorithm applied on non-interlaced images. The effectiveness of the method is demonstrated by experiments on popular tracking-by-detection datasets such as the PETS 2009 and TUD. Experimental results show that the proposed framework outperforms several state-of-the-art MOT-TbD methods.

The detection and localization of objects on point cloud provided by LiDAR sensors have various applications, especially for autonomous driving. As methods are being more efficient in terms of performance and execution speed, they operate in an end-to-end fashion, not allowing in case of errors the evaluation of which areas could cause failures. We propose in this paper a simple method that delivers from a simple statistical voxel encoding an intermediate top view heatmap that illustrates the global state of the scene seen by the network. Completely detected vehicles are represented on this map as elliptic blobs. Difficult cases such as occluded cars may not illustrate a complete spot, however their mark may still indicate that a possible hazard, allowing planification algorithms to decide a reduction of the velocity as something may suddenly appear. Furthermore, thanks to the alternative representation of objects, detections in terms of bounding boxes can be extracted even from the intermediate map. In addition to its implementation simplicity, the proposed architecture reaches high level of performance on the KITTI detection benchmark.

In today's world, automatic navigation for a robotic device (autonomous vehicle and robot) is a pre-requisite for many complex tasks, which requires a robust localization method. We focus in this paper on the topic of localizing such a robot into an absolute and imprecise map. We propose a multi-sensor self-localization method, which is simultaneously able to operate with an imprecise map, as well as to improve the precision of an already existing one. The method uses split covariance intersection filter as well as an a priori selection of the best informative measurements out of all possible measurement sources at each time step. This selection scheme is based on an “added Shannon information”-based criterion. We demonstrate in operation via statistical analysis the consistency of a refined map obtained from a biased map while keeping vehicle localization integrity. On top of this, we demonstrate solving of the so-called kidnapped-robot problem using the same framework.

Intelligent traffic systems for traffic surveillance and monitoring have become a topic of great interest to some cities in the world. Generally, the existing traffic surveillance systems are made up of costly equipment with complicated operational procedures and have difficulties with congestion, occlusion, and lighting night/day and day/night transitions. In this paper, we propose an embedded system for traffic surveillance that can be utilized under these challenging conditions. This system analyses traffic and particularly focuses on the problem of detecting and categorizing traffic objects in several traffic scenarios. Moreover, it contains a robust detector produced by an original specialization framework. The proposed specialization framework utilizes a generic deep detector so as to improve the detection accuracy in a specific traffic scenario. The experiments demonstrate that the proposed specialization framework presents encouraging results for multi-traffic object detection and outperforms the state-of-the-art specialization frameworks on several public traffic datasets.

Learning complex behaviors through reinforcement learning is particularly challenging when reward is only available upon successful completion of the full behavior. In manipulation robotics, so-called shaping rewards are often used to overcome this problem. However, these usually require human engineering or (partial)world models describing, e.g., the kinematics of the robot or high-level modules for perception. Here we propose an alternative method to learn an object palm-touching task through a weakly-supervised and stagewise learning of simpler tasks. First, the robot learns to fixate the object with its cameras. Second, the robot learns eye-hand coordination by learning to fixate its end effector. Third, using the previously acquired skills an informative shaping reward can be computed which facilitates efficient learning of the object palm-touching task. We demonstrate in simulation that learning the full task with this shaping reward is comparable to learning with an informative supervised reward.

In this paper, we present a novel approach, called Deep MANTA (Deep Many-Tasks), for many-task vehicle analysis from a given image. A robust convolutional network is introduced for simultaneous vehicle detection, part localization, visibility characterization and 3D dimension estimation. Its architecture is based on a new coarse-to-fine object proposal that boosts the vehicle detection. Moreover, the Deep MANTA network is able to localize vehicle parts even if these parts are not visible. In the inference, the networks outputs are used by a real time robust pose estimation algorithm for fine orientation estimation and 3D vehicle localization. We show in experiments that our method outperforms monocular state-of-the-art approaches on vehicle detection, orientation and 3D location tasks on the very challenging KITTI benchmark.

Due to its ability to learn complex behaviors in high-dimensional state-action spaces, deep reinforcement learning algorithms have attracted much interest in the robotics community. For a practical reinforcement learning implementation on a robot, it has to be provided with an informative reward signal that makes it easy to discriminate the values of nearby states. To address this issue, prior information, e.g. in the form of a geometric model, or human supervision are often assumed. This paper proposes a method to learn binocular fixations without such prior information. Instead, it uses an informative reward requiring little supervised information. The reward computation is based on an anomaly detection mechanism which uses convolutional autoencoders. These detectors estimate in a weakly supervised way an object's pixellic position. This position estimate is affected by noise, which makes the reward signal noisy. We first show that this affects both the learning speed and the resulting policy. Then, we propose a method to partially remove the noise using regression on the detection change given sensor data. The binocular fixation task is learned in a simulated environment on an object training set with various shapes and colors. The learned policy is compared with another one learned with a highly informative and noiseless reward signal. The tests are carried out on the training set and on a test set of new objects. We observe similar performances, showing that the environment-encoding step can replace the prior information.

Given an untextured 3D car models dataset, are we able to learn a robust make and model classifier which will be applied on real color images? One solution consists in finding a common representation between synthetic edge images and real color images. To address this issue, we introduce novel edge-color invariant features for 2D/3D car fine-grained classification. These features are learned simultaneously on real color and edge images of cars using a deep architecture. Using these accurate features, we propose to learn on edges synthetic images a fine-grained classifier which will be afterwards applied to real color images. The 3D car models dataset allows to automatically generate multi-views synthetic images using non-photorealistic edge rendering. Experimentally, we show efficiency of our learned edge-color invariant features for make and model recognition.

In mobile robotics, in order to accomplish a task, self-localization is of paramount importance. It is generally granted that robot localization evolving in an outdoor environment needs an absolute localization technique, which can be map-based. While most of map-based techniques use a static map of the environment, we propose here to handle a dynamic map, which could originate from publicly available Geographical Information Systems (GIS) in which imprecise mapped landmarks would have their own localization updated, unobservable mapped landmarks would be deleted and newly observable landmarks would be added. In this article, our contribution is to demonstrate the possibility to delete dynamically repositionable landmarks in a dynamic environment map on a real vehicle online in real-time, using a partially separated vehicle localization integrity and landmark existence probabilities, based on a Bayesian graphical network formalism.

Summary This paper presents an open‐source, generic and efficient implementation of a very popular nonlinear optimization method: the Levenberg–Marquardt algorithm (LMA). This minimization algorithm is well known and hundreds of implementations have already been released. However, none of them offer at the same time a high level of genericity, a friendly syntax and a high computational performance. In this paper, we propose a solution to gather all those advantages in one library named LMA. The main challenge is to implement an efficient solver for every encounter problem. To overcome this difficulty, LMA uses compile time algorithms to design a code specific to the given optimization problem. The features of LMA are presented and the performances are compared with the state‐of‐the‐art best alternatives through extensive benchmarks on different kind of problems. Copyright © 2017 John Wiley & Sons, Ltd.

We propose a new approach for accurate car pose estimation in images using only a dataset of 3D untextured models. Our algorithm detects both a car and its 3D pose. It is based on the matching of 3D models with the car in the image. With a part detector based on Convolutional Neural Networks, interest points corresponding to predefined 3D parts are extracted from the image. Then, we use the car geometry to find which parts are relevant across viewpoints. Finally, a 2D/3D pose estimator is used to recover the 3D pose of the car. The main contribution is to learn appearance and geometry models from 3D models dataset only. Experiments show that the method is very competitive for car detection and coarse viewpoint classification and improves the 3D pose estimation over the state-of-the-art methods.

In this paper, we propose a new algorithm, named MCSLAM (Multiple Constrained SLAM ), designed to dynamically adapt each optimization to the variable number of parameters families (sensor pose, roughly known objects poses and dimensions, delay between sensors, ...) and heterogeneous constraints (reprojection error, distance from points or edges to object surface, acceleration...). The proposed algorithm is based on three contributions: 1) a new Levenberg-Marquardt C++ library named LMA and freely available 1 2) an architecture allowing a high level of flexibility and performances 3) a real-time usage of a temporal spline curve as the parametric trajectory model that provides an efficient way to add heterogeneous constraints within the optimization. The main idea of LMA is to provide a simple interface with a nonintrusive mechanism of adaptation to a problem while maintaining good performances. LMA works as a meta-program using C++ template to analyse at compile-time (CT) the problem to optimize from a list of C++ functors. The parameters are deduced from the functors arguments and the degree of freedom (dof ) of each parameter is defined by the user. Then LMA generates a data structure to store the functors and the parameters in tuple according to the number of parameters families and constraints. The resolution of the normal equations is written efficiently using a sparse representation constructed with a set of small matrices of static sizes. The LMA library solves the normal equations using dense Cholesky or a sparse PCG specially designed to manage little matrices of static size. It also implements the classical optimization tricks to be effective on little, medium, and big size problems. LMA also implements common features as automatic differentiation and robust cost functions. The continuous-time representation of the trajectory is used to deal with constraints on the motion. This allows to mix data from many unsynchronised sensors and evolution model. To apply constraints on the 3D structure of the environment, 3D models of coarsely knowns shapes are used. To represent the motion, we use the uniform cumulative b-spline described by Lovegrove et al. [2] but we separate position and orientation in two different splines and we use the Rodriguez formula to compute the exponential and the logarithm of SO3 group. Moreover, we adapt the keyframe based SLAM to deal with the spline: key-frames are constrained to be on the spline. We also use every inter-key-frame poses computed by the localization process to apply a weak constraint on the spline. A temporal sliding window of 3 seconds is used to select, from the SLAM and the IMU, the more recent data used to constrain the spline. The MCSLAM algorithm is based on a graph composed by constraints, parameters and dependencies. First, the problem configuration is analysed at compile-time to generate a specified LMA solver, whose cost function to minimize is the sum of the constraints C dynamically

We propose a new real-time CSLAM algorithm (Constrained Simultaneous Localization And Mapping), named DCSLAM (dynamically constrained SLAM), designed to dynamically adapt each optimization to the variable number of parameters families and heterogeneous constraints. An automatic method is used to generate, from an exhaustive list of constraints, a dedicated optimization algorithm. This is, to our knowledge, the only implementation that combines flexibility and performance. The proposed experiments show the effectiveness of our approach in terms of accuracy and execution time compared to the state of the art on several public benchmarks of varying complexity. An augmented reality application mixing heterogeneous objects and constraints is also available.

This paper presents a new approach for multi-objects tracking from a video camera moving in an unknown environment. The tracking involves static objects of different known shapes, whose poses and sizes are determined online. For augmented reality applications, objects must be precisely tracked even if they are far from the camera or if they are hidden. Camera poses are computed using simultaneous localization and mapping (SLAM) based on bundle adjustment process to optimize problem parameters. We propose to include in an incremental bundle adjustment the parameters of the observed objects as well as the camera poses and 3D points. We show, through the example of 3D models of basics shapes (planes, parallelepipeds, cylinders and spheres) coarsely initialized online using a manual selection, that the joint optimization of parameters constrains the 3D points to approach the objects, and also constrains the objects to fit the 3D points. Moreover, we developed a generic and optimized library to solve this modified bundle adjustment and demonstrate the high performance of our solution compared to the state of the art alternative. Augmented reality experiments in realtime demonstrate the accuracy and the robustness of our method.

Learning and recognizing activity in videos is an especially important task in computer vision. However, it is hard to perform. In this paper, we propose a new method by combining local and global context information to extract a bag-of-words-like representation of a single space-time point. Each spacetime point is described by a bag of visual words that encodes its relationships with the remaining space-time points in the video, defining the space-time context. Experiments on the KTH benchmark of action recognition, show that our approach performs accurately compared to the state-of-the-art.

In this paper, we present a new algorithm named IPCC for Iterative Photo Consistency Check. The goal of this one is to detect a posteriori moving object in both camera and range data. The range data may be provided by different sensor such as: Riegl, Kinect or Velodyne with no distinction. The key idea is to consider that range data acquired on static objects are photo-consistent, they have the same color and texture in all the camera images, but range data acquired on moving object are not photo-consistent. The main problem is to take in account that range finding sensor and camera are not synchronous, so what is seen in camera is not what range finding sensors acquire. This work propose an original way to estimate photo-consistency of range data by using his 3D neighborhood as a texture descriptor. A Gaussian mixture method has been developed to deal with occluded background. Moreover, we will seen how to remove non photo-consistent range data from the scene by an erosion process and how to repair images by inpainting. Finally, experiments will show the relevance of the proposed method in terms of both accuracy and computation time.

We present in this paper a novel solution for temporal segmentation of human gestures that takes advantage of the skeletal-joints streams offered by the Kinect sensor. Our contribution consist in introducing an improved skeletal representation and its usage in a multilayer motion delimitation that distinguishes the non-vocabulary actions. The evaluation of the solution is presented on a subset of the Chalearn Gesture Challenge (CGC) 2014 dataset. The obtained temporal segmentation is better than the CGC baseline methods and has proved to be important for the task of human-action recognition.

In addition to multi-class classification, the multi-class object detection task consists further in classifying a dominating background label. In this work, we present a novel approach where relevant classes are ranked higher and background labels are rejected. To this end, we arrange the classes into a tree structure where the classifiers are trained in a joint framework combining ranking and classification constraints. Our convex problem formulation naturally allows to apply a tree traversal algorithm that searches for the best class label and progressively rejects background labels. We evaluate our approach on the PASCAL VOC 2007 dataset and show a considerable speed-up of the detection time with increased detection performance.

In this paper, we present a new algorithm named IPCC for Iterative Photo Consistency Check. The goal of this one is to detect a posteriori moving object in both camera and range data. The range data may be provided by different sensor such as: Riegl, Kinect or Velodyne with no distinction. The key idea is to consider that range data acquired on static objects are photo-consistent, they have the same color and texture in all the camera images, but range data acquired on moving object are not photo-consistent. The main problem is to take in account that range finding sensor and camera are not synchronous, so what is seen in camera is not what range finding sensors acquires. The major contribution in this paper is an original way to find non photo-consistent range data using the camera images in an erosion process of the scene. Experiments show the relevance of the proposed method in terms of both accuracy and computation time.

This paper presents a fast method to estimate the probability of occupancy of a space point from a huge set of 3D rays represented in a common reference. These data can come from any range finding sensor such as : Lidar, Kinect or Velodyne. The key idea is to consider that the occupancy of a space 3D point is linked to 1) the number of 3D point belonging to a local volume around the point and 2) the number of rays crossing through the same volume. We propose a probabilistic non-parametric framework based on KNN estimator. The major contribution of the paper is an original solution to search rays in the neighborhood of a 3D point with a five dimensional binary tree that can handle several millions measurements. Experiments shows the relevance of the proposed method in terms of both accuracy and computation time. Moreover, the resulting method has been applied to three different 3D sensors: a Kinect, a 3D Lidar (Velodyne HDL-64E) and a mono-planar Lidar.

This paper deals with the design of a generic visual tracking algorithm suitable for a large class of camera (single viewpoint sensors). It is based on the estimation of the relationship between observations and motion on the sphere. This is efficiently achieved using a kernel-based regression function on a generic linearly-weighted sum of non-linear basis functions. We also present two set of experiments. The first one shows the efficiency of our algorithm through the tracking in video sequences acquired with three types of cameras (conventional, dioptric-fisheye and catadioptric). The real-time performances will be shown by tracking one or several planes. The second set of experiments presents an application of our tracking algorithm to visual servoing with a fisheye camera.

Recent face detectors used in human robot interaction are boosted cascades. These cascades can detect upright faces but are very sensible to occlusions. We propose a generic framework to handle occlusions at prediction time in a boosted cascade. The contribution is a probabilistic formulation of the cascade structure that considers the uncertainty introduced by missing weak classifiers. This new formulation involves two problems: (1) the approximation of posterior probabilities on each level and (2) the computation of thresholds on these probabilities to make a decision. Both problems are studied and solutions are proposed and evaluated. The method is then applied on the problem of occluded faces detection. Experimental results are provided on classic databases to evaluate the proposed solution related to the basic one.

Pedestrians are the most vulnerable participants in urban traffic. The first step toward protecting pedestrians is to reliably detect them in a real-time framework. In this paper, a new approach is presented for pedestrian detection in urban traffic conditions using a multilayer laser sensor mounted onboard a vehicle. This sensor, which is placed on the front of a vehicle, collects information about the distance distributed according to four planes. Like a vehicle, a pedestrian constitutes, in the vehicle environment, an obstacle that must be detected, and located and then identified and tracked if necessary. To improve the robustness of pedestrian detection using a single laser sensor, a detection system based on the fusion of information located in the four laser planes is proposed. The method uses a nonparametric kernel-density-based estimation of pedestrian position of each laser plane. The resulting pedestrian estimations are then sent to a decentralized fusion according to the four planes. Temporal filtering of each object is finally achieved within a stochastic recursive Bayesian framework (particle filter), allowing a closer observation of pedestrian random movement dynamics. Many experimental results are given and validate the relevance of our pedestrian-detection algorithm with regard to a method using only a single-row laser-range scanner.

This paper addresses real-time automatic visual tracking, labeling and classification of a variable number of objects such as pedestrians or/and vehicles, under time-varying illumination conditions. The illumination and multi-object configuration are jointly tracked through a Markov Chain Monte-Carlo Particle Filter (MCMC PF). The measurement is provided by a static camera, associated to a basic foreground / background segmentation. As a first contribution, we propose in this paper to jointly track the light source within the Particle Filter, considering it as an additionnal object. Illumination-dependant shadows cast by objects are modeled and treated as foreground, thus avoiding the difficult task of shadow segmentation. As a second contribution, we estimate object category as a random variable also tracked within the Particle Filter, thus unifying object tracking and classification into a single process. Real time tracking results are shown and discussed on sequences involving various categories of users such as pedestrians, cars, light trucks and heavy trucks.

We present a multi modal sequential importance resampling particle filter algorithm for object tracking. We consider a hidden state sequence linked to several observation sequences given by different sensors. In a particle filter based framework, each sensor provides a likelihood (weight) associated to each particle and simple rules are applied to merge the different weights such as addition or product. We propose an original algorithm based on likelihood ratios to merge the observations within the sampling step. The algorithm is compared with classic fusion operations on toy examples. Moreover, we show that the method gives satisfactory results on a real vehicle tracking application.

Pedestrian recognition in images is a challenging task. Indeed a generic model must be able to describe the huge variability of pedestrians. We propose a learning based approach using a training set composed by positive and negative samples. A simple description of each candidate image provides a huge feature vector from which can be built weak classifiers. We select a subset of relevant weak classifiers using a classic AdaBoost algorithm. The resulting subset is then used as binary vectors in a kernel based machine learning classifier (like SVM, RVM, ...). The major contribution of the paper is the original association of an AdaBoost algorithm to select the relevant weak classifiers, followed by a SVM like classifier for which input data are given by the selected weak classifiers. Kernel based machine learning provides non-linear separator into the weak classifier space while standard AdaBoost gives a linear one. Performances of this method are compared to a classical AdaBoost method.

Pedestrian safety is a primary traffic issue in urban environment. This article deals with the detection of pedestrians by means of a laser sensor. This sensor, placed on the front of a vehicle collects information about distance distributed according to 4 laser planes. Like a vehicle, a pedestrian constitutes in the vehicle environment an obstacle which must be detected, located, then identified and tracked if necessary. In order to improve the robustness of pedestrian detection using a single laser sensor we propose here a detection system based on the fusion of information located in the 4 laser planes. In this paper, we propose a Parzen kernel method that allows first to isolate the ldquopedestrian objectsrdquo in each plane and then to carry out a decentralized fusion according to the 4 laser planes. Finally, to improve our pedestrian detection algorithm we use a MCMC based PF method allowing a closer observation of pedestrian random movement dynamics. Many experimental results validate and show the relevance of our pedestrian detection algorithm in regard to a method using only a single-row laser-range scanner.

In this paper we present a method for computing the localization of a mobile robot with reference to a learning video sequence. The robot is first guided on a path by a human, while the camera records a monocular learning sequence. Then a 3D reconstruction of the path and the environment is computed off line from the learning sequence. The 3D reconstruction is then used for computing the pose of the robot in real time (30 Hz) in autonomous navigation. Results from our localization method are compared to the ground truth measured with a differential GPS.

This paper deals with areas detection and tracking using color based image processing. The application addressed concerns human head and hands tracking. We propose an original method in order to locate, for a foreground image, colored areas. Foreground image is provided by a probabilistic way using Gaussian mixture models (GMM) of probability density functions. The temporal tracking is then achieved by a particle filter, which is well adapted to partial occlusions and non Gaussian models.

The paper deals with autonomous environment mapping, localisation and navigation using exclusively monocular vision and multiple 2D pattern tracking. The environment map is a mosaic of 2D patterns detected on the ceiling plane and used as natural landmarks. The robot is able to reproduce learned trajectories defined by key images representing the visual memory. The pattern tracker is based on particle filetring. It uses both image contours and gray scale level variations to track efficiently 2D patterns on cluttered background. An original observation model used for filter state updating is presented.

In this paper we present a method for computing the localization of a mobile robot with reference to a learning video sequence. The robot is first guided on a path by a human, while the camera records a monocular learning sequence. Then the computer builds a map of the environment. This is done by first extracting key frames from the learning sequence. Then the epipolar geometry and camera motion are computed between key frames. Additionally a hierarchical bundle adjustment is used to refine the reconstruction. The map stored for the localization include the position of the camera associated with each key frame as well as a set of interest points detected in the images and reconstructed in 3D. Using this map it is possible to compute the localization of the robot in real time during the automatic driving phase. 1

Real time vision systems in an outdoor environment involve an increase of the algorithm complexity in order to solve the application with an acceptable autonomy. In our application of help guidance for agricultural mobile machines, the vehicle has to follow the limit between a mowed and an unmowed natural zone. After the extraction of the image by a CCD camera, a low level algorithm computes some luminance and texture parameters for each elementary site. We define a correlation function for each parameter. The original feature of our work is a geometrical characterization of the correlation functions, as well as performs belief functions associated with each parameter. The uncertainty notion, expressed by the use of the theory of evidence allows the control of the reliability of the estimation. This approach is very important in outdoor environment where the system can be confronted in many situations.