Time‐of‐flight (TOF) sensors provide real‐time depth information at high frame‐rates. One issue with TOF sensors is the usual high level of noise (i.e . the depth measure's repeatability within a static setting). However, until now, TOF sensors’ noise has not been well studied. The authors show that the commonly agreed hypothesis that noise depends only on the amplitude information is not valid in practice. They empirically establish that the noise follows a signal‐dependent Gaussian distribution and varies according to pixel position, depth and integration time. They thus consider all these factors to model noise in two new noise models. Both models are evaluated, compared and used in the two following applications: depth noise removal by depth filtering and uncertainty (repeatability) estimation in three‐dimensional measurement.

Depth cameras open new possibilities in fields such as 3D reconstruction, Augmented Reality and video-surveillance since they provide depth information at high frame-rates. However, like any sensor, they have limitations related to their technology. One of them is depth distortion. In this paper, we present a method to estimate depth correction for depth cameras. The proposed method is based on two steps. The first one is a nonplanarity correction that needs depth measurement of different plane views. The second one is an affinity correction that,contrary to state of the art approaches, requires a very small set of ground truth measurements. Thus, it is more easy to use compared to other methods and does not need a large set of accurate ground truth that is extremely difficult to obtain in practice. Experiments on both simulated and real data show that the proposed approach improve also the depth accuracy compare to state of the art methods.

Time-of-Flight (TOF) cameras measure, in real-time, the distance between the camera and objects in the scene. This opens new perspectives in different application fields: 3D reconstruction, Augmented Reality, video-surveillance, etc. However, like any sensor, TOF cameras have limitations related to their technology. One of them is distance distortion. In this paper, we present a new depth calibration method (estimation of distance distortion) for TOF cameras. Our approach has several advantages. First, it is based on a non-parametric model, contrary to most of the other methods. Second, it models under the same formalism the distortion variation according to the distance and the pixel position in the image. This improves calibration accuracy even at the image boundaries which are typically more distorted than the image center. A comparison with two state of the art parametric methods is presented.

Constant Guillot1 [email protected] Maxime Taron1 [email protected] Patrick Sayd1 [email protected] Quoc-Cuong Pham1 [email protected] Christophe Tilmant2 [email protected] Jean-Marc Lavest2 [email protected] 1 CEA LIST Laboratoire Vision et Ingenierie des Contenus, BP 94, Gif-sur-Yvette, F-91191 France 2 LASMEA UMR 6602, PRES Clermont Universite/CNRS, 63177 Aubiere cedex, France

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well handled. In 2D image-based registration, self-occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based nonrigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based, "direct" data term for registration. Self-occlusions are detected as shrinkage areas in the 2D warp. Experimental results on several challenging data sets show that our approach successfully registers images with self-occlusions while effectively detecting the self-occluded regions.

We address the problem of deformable shape and motion recovery from point correspondences in multiple perspective images. We use the low-rank shape model, i.e. the 3D shape is represented as a linear combination of unknown shape bases. We propose a new way of looking at the low-rank shape model. Instead of considering it as a whole, we assume a coarse-to-fine ordering of the deformation modes, which can be seen as a model prior. This has several advantages. First, the high level of ambiguity of the original low-rank shape model is drastically reduced since the shape bases can not anymore be arbitrarily re-combined. Second, this allows us to propose a coarse-to-fine reconstruction algorithm which starts by computing the mean shape and iteratively adds deformation modes. It directly gives the sought after metric model, thereby avoiding the difficult upgrading step required by most of the other methods. Third, this makes it possible to automatically select the number of deformation modes as the reconstruction algorithm proceeds. We propose to incorporate two other priors, accounting for temporal and spatial smoothness, which are shown to improve the quality of the recovered model parameters. The proposed model and reconstruction algorithm are successfully demonstrated on several videos and are shown to outperform the previously proposed algorithms.

We address the issue of real-time pedestrians detection in a urban environment. This is a challenging task owing to the high variability of appearances and poses that humans can have and to the complexity of backgrounds. We propose a solution made of gradient-based local descriptors combined to form strong classifiers and organized in a cascaded detector. We developed for this an extension of the Histograms of Oriented Gradients (HOGs) and added a new component to the histogram which represents the strength of edges or the amount of information in the histogram support. We also implemented a learning algorithm based on Real Adaboost where two phases – selection first, then refinement of weights – provide more robustness to the detector. We evaluated our system by comparing it to the cascaded detector of Haar features of Viola & Jones [7] and to the SVM of HOGs features of Dalal & Triggs [1]. To ensure an equitable and valid comparison, we used the database proposed in [1]. Our system outperforms them in detection results and in time needs. 1

The direct registration problem for images of a deforming surface has been well studied. Parametric flexible warps based, for instance, on the Free-Form Deformation or a Radial Basis Function such as the Thin-Plate Spline, are often estimated using additive Gauss-Newton-like algorithms. The recently proposed compositional framework has been shown to be more efficient, but cannot be directly applied to such non-groupwise warps. Our main contribution in this paper is the Feature-Driven framework. It makes possible the use of compositional algorithms for most parametric warps such as those above mentioned. Two algorithms are proposed to demonstrate the relevance of our Feature-Driven framework: the Feature-Driven Inverse Compositional and the Feature-Driven Learning-based algorithms. As another contribution, a detailed derivation of the Feature-Driven warp parameterization is given for the Thin-Plate Spline and the Free-Form Deformation. We experimentally show that these two types of warps have a similar representational power. Experimental results show that our Feature-Driven registration algorithms are more efficient in terms of computational cost, without loss of accuracy, compared to existing methods.

International audience

This article describes a video-surveillance system developed within the ISCAPS project. Thermal imaging provides a robust solution to visibility change (illumination, smoke) and is a relevant technology for discriminating humans in complex scenes. In this article, we demonstrate its efficiency for posture analysis in dense groups of people. The objective is to automatically detect several persons lying down in a very crowded area. The presented method is based on the detection and segmentation of individuals within groups of people using a combination of several weak classifiers. The classification of extracted silhouettes enables to detect abnormal situations. This approach was successfully applied to the detection of terrorist gas attacks on railway platform and experimentally validated in the project. Some of the results are presented here.

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well-handled. In 2D image-based registration, self- occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based non-rigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based, i.e. direct, data term for registration. Self-occlusions are detected as shrinking areas in the 2D warp. Experimental results on several challenging datasets show that our approach successfully registers images with self-occlusions while effectively detecting the occluded regions.

The augmentation problem for images of a deforming surface has been studied since recently. The surface is usually assumed not to be self-occluding. Two dimensional deformation estimation in the presence of self-occlusions is very challenging. This paper proposes a specific framework explicitly modeling self-occlusions for augmented reality applications. The basic idea is to detect self-occlusions as warp shrinkage areas. Deformations are initially estimated via direct non-rigid image registration. Temporal smoothness is then used to refine the warps and the image are augmented. Experimental results on several challenging datasets show that our approach convincingly augments self-occluded surfaces. Associated videos are available on the first author's Web homepage.

In this paper we describe a method that estimates the motion of a calibrated camera (settled on an experimental vehicle) and the tridimensional geometry of the environment. The only data used is a video input. In fact, interest points are tracked and matched between frames at video rate. Robust estimates of the camera motion are computed in real-time, key-frames are selected and permit the features 3D reconstruction. The algorithm is particularly appropriate to the reconstruction of long images sequences thanks to the introduction of a fast and local bundle adjustment method that ensures both good accuracy and consistency of the estimated camera poses along the sequence. It also largely reduces computational complexity compared to a global bundle adjustment. Experiments on real data were carried out to evaluate speed and robustness of the method for a sequence of about one kilometer long. Results are also compared to the ground truth measured with a differential GPS.

Registering images of a deforming surface is a well-studied problem. It is common practice to describe the image deformation fields with thin-plate splines. This has the advantage to involve small numbers of parameters, but has the drawback that the 3D surface is not explicitly reconstructed. We propose an image deformation model combining thin-plate splines with 3D entities-a 3D control mesh and a camera-overcoming the above mentioned drawback. An original solution to the non-rigid image registration problem using this model is proposed and demonstrated on simulated and real data.

This paper introduces an incremental method for "structure from motion" of complex scenes from a video sequence. More precisely, we estimate the 3D positions of the viewed points in images and the camera positions and orientations through the sequence. The method can be seen as a fast but accurate alternative to classical reconstruction methods that use bundle adjustment, and that can become slow and computation time expensive for very long scenes. Our results are compared to the reconstruction obtained by the classical hierarchical bundle adjustment method. They have also been successfully used as a reference sequence for the vision based localization of an autonomous mobile robot

A complete tool chain starting with stereo photogrammetry based digitization of artefacts, their refinement, collection and management with other multimedia data, and visualization using virtual and augmented reality is presented. Our system provides a one-stop-solution for museums to create, manage and present both content and context for virtual exhibitions. Interoperability and standards are also key features of our system allowing both small and large museums to build a bespoke system suited to their needs.

We present a supervised approach to recover 3D models of buildings from multiple uncalibrated views. With this method the user matches 3D vertices in the images and defines the 3D model of the building with the help of elementary and intuitive geometric constraints. At the same time, a graph describing relationships between vertices is built. Then, unknown parameters of this graph are estimated non-linearly through a bundle adjustment to recover the building model and the camera parameters. This method asserts that geometric rules are perfectly respected. This approach is used to recover independently 3D parts of the building with suitable images. Then all these independent 3D models are merged to obtain a full multiscale model of the building. An example on real images is given.

Bundle adjustment is one of the cornerstone to recover the scene structure from a sequence of images. The main drawback of this technique, due to nonlinear optimisation, is the need of initial conditions for intrinsic and extrinsic camera parameters and for the 3D structure that we want to reconstruct.