We present a novel and effective method for detecting 3D primitives in cluttered, unorganized point clouds, without axillary segmentation or type specification. We consider the quadric surfaces for encapsulating the basic building blocks of our environments - planes, spheres, ellipsoids, cones or cylinders, in a unified fashion. Moreover, quadrics allow us to model higher degree of freedom shapes, such as hyperboloids or paraboloids that could be used in non-rigid settings. We begin by contributing two novel quadric fits targeting 3D point sets that are endowed with tangent space information. Based upon the idea of aligning the quadric gradients with the surface normals, our first formulation is exact and requires as low as four oriented points. The second fit approximates the first, and reduces the computational effort. We theoretically analyze these fits with rigor, and give algebraic and geometric arguments. Next, by re-parameterizing the solution, we devise a new local Hough voting scheme on the null-space coefficients that is combined with RANSAC, reducing the complexity from O(N4) to O(N3) (three points). To the best of our knowledge, this is the first method capable of performing a generic cross-type multi-object primitive detection in difficult scenes without segmentation. Our extensive qualitative and quantitative results show that our method is efficient and flexible, as well as being accurate.

It is well known that the accuracy of a calibration depends strongly on the choice of camera poses from which images of a calibration object are acquired. We present a system - Calibration Wizard - that interactively guides a user towards taking optimal calibration images. For each new image to be taken, the system computes, from all previously acquired images, the pose that leads to the globally maximum reduction of expected uncertainty on intrinsic parameters and then guides the user towards that pose. We also show how to incorporate uncertainty in corner point position in a novel principled manner, for both, calibration and computation of the next best pose. Synthetic and real-world experiments are performed to demonstrate the effectiveness of Calibration Wizard.

This paper proposes a segmentation-free, automatic and efficient procedure to detect general geometric quadric forms in point clouds, where clutter and occlusions are inevitable. Our everyday world is dominated by man-made objects which are designed using 3D primitives (such as planes, cones, spheres, cylinders, etc.). These objects are also omnipresent in industrial environments. This gives rise to the possibility of abstracting 3D scenes through primitives, thereby positions these geometric forms as an integral part of perception and high level 3D scene understanding. As opposed to state-of-the-art, where a tailored algorithm treats each primitive type separately, we propose to encapsulate all types in a single robust detection procedure. At the center of our approach lies a closed form 3D quadric fit, operating in both primal & dual spaces and requiring as low as 4 oriented-points. Around this fit, we design a novel, local null-space voting strategy to reduce the 4-point case to 3. Voting is coupled with the famous RANSAC and makes our algorithm orders of magnitude faster than its conventional counterparts. This is the first method capable of performing a generic cross-type multi-object primitive detection in difficult scenes. Results on synthetic and real datasets support the validity of our method.

High resolution population count data are vital for numerous applications such as urban planning, transportation model calibration, and population growth impact measurements, among others. In this work, we present and evaluate an end-to-end framework for computing disaggregated population mapping employing convolutional neural networks (CNNs). Using urban data extracted from the OpenStreetMap database, a set of urban features are generated which are used to guide population density estimates at a higher resolution. A population density grid at a 200 by 200 meter spatial resolution is estimated, using as input gridded population data of 1 by 1 kilometer. Our approach relies solely on open data with a wide geographical coverage, ensuring replicability and potential applicability to a great number of cities in the world. Fine-grained gridded population data is used for 15 French cities in order to train and validate our model. A stand-alone city is kept out for the validation procedure. The results demonstrate that the neural network approach using massive OpenStreetMap data outperforms other approaches proposed in related works.

This paper proposes a unified theory for calibrating a wide variety of camera models such as pinhole, fisheye, cata-dioptric, and multi-camera networks. We model any camera as a set of image pixels and their associated camera rays in space. Every pixel measures the light traveling along a (half-) ray in 3-space, associated with that pixel. By this definition, calibration simply refers to the computation of the mapping between pixels and the associated 3D rays. Such a mapping can be computed using images of calibration grids, which are objects with known 3D geometry, taken from unknown positions. This general camera model allows to represent non-central cameras; we also consider two special subclasses, namely central and axial cameras. In a central camera, all rays intersect in a single point, whereas the rays are completely arbitrary in a non-central one. Axial cameras are an intermediate case: the camera rays intersect a single line. In this work, we show the theory for calibrating central, axial and non-central models using calibration grids, which can be either three-dimensional or planar.

Population in urban areas has been increasing at an alarming rate in the last decades. This evidence, together with the rising availability of massive data from cities, has motivated research on sustainable urban development. In this paper we present a GIS-based land use mix analysis framework to help urban planners to compute indices for mixed uses development, which may be helpful towards developing sustainable cities. Residential and activities land uses are extracted using OpenStreetMap crowd-sourcing data. Kernel density estimation is performed for these land uses, and then used to compute the mixed uses indices. The framework is applied to several cities, analyzing the land use mix output.

Pushbroom cameras are widely used for earth observation applications. This sensor acquires 1D images over time and uses the straight motion of the satellite to sweep out a region of space and build a 2D image. The stability of the satellite is critical during the pushbroom acquisition process. Therefore its attitude is assumed to be constant overtime. However, the recent manufacture of smaller and lighter satellites to reduce launching cost has weakened this assumption. Small oscillations of the satellite's attitude can result in noticeable warps in images, and geolocation information is lost as the satellite does not capture what it ought to. Current solutions use inertial sensors to control the attitude and correct the images, but they are costly and of limited precision. As the warped images do contain information about attitude variations, we suggest using image registration to estimate them. We exploit the geometry of the focal plane and the stationary nature of the disturbances to recover undistorted images. We embed the estimation in a Bayesian framework where image registration, a prior on attitude variations and a radiometric correction model are fused to retrieve the motion of the satellite. We illustrate the performance of our algorithm on four satellite datasets.

We present a joint prior that takes intensity and depth information into account. The prior is defined using a flexible Field-of-Experts model and is learned from a dataBase of natural images. It is a generative model and has an efficient method for sampling. We use sampling from the model to perform in painting and up sampling of depth maps when intensity information is available. We show that including the intensity information in the prior improves the results obtained from the model. We also compare to another two-channel in painting approach and show superior results.

3D reconstruction of transparent refractive objects like a plastic bottle is challenging: they lack appearance related visual cues and merely reflect and refract light from the surrounding environment. Amongst several approaches to reconstruct such objects, the seminal work of Light-Path triangulation is highly popular because of its general applicability and analysis of minimal scenarios. A light-path is defined as the piece-wise linear path taken by a ray of light as it passes from source, through the object and into the camera. Transparent refractive objects not only affect the geometric configuration of light-paths but also their radiometric properties. In this paper, we describe a method that combines both geometric and radiometric information to do reconstruction. We show two major consequences of the addition of radiometric cues to the light-path setup. Firstly, we extend the case of scenarios in which reconstruction is plausible while reducing the minimal requirements for a unique reconstruction. This happens as a consequence of the fact that radiometric cues add an additional known variable to the already existing system of equations. Secondly, we present a simple algorithm for reconstruction, owing to the nature of the radiometric cue. We present several synthetic experiments to validate our theories, and show high quality reconstructions in challenging scenarios.

This paper presents a single camera and laser system dedicated to the realtime estimation of attitude and altitude for unmanned aerial vehicles (UAV) under low illumination conditions to dark environments. The fisheye camera allows to cover a large field of view (FOV). The approach, close to structured light systems, uses the geometrical information obtained by the projection of a laser circle onto the ground plane and perceived by the camera. We propose some experiments based on simulated data and real sequences. The results show good agreement with the ground truth values from the commercial sensors in terms of its accuracy and correctness. The results also prove its suitability for autonomous take-off and landing as well as for the case of low altitude manoeuvre in dark, GPS signal deficient unknown environments with no prebuilt map. It also provides room for additional payload to be used for different applications due to it being inexpensive and use of light weight micro-camera and laser system.

We present a method for real-time 3D object instance detection that does not require a time-consuming training stage, and can handle untextured objects. At its core, our approach is a novel image representation for template matching designed to be robust to small image transformations. This robustness is based on spread image gradient orientations and allows us to test only a small subset of all possible pixel locations when parsing the image, and to represent a 3D object with a limited set of templates. In addition, we demonstrate that if a dense depth sensor is available we can extend our approach for an even better performance also taking 3D surface normal orientations into account. We show how to take advantage of the architecture of modern computers to build an efficient but very discriminant representation of the input images that can be used to consider thousands of templates in real time. We demonstrate in many experiments on real data that our method is much faster and more robust with respect to background clutter than current state-of-the-art methods.

International audience

This paper extends classical object pose and relative camera motion estimation algorithms for imaging sensors sampling the scene through light-paths. Many algorithms in multi-view geometry assume that every pixel observes light traveling in a single line in space. We wish to relax this assumption and address various theoretical and practical issues in modeling camera rays as piece-wise linear-paths. Such paths consisting of finitely many linear segments are typical of any simple camera configuration with reflective and refractive elements. Our main contribution is to propose efficient algorithms that can work with the complete light-path without knowing the correspondence between their individual segments and the scene points. Second, we investigate light-paths containing infinitely many and small piece-wise linear segments that can be modeled using simple parametric curves such as conics. We show compelling simulations and real experiments, involving catadioptric configurations and mirages, to validate our study.

Access to the scene irradiance is a desirable feature in many computer vision algorithms. Applications like BRDF estimation, relighting or augmented reality need measurements of the object's photometric properties and the simplest method to get them is using a camera. However, the first step necessary to achieve this goal is the computation of the function that relates scene irradiance to image intensities. In this paper we propose to exploit the large variety of an object's appearances in photo collections to recover this non linear function for each of the cameras that acquired the available images. This process, also known as radiometric calibration, uses an unstructured set of images, to recover the camera's geometric calibration and a 3D scene model, using available methods. From this input, the camera response function is estimated for each image. This highly ill-posed problem is made tractable by using appropriate priors. The proposed approach is based on the empirical prior on camera response functions introduced by Grossberg and Nayar. Linear methods are proposed that allow to compute approximate solutions, which are then refined by non-linear least squares optimization.

Most of the computer and robot-vision algorithms are designed mainly for opaque objects and non-opaque objects have received less attention, in spite of them being omnipresent in man-made environments. With an increasing usage of such objects, especially those made of glass, plastic etc., it becomes necessarily important to detect this class of objects while building a robot navigation system. Obstacle avoidance forms a primary yet challenging task in mobile robot navigation. The main objective of this paper is to present an algorithm to detect and avoid obstacles that are made of semi-transparent materials, such as plastic or glass. The algorithm makes use of a technique called the collective-reward based approach to detect such objects from single images captured by an uncalibrated camera in a live video stream. Random selection techniques are incorporated in the method to make the algorithm run in real-time. A mobile robot then uses the information after detection to perform an obstacle avoidance maneuver. Experiments were conducted on a real robot to test the efficacy of the algorithm.

This paper identifies and fills the probably last two missing items in minimal pose estimation algorithms using points and lines. Pose estimation refers to the problem of recovering the pose of a calibrated camera given known features (points or lines) in the world and their projections on the image. There are four minimal configurations using point and line features: 3 points, 2 points and 1 line, 1 point and 2 lines, 3 lines. The first and the last scenarios that depend solely on either points or lines have been studied a few decades earlier. However the mixed scenarios, which are more common in practice, have not been solved yet. In this paper we show that it is indeed possible to develop a general technique that can solve all four scenarios using the same approach and that the solutions involve computing the roots of either a 4th degree or an 8th degree equation. The centerpiece of our method is a simple and generic method that uses collinearity and coplanarity constraints for solving the pose. In addition to validating the performance of these algorithms in simulations, we also show a compelling application for geo-localization using image sequences and coarse (plane-based) 3D models of GPS challenged urban canyons.

This paper presents a system dedicated to the real-time estimation of attitude and altitude for unmanned aerial vehicles (UAV) under low light and dark environment. This system consists in a fisheye camera, which allows to cover a large field of view (FOV), and a laser circle projector mounted on a fixed baseline. The approach, close to structured light systems, uses the geometrical information obtained by the projection of the laser circle onto the ground plane and perceived by the camera. We present a theoretical study of the system in which the camera is modelled as a sphere and show that the estimation of a conic on this sphere allows to obtain the attitude and the altitude of the robot. We propose some experiments based on simulated data and real sequences. The estimated attitude and altitude from our method are comparable with commercial sensors in terms of its accuracy and correctness. The results also prove its suitability for autonomous take-off and landing as well as for the case of low altitude manoeuvre in dark environments. It also provides room for additional payload to be used for different applications due to use of light weight micro-camera and laser system.

Linear pushbroom cameras are widely used in passive remote sensing from space as they provide high resolution images. In earth observation applications, where several pushbroom sensors are mounted in a single focal plane, small dynamic disturbances of the satellite's orientation lead to noticeable geometrical distortions in the images. In this paper, we present a global method to estimate those disturbances, which are effectively vibrations. We exploit the geometry of the focal plane and the stationary nature of the disturbances to recover undistorted images. To do so, we embed the estimation process in a Bayesian framework. An autoregressive model is used as a prior on the vibrations. The problem can be seen as a global image registration task where multiple pushbroom images are registered to the same coordinate system, the registration parameters being the vibration coefficients. An alternating maximisation procedure is designed to obtain Maximum a Posteriori estimates (MAP) of the vibrations as well as of the autoregressive model coefficients. We illustrate the performance of our algorithm on various datasets of satellite imagery1.

Satellite image registration has been investigated for several years. Nevertheless, little attention has been paid to the linear geometry of the satellite's imaging sensor, often consisting of several pushbroom cameras. Each pushbroom camera captures 1-D image and uses straight motion of the satellite to build a 2-D image. Yet, attitude variations of the satellite during the aquisition process can lead to significant distortions in the 2-D image. In this paper, we expose the problem and present a constrained image registration method to estimate the satellite's attitude variations, and thus correct the distorted images. We use a Lucas Kanade framework and a piecewise polynomial model under constraints to deduce the registration equation. The performances of our algorithm are shown on two satellite datasets.

We describe a pipeline for structure-from-motion with mixed camera types, namely omni directional and perspective cameras. The steps of the pipeline can be summarized as calibration, point matching, pose estimation, triangulation and bundle adjustment. For these steps, we either propose improved methods or modify existing perspective camera methods to make the pipeline more effective and automatic when employed for hybrid camera systems.

Pushbroom sensors are widely used for earth observation from space; this camera captures 1-D image and uses straight motion of the satellite to build a 2-D image. Yet, attitude variations of the satellite during the aquisition process can lead to significant distortions in the 2-D image. Current solutions use inertial sensors to record attitude variations and correct images, but they are expensive and not accurate enough for high frequency variations. In this paper, we expose the problem and present an image registration method to estimate the satellite's attitude variations, and thus correct the distorted images. We use a Lucas Kanade framework to deduce the registration equation; the performances of our algorithm are shown on two satellite datasets.

This paper investigates the problem of geo-localization in GPS challenged urban canyons using only skylines. Our proposed solution takes a sequence of upward facing omnidirectional images and coarse 3D models of cities to compute the geo-trajectory. The camera is oriented upwards to capture images of the immediate skyline, which is generally unique and serves as a fingerprint for a specific location in a city. Our goal is to estimate global position by matching skylines extracted from omni-directional images to skyline segments from coarse 3D city models. Under day-time and clear sky conditions, we propose a sky-segmentation algorithm using graph cuts for estimating the geo-location. In cases where the skyline gets affected by partial fog, night-time and occlusions from trees, we propose a shortest path algorithm that computes the location without prior sky detection. We show compelling experimental results for hundreds of images taken in New York, Boston and Tokyo under various weather and lighting conditions (daytime, foggy dawn and night-time).

Abstract A perspective image represents the spatial relationships of objects in a scene as they appear from a single viewpoint. In contrast, a multi‐perspective image combines what is seen from several viewpoints into a single image. Despite their incongruity of view, effective multi‐perspective images are able to preserve spatial coherence and can depict, within a single context, details of a scene that are simultaneously inaccessible from a single view, yet easily interpretable by a viewer. In computer vision, multi‐perspective images have been used for analysing structure revealed via motion and generating panoramic images with a wide field‐of‐view using mirrors. In this STAR, we provide a practical guide on topics in multi‐perspective modelling and rendering methods and multi‐perspective imaging systems. We start with a brief review of multi‐perspective image techniques frequently employed by artists such as the visual paradoxes of Escher, the Cubism of Picasso and Braque and multi‐perspective panoramas in cel animations. We then characterize existing multi‐perspective camera models, with an emphasis on their underlying geometry and image properties. We demonstrate how to use these camera models for creating specific multi‐perspective rendering effects. Furthermore, we show that many of these cameras satisfy the multi‐perspective stereo constraints and we demonstrate several multi‐perspective imaging systems for extracting 3D geometry for computer vision. The participants learn about topics in multi‐perspective modelling and rendering for generating compelling pictures for computer graphics and in multi‐perspective imaging for extracting 3D geometry for computer vision. We hope to provide enough fundamentals to satisfy the technical specialist without intimidating curious digital artists interested in multi‐perspective images. The intended audience includes digital artists, photographers and computer graphics and computer vision researchers using or building multi‐perspective cameras. They will learn about multi‐perspective modelling and rendering, along with many real world multi‐perspective imaging systems.

International audience

International audience

We present a new shape-from-distortion framework for recovering specular (reflective/refractive) surfaces. While most existing approaches rely on accurate correspondences between 2D pixels and 3D points, we focus on analyzing the curved images of 3D lines which we call curved line images or CLIs. Our approach models CLIs of local reflections or refractions using the recently proposed general linear cameras (GLCs). We first characterize all possible CLIs in a GLC. We show that a 3D line will appear as a conic in any GLC. For a fixed GLC, the conic type is invariant to the position and orientation of the line and is determined by the GLC parameters. Furthermore, CLIs under single reflection/refraction can only be lines or hyperbolas. Based on our new theory, we develop efficient algorithms to use multiple CLIs to recover the GLC camera parameters. We then apply the curvature-GLC theory to derive the Gaussian and mean curvatures from the GLC intrinsics. This leads to a complete distortion-based reconstruction framework. Unlike conventional correspondence-based approaches that are sensitive to image distortions, our approach benefits from the CLI distortions. Finally, we demonstrate applying our framework for recovering curvature fields on both synthetic and real specular surfaces.

Stereo matching and volumetric reconstruction are the most explored 3D scene recovery techniques in computer vision. Many existing approaches assume perspective input images and use the epipolar constraint to reduce the search space and improve the accuracy. In this paper we present a novel framework that uses multi-perspective cameras for stereo matching and volumetric reconstruction. Our approach first decomposes a multi-perspective camera into piecewise primitive General Linear Cameras or GLCs. A pair of GLCs in general do not satisfy the epipolar constraint. However, they still form a nearly stereo pair. We develop a new Graph-Cut-based algorithm to account for the slight vertical parallax using the GLC ray geometry. We show that the recovered pseudo disparity map conveys important depth cues analogous to perspective stereo matching. To more accurately reconstruct a 3D scene, we develop a new multi-perspective volumetric reconstruction method. We discretize the scene into voxels and apply the GLC back-projections to map the voxel onto each input multi-perspective camera. Finally, we apply the graph-cut algorithm to optimize the 3D embedded voxel graph. We demonstrate our algorithms on both synthetic and real multi-perspective cameras. Experimental results show that our methods are robust and reliable.

We present a novel calibration technique for all central catadioptric cameras using images of planar grids. We adopted the well-known sphere camera model to describe the catadioptric projection. We show that, using the so-called lifted coordinates, a linear relation mapping the grid points to the corresponding points on the image plane can be written as a 6 × 6 matrix Hcata, which acts like the classical 3 × 3 homography for perspective cameras. We show how to compute the image of the absolute conic (IAC) from at least 3 homographies and how to recover from it the intrinsic parameters of the catadioptric camera. In the case of paracatadioptric cameras one such homography is enough to estimate the IAC, thus allowing the calibration from a single image.

This purely theoretical work investigates the problem of artificial singularities in camera self-calibration. Self-calibration allows one to upgrade a projective reconstruction to metric and has a concise and well-understood formulation based on the Dual Absolute Quadric (DAQ), a rank-3 quadric envelope satisfying (nonlinear) ‘spectral constraints’: it must be positive of rank 3. The practical scenario we consider is the one of square pixels, known principal point and varying unknown focal length, for which generic Critical Motion Sequences (CMS) have been thoroughly derived. The standard linear self-calibration algorithm uses the DAQ paradigm but ignores the spectral constraints. It thus has artificial CMSs, which have barely been studied so far. We propose an algebraic model of singularities based on the confocal quadric theory. It allows to easily derive all types of CMSs. We first review the already known generic CMSs, for which any self-calibration algorithm fails. We then describe all CMSs for the standard linear self-calibration algorithm; among those are artificial CMSs caused by the above spectral constraints being neglected. We then show how to detect CMSs. If this is the case it is actually possible to uniquely identify the correct self-calibration solution, based on a notion of signature of quadrics. The main conclusion of this paper is that a posteriori enforcing the spectral constraints in linear self-calibration is discriminant enough to resolve all artificial CMSs.

We present algorithms for plane-based calibration of general radially distorted cameras. By this, we understand cameras that have a distortion center and an optical axis such that the projection rays of pixels lying on a circle centered on the distortion center form a right viewing cone centered on the optical axis. The camera is said to have a single viewpoint (SVP) if all such viewing cones have the same apex (the optical center); otherwise, we speak of NSVP cases. This model encompasses the classical radial distortion model [5], fisheyes, and most central or noncentral catadioptric cameras. Calibration consists in the estimation of the distortion center, the opening angles of all viewing cones, and their optical centers. We present two approaches of computing a full calibration from dense correspondences of a single or multiple planes with known euclidean structure. The first one is based on a geometric constraint linking viewing cones and their intersections with the calibration plane (conic sections). The second approach is a homography-based method. Experiments using simulated and a broad variety of real cameras show great stability. Furthermore, we provide a comparison with Hartley-Kang's algorithm [12], which, however, cannot handle such a broad variety of camera configurations, showing similar performance.

International audience

A generic imaging model refers to a non-parametric camera model where every camera is treated as a set of unconstrained projection rays. Calibration would simply be a method to map the projection rays to image pixels; such a mapping can be computed using plane based calibration grids. However, existing algorithms for generic calibration use more point correspondences than the theoretical minimum. It has been well-established that non-minimal solutions for calibration and structure-from-motion algorithms are generally noise-prone compared to minimal solutions. In this work we derive minimal solutions for generic calibration algorithms. Our algorithms for generally central cameras use 4 point correspondences in three calibration grids to compute the motion between the grids. Using simulations we show that our minimal solutions are more robust to noise compared to non-minimal solutions. We also show very accurate distortion correction results on fisheye images.

A perspective image represents the spatial relationships of objects in a scene as they appear from a single viewpoint. In contrast, a multiperspective image combines what is seen from several viewpoints into a single image. Despite their incongruity of view, effective multiperspective images are able to preserve spatial coherence and can depict, within a single context, details of a scene that are simultaneously inaccessible from a single view, yet easily interpretable by a viewer. In computer vision, multiperspective images have been used for analyzing structure revealed via motion and generating panoramic images with a wide field-of-view using mirrors.

This paper addresses the problem of image-based surface reconstruction. The main contribution is the computation of the exact derivative of the reprojection error functional. This allows its rigorous minimization via gradient descent surface evolution. The main difficulty has been to correctly take into account the visibility changes that occur when the surface moves. A geometric and analytical study of these changes is presented and used for the computation of derivative. Our analysis shows the strong influence that the movement of the contour generators has on the reprojection error. As a consequence, during the proper minimization of the reprojection error, the contour generators of the surface are automatically moved to their correct location in the images. Therefore, current methods adding additional silhouettes or apparent contour constraints to ensure this alignment can now be understood and justified by a single criterion: the reprojection error.

We present an algorithm for plane-based self-calibration of cameras with radially symmetric distortions given a set of sparse feature matches in at least two views. The projection function of such cameras can be seen as a projection with a pinhole camera, followed by a non-parametric displacement of the image points in the direction of the distortion center. The displacement is a function of the points' distance to the center. Thus, the generated distortion is radially symmetric. Regular cameras, fish-eyes as well as the most popular central catadioptric devices can be described by such a model. Our approach recovers a distortion function consistent with all the views, or estimates one for each view if they are taken by different cameras. We consider a least squares algebraic solution for computing the homography between two views that is valid for rectified (undistorted) point correspondences. We observe that the terms of the function are bilinear in the unknowns of the homography and the distortion coefficient associated to each point. Our contribution is to approximate this non-convex problem by a convex one. To do so, we replace the bilinear terms by a set of new variables and obtain a linear least squares problem. We show that like the distortion coefficients, these variables are subject to monotonicity constraints. Thus, the approximate problem is a convex quadratic program. We show that solving it is sufficient for accurately estimating the distortion parameters. We validate our approach on simulated data as well as on fish-eye and catadioptric cameras. We also compare our solution to three state-of-the-art algorithms and show similar performance.

This paper addresses the problem of reconstructing the geometry and color of a Lambertian scene, given some fully calibrated images acquired with wide baselines. In order to completely model the input data, we propose to represent the scene as a set of colored depth maps, one per input image. We formulate the problem as a Bayesian MAP problem which leads to an energy minimization method. Hidden visibility variables are used to deal with occlusion, reflections and outliers. The main contributions of this work are: a prior for the visibility variables that treats the geometric occlusions; and a prior for the multiple depth maps model that smoothes and merges the depth maps while enabling discontinuities. Real world examples showing the efficiency and limitations of the approach are presented.

We consider the problem of calibrating a highly generic imaging model, that consists of a non-parametric association of a projection ray in 3D to every pixel in an image. Previous calibration approaches for this model do not seem to be directly applicable for cameras with large fields of view and non-central cameras. In this paper, we describe a complete calibration approach that should in principle be able to handle any camera that can be described by the generic imaging model. Initial calibration is performed using multiple-images of overlapping calibration grids simultaneously. This is then improved using pose estimation and bundle adjustment-type algorithms. The approach has been applied on a wide variety of central and non-central cameras including fisheye lens, catadioptric cameras with spherical and hyperbolic mirrors, and multi-camera setups. We also consider the question if non-central models are more appropriate for certain cameras than central models.

We consider the structure-from-motion problem for a highly general imaging model where cameras are modeled as possibly unconstrained sets of projection rays. This allows to describe most existing camera types, including pinhole cameras, sensors with radial or more general distortions, catadioptric cameras (central or non-central), etc. We introduce a hierarchy of general camera models: the most general model has unconstrained projection rays whereas the most constrained model dealt with here is the central model, where all rays pass through a single point. Intermediate models are what we call axial cameras (all rays touch a single line), and x-slit cameras (rays touch two lines). The foundations for a multi-view geometry of completely non-central cameras are given, leading to the formulation of multi-view matching tensors, analogous to the fundamental/essential matrices, trifocal and quadrifocal tensors of perspective cameras. This framework is then specialized explicitly for the two-view case, for the intermediate camera types mentioned above.

Camera calibration with one-dimensional objects is based on an algebraic constraint on the image of the absolute conic. We give an alternative derivation to this constraint, allowing a geometrical interpretation. From this, we derive the degenerate cases, or critical motions, where the calibration algorithm fails. We also show that constraints on the intrinsic parameters lead to simplified closed-form solutions and a reduced set of critical motions. A simulation and a real data experiment is performed to evaluate the accuracy of the calibration result for motions close to being critical.

This paper concerns the incorporation of geometric information in camera calibration and 3D modeling. Using geometric constraints enables more stable results and allows us to perform tasks with fewer images. Our approach is motivated and developed within a framework of semi-automatic 3D modeling, where the user defines geometric primitives and constraints between them. It is based on the observation that constraints, such as coplanarity, parallelism, or orthogonality, are often embedded intuitively in parallelepipeds. Moreover, parallelepipeds are easy to delineate by a user and are well adapted to model the main structure of, e.g., architectural scenes. In this paper, first a duality that exists between the shape parameters of a parallelepiped and the intrinsic parameters of a camera is described. Then, a factorization-based algorithm exploiting this relation is developed. Using images of parallelepipeds, it allows us to simultaneously calibrate cameras, recover shapes of parallelepipeds, and estimate the relative pose of all entities. Besides geometric constraints expressed via parallelepipeds, our approach simultaneously takes into account the usual self-calibration constraints on cameras. The proposed algorithm is completed by a study of the singular cases of the calibration method. A complete method for the reconstruction of scene primitives that are not modeled by parallelepipeds is also briefly described. The proposed methods are validated by various experiments with real and simulated data, for single-view as well as multiview cases.

The purpose of this paper is to give a very simple method for nonlinearly estimating the fundamental matrix using the minimum number of seven parameters. Instead of minimally parameterizing it, we rather update what we call its orthonormal representation, which is based on its singular value decomposition. We show how this method can be used for efficient bundle adjustment of point features seen in two views. Experiments on simulated and real data show that this implementation performs better than others in terms of computational cost, i.e., convergence is faster, although methods based on minimal parameters are more likely to fall into local minima than methods based on redundant parameters.

International audience

We consider the problem of camera self-calibration from images of a planar object with unknown Euclidean structure. The general case of possibly varying focal length is addressed. This problem is nonlinear in general. One of our contributions is a nonlinear approach that makes abstraction of the (possibly varying) focal length, resulting in a computationally efficient algorithm. In addition, it does not require a good initial estimate of the focal length, unlike previous approaches. As for the initialization of other parameters, we propose a practical approach that simply requires taking one image in roughly fronto-parallel position. Closed-form solutions for various configurations of unknown intrinsic parameters are provided. Our methods are evaluated and compared to previous approaches using simulated and real images. Besides our practical contributions, we also provide a detailed geometrical interpretation of the principles underlying our approach.

We address the problem of camera motion and structure reconstruction from line correspondences across multiple views, from initialization to final bundle adjustment. One of the main difficulties when dealing with line features is their algebraic representation. First, we consider the triangulation problem. Based on Plucker coordinates to represent the lines, we propose a maximum likelihood algorithm, relying on linearising the Plucker constraint, and on a Plucker correction procedure to compute the closest Plucker coordinates to a given 6-vector. Second, we consider the bundle adjustment problem. Previous overparameterizations of 3D lines induce gauge freedoms and/or internal consistency constraints. We propose the orthonormal representation, which allows handy nonlinear optimization of 3D lines using the minimum 4 parameters, within an unconstrained nonlinear optimizer. We compare our algorithms to existing ones on simulated and real data.

We present an novel algorithm that reconstructs voxels of a general 3D specular surface from multiple images of a calibrated camera. A calibrated scene (i.e. points whose 3D coordinates are known) is reflected by the unknown specular surface onto the image plane of the camera. For every viewpoint, surface normals are associated to the voxels traversed by each projection ray formed by the reflection of a scene point. A decision process then discards voxels whose associated surface normals are not consistent with one another. The output of the algorithm is a collection of voxels and surface normals in 3D space, whose quality and size depend on user-set thresholds. The method has been tested on synthetic and real images. Visual and quantified experimental results are presented.

We present a new approach to 3D scene modeling based on geometric constraints. Contrary to the existing methods, we can quickly obtain 3D scene models that respect the given constraints exactly. Our system can describe a large variety of linear and nonlinear constraints in a flexible way. To deal with the constraints, we decided to exploit the properties of the GPDOF algorithm developed in the Constraint Programming community (Trombettoni, 1998). The approach is based on a dictionary of so-called r-methods, based on theorems of geometry, which can solve a subset of geometric constraints in a very efficient way. GPDOF is used to find, in polynomial-time, a reduced parameterization of a scene, and to decompose the equation system, induced by constraints, into a sequence of r-methods. We have validated our approach in reconstructing, from images, 3D models of buildings based on linear and quadratic geometric constraints.

Traditionally, building 3D reconstructions of large scenarios such as a museum or historical site has been costly, time consuming and required the contribution of expert personnel. Usually the results showed an artificial look and had little interactivity. However, newly developed technologies in the areas of video analysis, camera calibration and texture fusion allow us to think in a much more satisfying scenario where the user with the only aid of a domestic video camera is able to acquire all the information it is required to construct the 3D model of the desired environment in an easy and comfortable manner. In this paper, the results obtained in the EC funded project VISIRE are presented. VISIRE attempts to construct photorealistic 3D models of large scenarios using as input multiple freehand video sequences. Once acquired, the computer vision software processes the video information off-line in order to obtain the 3D mesh together with the textures required to obtain a 3D model highly resembling the original.

We study the problem of aligning two 3D line reconstructions expressed in Plucker line coordinates. We introduce the 6/spl times/6 3D line motion matrix that acts on Plucker coordinates in projective, affine or Euclidean space. We characterize its algebraic properties and its relation to the usual 4/spl times/4 point motion matrix, and propose various methods for estimating 3D motion from line correspondences, based on image-related and 3D cost functions. We assess the quality of the different estimation methods using simulated data and real images.

We present several methods for the estimation of relative pose between planes and cameras, based on projections of sets of coplanar features in images. While such methods exist for simple cases, especially one plane seen in one or several views, the aim of this paper is to propose solutions for multi-plane multi-view situations, possibly with little overlap. We propose a factorization-based method for the general case of n planes seen in m views. A mechanism for computing missing data, i.e. when one or several of the planes are not visible in one or several of the images, is described. Experimental results for real images are shown.

We examine the potential use of gravity for camera calibration and pose estimation purposes. Concretely, objects being launched or dropped follow trajectories dictated by the law of gravity. We examine if video sequences of such trajectories give us exploitable constraints for estimating the imaging geometry. It is shown that it is possible to estimate the infinite homography and the epipolar geometry between pairs of views from this input, from which we can estimate (some) intrinsic parameters and relative pose. There are less singularities compared to approaches that do not use the information that the observed trajectories follow gravity. In this paper, we sketch the geometric principles of our idea and validate them by numerical simulations.

We introduce the concept of self-calibration of a 1D projective camera from point correspondences, and describe a method for uniquely determining the two internal parameters of a 1D camera, based on the trifocal tensor of three 1D images. The method requires the estimation of the trifocal tensor which can be achieved linearly with no approximation unlike the trifocal tensor of 2D images and solving for the roots of a cubic polynomial in one variable. Interestingly enough, we prove that a 2D camera undergoing planar motion reduces to a 1D camera. From this observation, we deduce a new method for self-calibrating a 2D camera using planar motions. Both the self-calibration method for a 1D camera and its applications for 2D camera calibration are demonstrated on real image sequences.

We consider the self-calibration problem for perspective cameras and especially the classical Kruppa equation approach. It is known that for several common types of camera motion, self-calibration is degenerate, which manifests itself through the existence of ambiguous solutions. The author previously (1997, 1999) studied these critical motion sequences and showed their importance for practical applications. Here, we reveal a type of camera motion that is not critical for the generic self-calibration problem, but for which the Kruppa equation approach fails. This is the case if the optical centers of all cameras lie on a sphere and if the optical axes pass through the sphere's center, a very natural situation for 3D object modeling from images. Results of simulated experiments demonstrate the instability of numerical self-calibration algorithms in near-degenerate configurations.

Scene geometry can be inferred from point correspondences between two images. The inference process includes the selection of a model. Four models are considered: background (or null), collineation, affine fundamental matrix and fundamental matrix. It is shown how Minimum Description Length (MDL) can be used to compare the different models. The main result is that there is little reason for preferring the fundamental matrix model over the collineation model, even when the former the ‘true’ model.

. We consider the self-calibration problem for a moving camera whose intrinsic parameters are known, except the focal length, which may vary freely across different views. The conditions under which the determination of the focal length's values for an image sequence is not possible, are derived. These depend only on the camera's motions. We give a complete catalogue of the so-called critical motion sequences. This is then used to derive the critical motion sequences for stereo systems with variable focal lengths. 1 Introduction One of the major goals of computer vision is the recovery of spatial information about the environment. Classical approaches assume that the cameras are calibrated beforehand, but a great interest in uncalibrated vision and on-line calibration has arisen during the last decade. A key result is that even with completely uncalibrated cameras, spatial information -- projective structure -- can be obtained: the scene can be reconstructed up to an unknown project...

International audience

We present a general algorithm for plane-based calibration that can deal with arbitrary numbers of views and calibration planes. The algorithm can simultaneously calibrate different views from a camera with variable intrinsic parameters and it is easy to incorporate known values of intrinsic parameters. For some minimal cases, we describe all singularities, naming the parameters that can not be estimated. Experimental results of our method are shown that exhibit the singularities while revealing good performance in non-singular conditions. Several applications of plane-based 3D geometry inference are discussed as well.

We consider the self-calibration problem for perspective cameras, and especially the best known practical method, the so-called Kruppa equation approach. It is known that for several common types of camera motion, self-calibration is degenerate, which manifests itself through the existence of ambiguous solutions. In a previous paper, we have studied these critical motion sequences in detail and have revealed their importance for practical applications. In this paper, we reveal a type of camera motion that is not critical for the generic self-calibration problem, but for which the Kruppa equation approach fails. This is the case if all optical centers lie on a sphere and if the optical axes pass through the sphere's center. This situation is very natural for 3D object modeling from photographs or image sequences. Our result is thus a contribution to the understanding of how to successfully apply self-calibration in practice.

In this paper sequences of camera motions that lead to inherent ambiguities in uncalibrated Euclidean reconstruction or self-calibration are studied. Our main contribution is a complete, detailed classification of these critical motion sequences (CMS). The practically important classes are identified and their degrees of ambiguity are derived. We also discuss some practical issues, especially concerning the reduction of the ambiguity of a reconstruction.

We consider the problem of self-calibrating a moving camera which is equipped with a zoom lens. This consists essentially in estimating the 5 intrinsic parameters of the pinhole camera model. However, these are not independent. Thus, we propose to do a pre-calibration of the camera, with the aim to model the interdependence of intrinsic parameters. We show, that self-calibration then comes down to the estimation of only 1 intrinsic parameter. We propose a method which exploits this and which does not need an initialization of the intrinsic parameters. The method consists mainly in finding the closest common root of several low-degree polynomials. Besides the simple computations needed, it is easy to parallelize and thus well suited for future real-time application. The improvement of our method on self-calibration without pre-calibration is shown by experiment. Keywords : Self-calibration, Kruppa's Equations, Zoom 1 Introduction In this paper we address the problem of self-calibration...