We propose a new algorithm for finding an unknown number of geometric models, e.g., homographies. The problem is formalized as finding dominant model instances progressively without forming crisp point-to-model assignments. Dominant instances are found via a RANSAC-like sampling and a consolidation process driven by a model quality function considering previously proposed instances. New ones are found by clustering in the consensus space. This new formulation leads to a simple iterative algorithm with state-of-the-art accuracy while running in real-time on a number of vision problems - at least two orders of magnitude faster than the competitors on two-view motion estimation. Also, we propose a deterministic sampler reflecting the fact that real-world data tend to form spatially coherent structures. The sampler returns connected components in a progressively densified neighborhood-graph. We present a number of applications where the use of multiple geometric models improves accuracy. These include pose estimation from multiple generalized homographies; trajectory estimation of fast-moving objects; and we also propose a way of using multiple homographies in global SfM algorithms. Source code: https://github.com/danini/clustering-in-consensus-space.

We present a new solver for estimating a surface normal from a single affine correspondence in two calibrated views. The proposed approach provides a new globally optimal solution for this over-determined problem and proves that it reduces to a linear system that can be solved extremely efficiently. This allows for performing significantly faster than other recent methods, solving the same problem and obtaining the same globally optimal solution. We demonstrate on 15k image pairs from standard benchmarks that the proposed approach leads to the same results as other optimal algorithms while being, on average, five times faster than the fastest alternative. Besides its theoretical value, we demonstrate that such an approach has clear benefits, e.g., in image-based visual localization, due to not requiring a dense point cloud to recover the surface normal. We show on the Cambridge Landmarks dataset that leveraging the proposed surface normal estimation further improves localization accuracy. Matlab and C++ implementations are also published in the supplementary material.

Abstract We propose a novel minimal solver for sphere fitting via its 2D central projection, i.e., a special ellipse. The input of the presented algorithm consists of contour points detected in a camera image. General ellipse fitting problems require five contour points. However, taking advantage of the isotropic spherical target, three points are enough to define the tangent cone parameters of the sphere. This yields the sought ellipse parameters. Similarly, the sphere center can be estimated from the cone if the radius is known. These proposed geometric methods are rapid, numerically stable, and easy to implement. Experimental results—on synthetic, photorealistic, and real images—showcase the superiority of the proposed solutions to the state-of-the-art methods. A real-world LiDAR-camera calibration application justifies the utility of the sphere-based approach resulting in an error below a few centimeters.

We propose novel solvers for estimating the egomotion of a calibrated camera mounted to a moving vehicle from a single affine correspondence via recovering special homographies. For the first, second and third classes of solvers, the sought plane is expected to be perpendicular to one of the camera axes. For the fourth class, the plane is orthogonal to the ground with unknown normal, e.g., it is a building facade. All methods are solved via a linear system with a small coefficient matrix, thus, being extremely efficient. Both the minimal and over-determined cases can be solved by the proposed solvers. They are tested on synthetic data and on publicly available real-world datasets. The novel methods are more accurate or comparable to the traditional algorithms and are faster when included in state-of-the-art robust estimators. The source code is publicly available[1].

A new closed-form solver is proposed minimizing the algebraic error optimally, in the least squares sense, to estimate the relative planar motion of two calibrated cameras. The main objective is to solve the over-determined case, i.e., when a larger-than-minimal sample of point correspondences is given - thus, estimating the motion from at least three correspondences. The algorithm requires the camera movement to be constrained to a plane, e.g. mounted to a vehicle, and the image plane to be orthogonal to the ground.1The solver obtains the motion parameters as the roots of a 6th degree polynomial. It is validated both in synthetic experiments and on publicly available real-world datasets that using the proposed solver leads to results superior to the state-of-the-art in terms of geometric accuracy with no noticeable deterioration in the processing time.

Two solvers are proposed for estimating the extrinsic camera parameters from a single affine correspondence assuming general planar motion. In this case, the camera movement is constrained to a plane and the image plane is orthogonal to the ground. The algorithms do not assume other constraints, e.g. the non-holonomic one, to hold. A new minimal solver is proposed for the semi-calibrated case, i.e. the camera parameters are known except a common focal length. Another method is proposed for the fully calibrated case. Due to requiring a single correspondence, robust estimation, e.g. histogram voting, leads to a fast and accurate procedure. The proposed methods are tested in our synthetic environment and on publicly available real datasets consisting of videos through tens of kilometers. They are superior to the state-of-the-art both in terms of accuracy and processing time.

This paper investigates a novel calibration process of devices with different modalities, which is a critical step of computer vision applications. We propose a fully automatic extrinsic calibration of a LiDAR-camera system. Our approach applies sphere as their surfaces and contours can be accurately detected on point clouds and camera images, respectively. Experiments on synthetic and real data exhibit that our automatic algorithm is fast and robust and it yields accurate camera and LiDAR extrinsic parameters.

An optimal, in the least squares sense, method is proposed to estimate surface normals in both stereo and multi-view cases. The proposed algorithm exploits exclusively photometric information via affine correspondences and estimates the normal for each correspondence independently. The normal is obtained as a root of a quartic polynomial. Therefore, the processing time is negligible. Eliminating the outliers, we propose a robust extension of the algorithm that combines maximum likelihood estimation and iteratively re-weighted least squares. The method has been validated on both synthetic and publicly available real-world datasets. It is superior to the state of the art in terms of accuracy and processing time. Besides, we demonstrate two possible applications: 1) using our algorithm as the seed-point generation step of patch-based multi-view stereo method, the obtained reconstruction is more accurate, and the error of the 3D points is reduced by 30% on average and 2) multi-plane fitting becomes more accurate applied to the resulting oriented point cloud.

We propose a method to estimate the essential matrix using two affine correspondences for a pair of calibrated perspective cameras. Two novel, linear constraints are derived between the essential matrix and a local affine transformation. The proposed method is also applicable to the over-determined case. We extend the normalization technique of Hartley to local affinities and show how the intrinsic camera matrices modify them. Even though perspective cameras are assumed, the constraints can straightforwardly be generalized to arbitrary camera models since they describe the relationship between local affinities and epipolar lines (or curves). Benefiting from the low number of exploited points, it can be used in robust estimators, e.g. RANSAC, as an engine, thus leading to significantly less iterations than the traditional point-based methods. The algorithm is validated both on synthetic and publicly available data sets and compared with the state-of-the-art. Its applicability is demonstrated on two-view multi-motion fitting, i.e., finding multiple fundamental matrices simultaneously, and outlier rejection.

A minimal solution using two affine correspondences is presented to estimate the common focal length and the fundamental matrix between two semi-calibrated cameras - known intrinsic parameters except a common focal length. To the best of our knowledge, this problem is unsolved. The proposed approach extends point correspondence-based techniques with linear constraints derived from local affine transformations. The obtained multivariate polynomial system is efficiently solved by the hidden-variable technique. Observing the geometry of local affinities, we introduce novel conditions eliminating invalid roots. To select the best one out of the remaining candidates, a root selection technique is proposed outperforming the recent ones especially in case of high-level noise. The proposed 2-point algorithm is validated on both synthetic data and 104 publicly available real image pairs. A Matlab implementation of the proposed solution is included in the paper.

An intuitive approach is proposed for outlier recognition among 2D point correspondences. The main novelty of the proposed method is the exploitation of feature point topology provided by Delaunay triangulation. The solution obtained by minimizing an energy originated from neighboring correspondences in order to remove incorrectly paired points. Assuming local, approximately rigid structures, it is able cope with non-rigid scenes. However, if the epipolar geometry is estimable, the additional information is exploited as well. The proposed method - called Delaunay Filtering - is validated on the publicly available AdelaideRMF dataset and outperforms the state-of-the-art, robust model-regression techniques. It is presented that it can be applied to image pairs for which epipolar geometry-based solutions fail.

A new camera calibration approach is proposed that can utilize the affine transformations and surface normals of small spatial patches. Even though classical calibration algorithms use only point locations, images contain more information than simple 2D point coordinates. New methods are presented in this paper for the calibration problem with their closed-form solutions, then the estimated parameters are numerically refined. The accuracy of our novel methods is validated on synthesized testing data, and the real-world applicability is presented on the calibration of a 3D structured-light scanner.

Although modern reconstruction methods have advanced considerably, their ability to retrieve fine details, small features of a surface is limited. Applying texture can be used to conceal lack of geometry to some extent, but it becomes apparent when lighting varies. A more sophisticated solution is to map normal vectors to the surface of the model in order to alter the way it is rendered under different illuminations in accordance with the properties of the real object. The principle of photometric stereo is to reconstruct normals from a series of images. By illuminating the object with different directional light sources local surface orientation can be inferred. The contribution of this study is that the authors successfully applied the widely used bundle adjustment algorithm to photometric stereo problems when non‐parallel light rays illuminate the objects.

Motion-based 3D reconstruction (SfM) with missing data has been a challenging computer vision task since the late 90s. Under perspective camera model, one of the most difficult problems is camera auto-calibration which means determining the intrinsic camera parameters without using any known calibration object or assuming special properties of the scene. This paper presents a novel algorithm to perform camera auto-calibration from multiple images and dealing with the missing data problem. The method supposes semi-calibrated cameras (every intrinsic camera parameter except for the focal length is considered to be known) and constant focal length over all the images. The solution requires at least one image pair having at least eight common measured points. Tests verified that the algorithm is numerically stable and produces accurate results both on synthetic and real test sequences.

3D reconstruction with missing data has been a challenging computer vision task since the late 90s. This paper proposes a novel metric reconstruction algorithm dealing with the missing data problem. The algorithm is the adaption of the Fast Alternation method published by us in CAIP2007. We concentrate on metric instead of affine reconstruction because the quality of metric reconstruction is significantly better as it is demonstrated in this study. The solution is an alternation which consists of several substeps. All of these substeps are optimal with respect to the parameters that are being optimized. It is proved that the proposed algorithm converges to a local minimum. The solutions to the optimization subproblems in our approach are given by closed-form formulas, therefore the proposed method is relatively fast. 1