Abstract Cameras are the prevalent sensors used for perception in autonomous robotic systems, but their initial calibration may degrade over time due to dynamic factors. This may lead to a failure of downstream tasks, such as simultaneous localization and mapping (SLAM) or object recognition. Hence, a computationally lightweight process that detects the decalibration is of interest. We describe a modification of StOCaMo, an online calibration monitoring procedure for a stereoscopic system. The method uses robust kernel correlation based on epipolar constraints; it validates extrinsic calibration parameters on a single frame with no temporal tracking. In this paper, we present a modified StOCaMo with an improved recall rate on small decalibrations through a confirmation technique based on resampled variance. With fixed parameters learned on a realistic synthetic dataset from CARLA, StOCaMo and its proposed modification were tested on multiple sequences from two real-world datasets: KITTI and EuRoC MAV. The modification improved the recall of StOCaMo by 25 % (to 91 % and 82 %, respectively), and the accuracy by 12 % (to 94.7 % and 87.5 %, respectively), while labeling at most one-third of the input data as uninformative. The upgraded method achieved the rank correlation between StOCaMo V-index and downstream SLAM error of 0.78 (Spearman).

The relative poses of visual perception sensors distributed over a vehicle's body may vary due to dynamic forces, thermal dilations, or minor accidents. This article proposes two methods, Online CAlibration MOnitoring (OCAMO) and LTO, that monitor and track the LiDAR–camera extrinsic calibration parameters online. Calibration monitoring provides a certificate for reference-calibration parameters validity. Tracking follows the calibration parameters drift in time. OCAMO is based on an adaptive online stochastic optimization with a memory of past evolution. LTO uses a fixed-grid search for the optimal parameters per frame and without memory. Both methods use low-level point-like features, a robust kernel-based loss function, and work with a small memory footprint and computational overhead. Both include a preselection of informative data, which limits their divergence. The statistical accuracy of both calibration monitoring methods is over 98%, whereas OCAMO monitoring can detect small decalibrations better, and LTO monitoring reacts faster on abrupt decalibrations. The tracking variants of both methods follow random calibration drift with an accuracy of about$\mathbf {0.03^\circ }$in the yaw angle.

Certain classification tasks in computer vision require the classifier response to be computed in every pixel of an image. When combined with large, complex features, it becomes challenging to build such a classifier on a standard PC architecture and achieve real-time performance. We present an FPGA implementation of a car wheel classifier response computation, built as an instantiation of a generic classification system. An interesting optimization problem concerning performance and speed is addressed. Our implementation is running in real-time as a part of a more complex collision mitigation system based on car detection in video data.

Current collision mitigation systems focus on rear end collisions. To address the full spectrum of real world accidents, these systems will have to be enhanced to cover more traffic situations. Vehicle to vehicle accidents in crossing traffic situations make up around 25% of accidents in Germany. This paper discusses the requirements and differences compared to rear-end collisions. Presented here is an action concept that takes into account how the impact configuration is changed by breaking the host (impacting) vehicle. Based on this concept the requirements for the detection of crossing traffic were derived. These requirements were met by developing a video system based on a monocular wide field of view camera. It is further shown how this action concept and sensor were integrated into a demonstrator vehicle and evaluated in full scale testing.

A simple stereo matching algorithm is proposed that visits only a small fraction of disparity space in order to find a semi-dense disparity map. It works by growing from a small set of correspondence seeds. Unlike in known seed-growing algorithms, it guarantees matching accuracy and correctness, even in the presence of repetitive patterns. This success is based on the fact it solves a global optimization task. The algorithm can recover from wrong initial seeds to the extent they can even be random. The quality of correspondence seeds influences computing time, not the quality of the final disparity map. We show that the proposed algorithm achieves similar results as an exhaustive disparity space search but it is two orders of magnitude faster. This is very unlike the existing growing algorithms which are fast but erroneous. Accurate matching on 2-megapixel images of complex scenes is routinely obtained in a few seconds on a common PC from a small number of seeds, without limiting the disparity search range.

In stereo literature, there is no standard method for evaluating algorithms for semi-dense stereo matching. Moreover, existing evaluations for dense methods require a fixed parameter setting for the tested algorithms. In this paper, we propose a method that overcomes these drawbacks and still is able to compare algorithms based on a simple numerical value, so that reporting results does not take up much space in a paper. We propose evaluation of stereo algorithms based on Receiver Operating Characteristics (ROC) which captures both errors and sparsity. By comparing ROC curves of all tested algorithms we obtain the Feasibility Boundary, the best possible performance achieved by a set of tested stereo algorithms, which allows stereo algorithm users to select the proper method and parameter setting for a required application.

We pose robust matching with parametric and non-parametric constraints as the problem of finding a stable independent set (SIS) in an oriented graph whose vertices are all possible correspondences, whose edges capture the structure of the constraints and whose edge orientation represents pairwise comparison 'is better' based on correspondence quality, including the uncertainty of this comparison. We show SIS possess properties of both robustness and weak optimality. The main contribution of this paper is algorithmic speedup that results from exploiting the dependence between the standard uniqueness constraint and the parametric constraint. The general theory is demonstrated on the example of image stitching using homography model. The algorithm needs at most 𝜅N2calls of a procedure testing if two ellipse correspondences are consistent with a general homography. The previous known SIS algorithm needed O(N4) tests. Experiments show the method gives good results and is fast in practice with 𝜅≈0.3.

Abstract Computer Vision is a discipline whose ultimate goal is to interpret optical images of real scenes. It is well understood that such a problem is cursed by ambiguity of interpretation and uncertainty of evidence. Despite imperfectness of results due to the scenes never following our prior models exactly, Computer Vision has achieved a significant progress in the past two decades. This talk will outline the quest of 3D Computer Vision by describing a processing pipeline that receives a heap of unorganized images from unknown cameras and produces a consistent 3D geometric model together with camera calibrations. We will see how new algorithms allow the standard conception of the pipeline as a series of independent processing steps gradually transform to a single complex, yet efficient vision task. We will identify some points where linking Computer Vision and Computer Graphics would bring significant progress.

In this paper we propose rectification procedure for binocular stereoscopic vision that minimizes the loss of local image neighbourhood discriminability in rectified images. The optimality of the rectification is thus influenced by image contents. Such rectification helps seek for precise dense correspondences.

Local joint image modeling in stereo matching brings more discriminable and stable matching features. Such features reduce the need for strong prior models (continuity) and thus algorithms that are less prone to false positive artefacts in general complex scenes can be applied. One of the principal quality factors in area-based dense stereo is the matching window shape. As it cannot be selected without having any initial matching hypothesis we propose a stratified matching approach. The window adapts to high-correlation structures in disparity space found in pre-matching which is then followed by final matching. In a rigorous ground-truth experiment we show that Stratified Dense Matching is able to increase matching density 3×, matching accuracy 1.8×, and occlusion boundary detection 2 × as compared to a fixed-size rectangular windows algorithm. Performance on real outdoor complex scenes is also evaluated. 1

In this paper we propose and study a simple trinocular rectification method in which stratification to projective and affine components gives the rectifying homographies in a closed form. The class of trinocular rectifications which has 6 DOF is parametrized by an independent set of parameters with a geometric meaning. This offers the possibility to minimize rectification distortion in a natural way. It is shown experimentally on real data that our algorithm performs the rectification task correctly. As shown on groundtruth data using Confidently Stable Matching, trinocular matching significantly improves disparity map density and mismatch error, both depending on texture strength. Matching results on real complex scenes are reported. 1

We address the problem of automatically reconstructing m-manifolds of unknown topology from unorganized points in metric p-spaces obtained from a noisy measurement process . The point set is first approximated by a collection of oriented primitive fuzzy sets over a range of resolutions. Hierarchical multiresolution representation is then computed based on the relation of relative containment defined on the collection. Finally, manifold structure is recovered by establishing connectivity between these primitives based on proximity, compatibility of position and orientation and local topological constraints. The method has been successfully applied to the problem of surface reconstruction from polynocular-stereo data with many outliers.

In this paper we address an application of computer vision which can in the future change completely our way of communicating over the network. We present our version of a testbed for telecollaboration. It is based on a highly accurate and precise stereo algorithm. The results demonstrate the live (on-line) recovery of 3D models of a dynamically changing environment and the simultaneous display and manipulation of the models.

We study occluding contour artifacts in area-based stereo matching: they are false responses of the matching operator to the occlusion boundary and cause the objects extend beyond their true boundaries in disparity maps. Most of the matching methods suffer from these artifacts; the effect is so strong that it cannot be ignored. We show what gives rise to the artifacts and design a matching criterion that accommodates the presence of occlusions as opposed to methods that identify and remove the artifacts. This approach leads to the problem of measurement contamination studied in statistics. We show that such a problem is hard given finite computational resources, unless more independent measurements directly related to occluding contours is available. What can be achieved is a substantial reduction of he artifacts, especially for large matching templates. Reduced artifacts allow for easier hierarchical matching and for easy fusion of reconstructions from different viewpoints into a coherent whole.