In this paper, we present a novel 3D scene reconstruction framework from a single front-mounted stereo camera on a moving vehicle. We propose image triangulations to efficiently render a 3D scene only from 2D textures, while introducing tube meshes as an effective way to render out-of-frustum points. Furthermore, we derive a 3D extended Kalman filter to fuse stereo estimates temporally between frames and showcase a render pipeline, which exploits OpenGL shaders to offload computational costs from the CPU to the GPU. Our approach is able to increase the stereo accuracy compared to competing approaches on the KITTI visual odometry dataset. We also introduce a challenging view prediction evaluation scenario on the SYNTHIA dataset, in which our approach comes out on top in terms of SSIM, 1-NCC error and completeness.

Visual odometry in sparsely textured environments is still a difficult task. Point feature based approaches are well researched, but they need enough texture to yield good results. Recently, line features moved into focus as they only need structure and no texture. Indoor scenes are typically rich in structure. This is why line features are very well suited for this task. A line represents many points and, thus, provides a higher accuracy than a single point. We present a line-only approach that runs stable in textureless environments while achieving real-time performance. Our approach can compete well with existing point- and line-based methods and outperforms existing line-only methods. We developed a tracking method for line-segments that works well without descriptors. Additionally, we propose a heuristic filter that improves the tracking and arbitrary matching results with lines. Furthermore, an optimization using the Cayley representation for 3D lines is used, reducing the line parameters from usually six to four. We demonstrate our performance on the EuRoC benchmark and on a synthetic scene which we built to test the algorithms under particularly difficult conditions.

In this paper, we present an appearance based line descriptor which was developed with the help of machine learning. Our descriptor uses a ResNet which we modified in its size to improve the performance. We utilized the Unreal Engine and multiple scenes provided for it to create training data. The training was performed using a triplet loss, where the loss of the network is calculated with triplets each consisting of three lines including a matching pair and another non-matching line. During learning, the goal of the minimization function is to calculate descriptors with minimal descriptor distance to matching lines' descriptors and maximal descriptor distance to other lines' descriptors. We evaluate the performance of our descriptor on our synthetic datasets, on real-world stereo images from the Middlebury Stereo Dataset and on a benchmark for line segment matching. The results show that in comparison to state-of-the-art line descriptors our method achieves a greatly improved line matching accuracy.

In this paper, we propose a novel method for the 3D reconstruction of urban scenes using a front-facing stereo camera in a vehicle. Our point-based approach warps an active region of the reconstructed point cloud to the next frame and uses an extended information filter for the temporal fusion of clustered disparity estimates in pixel bins. We splat the information of projected pixels according to subpixel weights and discard uncertain points. This method allows us to remove redundant points required for the reconstruction and at the same time presents a significantly denser model than competing approaches with improved disparity estimates. Our approach avoids common visual artifacts like spurious objects in the reconstruction. This results in a reconstruction with higher visual fidelity compared to other approaches, which is important for immersive applications. We compare our proposed system to other approaches in a quantitative and qualitative evaluation on the KITTI odometry dataset.

We present LS-ELAS, a line segment extension to the ELAS algorithm, which increases the performance and robustness. LS-ELAS is a binocular dense stereo matching algorithm, which computes the disparities in constant time for most of the pixels in the image and in linear time for a small subset of the pixels (support points). Our approach is based on line segments to determine the support points instead of uniformly selecting them over the image range. This way we find very informative support points which preserve the depth discontinuity. The prior of our Bayesian stereo matching method is based on a set of line segments and a set of support points. Both sets are given to a constrained Delaunay triangulation to generate a triangulation mesh which is aware of possible depth discontinuities. We further increased the accuracy by using an adaptive method to sample candidate points along edge segments. We evaluated our algorithm on the Middlebury benchmark.

High dynamic range imaging is currently being introduced to television, cinema and computer games. While it has been found that a fixed encoding for high dynamic range imagery needs at least 11 to 12 bits of tonal resolution, current mainstream image transmission interfaces, codecs and file formats are limited to 10 bits. To be able to use current generation imaging pipelines, this paper presents a baseband quantization scheme that exploits content characteristics to reduce the needed tonal resolution per image. The method is of low computational complexity and provides robust performance on a wide range of content types in different viewing environments and applications.

Personalized soft-keyboards which adapt to a user's individual typing behavior can reduce typing errors on interactive displays. In multi-user scenarios a personalized model has to be loaded for each participant. In this paper we describe a user identification technique that is based on raw sensor data from an optical touch screen. For classification of users we use a multi-class support vector machine that is trained with grayscale images from the optical sensor. Our implementation can identify a specific user from a set of 12 users with an average accuracy of 97.51% after one keystroke. It can be used to automatically select individual typing models during free-text entry. The resulting authentication process is completely implicit. We furthermore describe how the approach can be extended to automatic loading of personal information and settings.

We present a framework that allows for localization based on very low resolution omnidirectional image data using regression techniques. Previous related methods are constrained to image data labeled with exact position information acquired in the training phase. We relax this constraint and propose to learn local heteroscedastic Gaussian processes by accumulating odometry data which can easily be acquired. The processes are used as a probabilistic map to predict recording positions of newly acquired images by a fusion of the uncertain training data. In contrast to many feature-based approaches, our framework does not rely on any explicit correspondences over images as well as over positions and only imposes very weak assumptions on the type and quality of the image representations.

Determining the physical location is a fundamental challenge in location based services and service robotics. This paper presents an approach to simultaneously determine the pose of a mobile robot and the positions of static wireless nodes based on a new technique to compute the angle of arrival of radio signals. For the overall localization process an Extended Kalman Filter is employed. For the initialization of the nodes positions a Particle Filter is used, to overcome the nonlinearity problem. Experiments show a mean accuracy for the position of the robot of 39mm for an indoor office environment and 80mm for an outdoor landscape environment.

We present a probabilistic localization and orientation estimation method for mobile agents equipped with omnidirectional vision. In our appearance-based framework, a scene is learned in an offline step by modeling the variation of the image energy in the frequency domain via Gaussian process regression. The metric localization of novel views is then solved by maximizing the joint predictive probability of the Gaussian processes using a particle filter which allows to incorporate a motion model in the prediction step. Based on the position estimate, a synthetic view is generated and used as a reference for the orientation estimation which is also performed in the Fourier space. Using real as well as virtual data, we show that this framework allows for robust localization in 2D and 3D scenes based on very low resolution images and with competitive computational load.

This paper introduces a new shape matching algorithm for computing correspondences between 3D surfaces that have undergone (approximately) isometric deformations. The new approach makes two main contributions: First, the algorithm is, unlike previous work, robust to “topological noise” such as large holes or “false connections”, which is both observed frequently in real-world scanner data. Second, our algorithm samples the space of feasible solutions such that uncertainty in matching can be detected explicitly. We employ a novel randomized feature matching algorithm in order to find robust subsets of geodesics to verify isometric consistency. The paper shows shape matching results for real world and synthetic data sets that could not be handled using previous deformable matching algorithms.

Abstract In this paper, we describe a new algorithm for detecting structural redundancy in geometric data sets. Our algorithm computes rigid symmetries, i.e., subsets of a surface model that reoccur several times within the model differing only by translation, rotation or mirroring. Our algorithm is based on matching locally coherent constellations of feature lines on the object surfaces. In comparison to previous work, the new algorithm is able to detect a large number of symmetric parts without restrictions to regular patterns or nested hierarchies. In addition, working on relevant features only leads to a strong reduction in memory and processing costs such that very large data sets can be handled. We apply the algorithm to a number of real world 3D scanner data sets, demonstrating high recognition rates for general patterns of symmetry.

We present a new technique for reconstructing a single shape and its nonrigid motion from 3D scanning data. Our algorithm takes a set of time-varying unstructured sample points that capture partial views of a deforming object as input and reconstructs a single shape and a deformation field that fit the data. This representation yields dense correspondences for the whole sequence, as well as a completed 3D shape in every frame. In addition, the algorithm automatically removes spatial and temporal noise artifacts and outliers from the raw input data. Unlike previous methods, the algorithm does not require any shape template but computes a fitting shape automatically from the input data. Our reconstruction framework is based upon a novel topology-aware adaptive subspace deformation technique that allows handling long sequences with complex geometry efficiently. The algorithm accesses data in multiple sequential passes, so that long sequences can be streamed from hard disk, not being limited by main memory. We apply the technique to several benchmark datasets, significantly increasing the complexity of the data that can be handled efficiently in comparison to previous work.

In this paper, we describe a system for the reconstruction of deforming geometry from a time sequence of unstructured, noisy point clouds, as produced by recent real-time range scanning devices. Our technique reconstructs both the geometry and dense correspondences over time. Using the correspondences, holes due to occlusion are filled in from other frames. Our reconstruction technique is based on a statistical framework: The reconstruction should both match the measured data points and maximize prior probability densities that prefer smoothness, rigid deformation and smooth movements over time. The optimization procedure consists of an inner loop that optimizes the 4D shape using continuous numerical optimization and an outer loop that infers the discrete 4D topology of the data set using an iterative model assembly algorithm. We apply the technique to a variety of data sets, demonstrating that the new approach is capable of robustly retrieving animated models with correspondences from data sets suffering from significant noise, outliers and acquisition holes.

Abstract In this paper, we propose a novel surface reconstruction technique based on Bayesian statistics: The measurement process as well as prior assumptions on the measured objects are modeled as probability distributions and Bayes’ rule is used to infer a reconstruction of maximum probability. The key idea of this paper is to define both measurements and reconstructions as point clouds and describe all statistical assumptions in terms of this finite dimensional representation. This yields a discretization of the problem that can be solved using numerical optimization techniques. The resulting algorithm reconstructs both topology and geometry in form of a well‐sampled point cloud with noise removed. In a final step, this representation is then converted into a triangle mesh. The proposed approach is conceptually simple and easy to extend. We apply the approach to reconstruct piecewise‐smooth surfaces with sharp features and examine the performance of the algorithm on different synthetic and real‐world data sets. Categories and Subject Descriptors (according to ACM CCS): I.5.1 [Models]: Statistical; I.3.5 [Computer Graphics]: Curve, surface, solid and object representations

We present a method to acquire a realistic, visually convincing 3D model of indoor office environments based on a mobile robot that is equipped with a laser range scanner and a panoramic camera. The data of the 2D laser scans are used to solve the SLAM problem and to extract walls. Textures for walls and floor are built from the images of a calibrated panoramic camera. Multiresolution blending is used to hide seams in the generated textures.

Environment maps, like texture maps or any other maps consisting of discretely stored data have to be properly filtered, if they are being resampled in the process of rendering an image. For environment maps, this is especially important, as the sampling rate is subject to extreme changes due to the curvature of the reflecting surfaces. However, for the same reason, the antialiasing is especially difficult to perform, as the sampling rate has to be determined for each pixel. We introduce a method to perform this calculation and determine the parameters for anisotropic filtering of the environment map. Instead of the BRDF, we employ roughness pyramids to account for the properties of the reflecting surface. The principles that are used to perform this antialiasing can be applied for antialiasing reflected textures in general e.g. in ray tracing.

In this paper we consider the problem of reconstructing triangular surfaces from given contours. An algorithm solving this problem has to decide which contours of two successive slices should be connected by the surface (branching problem), and, given that, which vertices of the assigned contours should be connected for the triangular mesh (correspondence problem). We present a new approach that solves both tasks in an elegant way. The main idea is to employ discrete distance fields enhanced with correspondence information. This allows us not only to connect vertices from successive slices in a reasonable way but also to solve the branching problem by creating intermediate contours where adjacent contours differ too much. Last but not least we show how the 2D-distance fields used in the reconstruction step can be converted to a 3D-distance field that can be advantageously exploited for distance calculations during a subsequent simplification step.

State of the art multiresolution modeling allows to selectively refine a coarse mesh of an object on the visually important parts. In this way it is possible to render the geometry of a given object accurately with a minimum number of triangles. There are also approaches that use cones of normal vectors to perform illumination dependent selective refinement. This includes the correct rendering of highlights and transitions between lit and unlit areas of an object. The drawback of these approaches is that using only the normal cone for the detection gives only a very poor criteria to detect the problematic cases. Therefore, too much refinement has to be performed which in most cases prevents real time update rates between successive frames. The authors describe a new approach for better estimation of normal deviations between different levels of detail. This allows for accurate lighting with a minimum number of triangles.

The EXACT (EXact Area Coverage calculaTion) algorithm presented in this paper solves the Hidden Surface Elimination (HSE) problem on the subpixel level.The use of subpixel masks for anti-aliasing causes some problems with the HSE on the pixel level that are difficult to overcome.The approximations of the well known A-buffer algorithm are replaced by an exact solution that avoids erratic pixels along intersecting or touching surfaces.With EXACT the HSE problem on the subpixel level is solved with the help of p-masks.P-masks (priority masks) are subpixel masks that indicate for each subpixel which one of two given planes is closer to the viewer.An algorithm to produce the p-masks in an efficient way and its hardware implementation are presented.The p-mask generator is used in a hardware implementation of an A-buffer algorithm in the form of a rendering pipeline.Of course the algorithm can also be used in software to enhance an existing A-buffer implementation.The paper ends with the description of the list processing architecture for which the EXACT A-buffer has been built 1 CR Categories and Subject Descriptors: I.3.

Antialiasing of edges is often performed with the help of subpixel masks that indicate which parts of the pixel are covered by the object that has to be drawn. For this purpose, subpixel masks have to be generated during the scan conversion of an image. This paper introduces a new algorithm for creating subpixel masks that avoids some problems of traditional algorithms, like aliasing of high frequencies or blinking of small moving objects. The new algorithm can be implemented by lookup tables that make use of the inherent symmetry of the algorithm. The results are compared with conventional supersampling . A hardware implementation is described.