Tracking the pose of a camera is at the core of visual localization methods used in many applications. As the observations of a camera are inherently affected by lighting, it has always been a challenge for these methods to cope with varying lighting conditions. Thus far, this issue has mainly been approached with the intent to increase robustness by choosing lighting invariant map representations. In contrast, our work aims at explicitly exploiting lighting effects for camera tracking. To achieve this, we propose a lighting adaptable map representation for indoor environments that allows real-time rendering of the scene illuminated by an arbitrary subset of the lamps contained in the model. Our method for estimating the light setting from the current camera observation enables us to adapt the model according to the lighting conditions present in the scene. As a result, lighting effects like cast shadows do no longer act as disturbances that demand robustness but rather as beneficial features when matching observations against the map. We leverage these capabilities in a direct dense camera tracking approach and demonstrate its performance in realworld experiments in scenes with varying lighting conditions.

We present a method for jointly classifying objects in depth maps and regressing amodal (extending beyond occluded parts) 3D bounding boxes in a way that is highly robust to occlusions. Our method is based on a novel multi-view convolutional neural network architecture with shared layers for both tasks, improving efficiency. The network processes views that encode object geometry and occlusion information and outputs class scores and bounding box coordinates in world coordinates, requiring no post-processing steps. We demonstrate the effectiveness of our method by example of fall detection, presenting a new dataset of 40k samples rendered from 3D models. On this dataset, our method achieves an average classification accuracy above 97% and a regression error below 10 cm at occlusion ratios of up to 90%. The dataset and trained models are publicly available.

The ability to build models of the environment is an essential prerequisite for many robotic applications. In recent years, mapping of dense surface geometry using RGB-D cameras has seen extensive progress. Many approaches build colored models, typically directly using the intensity values provided by the camera. Unfortunately, these intensities are inherently affected by illumination. Therefore, the resulting maps only represent the environment for one specific lighting condition. To overcome this limitation, we propose to build reflectance maps that are invariant against changes in lighting. Our approach estimates the diffuse reflectance of a surface by recovering its radiosity and the corresponding irradiance. As imperfections in this process can significantly degrade the reflectance estimate, we remove outliers in the high dynamic range radiosity estimation and propose a method to refine the reflectance estimate. Our system implements the whole pipeline for offline reconstruction of dense reflectance maps including the segmentation of light emitters in the scene. We demonstrate the applicability of our approach in real-world experiments under varying lighting conditions.

A mobile robot that accomplishes high level tasks needs to be able to classify the objects in the environment and to determine their location. In this paper, we address the problem of online object detection in 3D laser range data. The object classes are represented by 3D point-clouds that can be obtained from a set of range scans. Our method relies on the extraction of point features from range images that are computed from the point-clouds. Compared to techniques that directly operate on a full 3D representation of the environment, our approach requires less computation time while retaining the robustness of full 3D matching. Experiments demonstrate that the proposed approach is even able to deal with partially occluded scenes and to fulfill the runtime requirements of online applications.