Metric depth estimation from visual sensors is crucial for robots to perceive, navigate, and interact with their environment. Traditional range imaging setups, such as stereo or structured light cameras, face hassles including calibration, occlusions, and hardware demands, with accuracy limited by the baseline between cameras. Single- and multi-view monocular depth offers a more compact alternative, but is constrained by the unobservability of the metric scale. Light field imaging provides a promising solution for estimating metric depth by using a unique lens configuration through a single device. However, its application to single-view dense metric depth is under-addressed mainly due to the technology's high cost, the lack of public benchmarks, and proprietary geometrical models and software. Our work explores the potential of focused plenoptic cameras for dense metric depth. We propose a novel pipeline that predicts metric depth from a single plenoptic camera shot by first generating a sparse metric point cloud using a neural network, which is then used to scale and align a dense relative depth map regressed by a foundation depth model, resulting in a dense metric depth. To validate it, we curated the Light Field & Stereo Image Dataset11Dataset available at https://zenodo.org/records/14224205. (LFS) of real-world light field images with stereo depth labels, filling a current gap in existing resources. Experimental results show that our pipeline produces accurate metric depth predictions, laying a solid groundwork for future research in this field.22Work partially supported by the DLR Impulse Project SaiNSOR.

We present a Multi-Criteria Decision Making (MCDM) framework specifically designed for planetary exploration. Our work is based on PROMETHEE II, which allows operators to add task-specific criteria and conditions. We extended this algorithm to improve its resource usage by reducing the number of candidate exploration goals that have to be evaluated and compared. This is crucial when given a large number of goals, as is typical for outdoor environments. In addition, we identified five different criteria for planetary exploration, including a novel criterion that we call Direction of Interest (DOI), and use a categorization of these for further resource optimizations. We thereby ensure that the CPU usage of our decision making method can meet the limited budget of a space rover. We present simulated and real-world experiments with the Lightweight Rover Unit (LRU) and show a reduction of the processing time for decision making of approx. 70%.

Intuitive human robot interfaces like speech or gesture recognition are essential for gaining acceptance for robots in daily life. However, such interaction requires that the robot detects the human's intention to interact, tracks his position and keeps its sensor systems in an optimal configuration. Audio is a suitable modality for such task as it allows for detecting a speaker in arbitrary positions around the robot. In this paper, we present a novel approach for localization of sound sources by analyzing the frequency spectrum of the received signal and applying a motion model to the estimation process. We use an improved version of the Generalized Singular Value Decomposition (GSVD) based MUltiple SIgnal Classification (MUSIC) algorithm as a direction of arrival (DoA) estimator. Further, we introduce a motion model to enable robust localization in reverberant and echoic environments.We evaluate the system under real conditions in an experimental setup. Our experiments show that our approach outperforms current state-of-the-art algorithm and demonstrate the robustness against the previously mentioned disruptive factors.

A robotic system for search and rescue missions needs to efficiently explore and map new areas. In this paper, we present an integrated exploration strategy with active loop closing, which balances between the exploration speed and map quality. Specifically, it finds a trade-off between moving towards unknown space to gather new information and revisiting previous locations to improve localization accuracy and map quality through loop closures. Our integrated exploration is built upon a submap-based 6D SLAM system. Loop closure constraints originate from pairwise submap matches, which allow the optimization of an underlying SLAM graph. During exploration, we employ the expected information gain as well as the robot's localization uncertainty estimates to weigh exploration and revisiting actions online. We introduce the match effect as the expected impact of a loop closure on global optimization and consider this novel criterion together with the match likelihood and cost when evaluating the utility of revisiting previous locations. To demonstrate our approach, we present simulated and real-world experiments, comparing two variants of our novel method to a frontier-based exploration.

Active scene exploration incorporates object recognition methods for analyzing a scene of partially known objects and exploration approaches for autonomous modeling of unknown parts. In this work, recognition, exploration, and planning methods are extended and combined in a single scene exploration system, enabling advanced techniques such as multi-view recognition from planned view positions and iterative recognition by integration of new objects from a scene. Here, a geometry based approach is used for recognition, i.e. matching objects from a database. Unknown objects are autonomously modeled and added to the recognition database. Next-Best-View planning is performed both for recognition and modeling. Moreover, 3D measurements are merged in a Probabilistic Voxel Space, which is utilized for planning collision free paths, minimal occlusion views, and verifying the poses of the recognized objects against all previous information. Experiments on an industrial robot with attached 3D sensors are shown for scenes with household and industrial objects.

This work is focused on global registration of surface models such as homogeneous triangle meshes and point clouds. The investigated approach utilizes feature descriptors in order to assign correspondences between the data sets and to reduce complexity by considering only characteristic feature points. It is based on the decomposability of rigid motions into a rotation and a translation. The space of rotations is searched with a particle filter and scoring is performed by looking for clusters in the resulting sets of translations. We use features computed from homogeneous triangle meshes and point clouds that require low computation time. A major advantage of the approach proves to be the possible consideration of prior knowledge about the relative orientation. This is especially important when high noise levels produce deteriorated features that are hard to match correctly. Comparisons to existing algorithms show the method's competitiveness, and results in robotic applications with different sensor types are presented.

This paper describes an extension of the sequential scene analysis system presented by Hager and Wegbreit [12]. In contrast to the original system, which was limited to scenes consisting of geometric primitives, such as spheres, cuboids, and cylinders computed from range data, the extended system is capable of dealing with arbitrarily shaped objects computed from range and intensity images. An object model composed of a triangulated geometry and intensity-based SURF features is introduced. The integration of prior object models into the sequential scene parsing framework is described. The extended system is evaluated with respect to pose estimation and its ability to handle complex scene sequences. It is shown that the new object models enable accurate pose estimation and reliable recognition even in highly cluttered scenes.

We present a next-best-scan (NBS) planning approach for autonomous 3D modeling. The system successively completes a 3D model from complex shaped objects by iteratively selecting a NBS based on previously acquired data. For this purpose, new range data is accumulated in-the-loop into a 3D surface (streaming reconstruction) and new continuous scan paths along the estimated surface trend are generated. Further, the space around the object is explored using a probabilistic exploration approach that considers sensor uncertainty. This allows for collision free path planning in order to completely scan unknown objects. For each scan path, the expected information gain is determined and the best path is selected as NBS. The presented NBS approach is tested with a laser striper system, attached to an industrial robot. The results are compared to state-of-the-art next-best-view methods. Our results show promising performance with respect to completeness, quality and scan time.

This video presents the complete procedure for the optimal setup of the DLR MiroSurge telerobotic system for minimally invasive surgery. Two key features are implemented. First, optimization algorithms preoperatively determine several setups that are then rated and selected by the surgeon. Second, the intraoperative situation is taken into account. The newly developed VR-Map device together with fast registration and optimization algorithms enable a quick procedure to assure the optimal patient-specific setup of the robotic system.

The procedure of manually generating a 3D model of an object is very time consuming for a human operator. Next best-view (NBV) planning is an important aspect for automation of this procedure in a robotic environment. We propose a surface-based NBV approach, which creates a triangle surface from a real-time data stream and determines viewpoints similar to human intuition. Thereby, the boundaries in the surface are detected and a quadratic patch for each boundary is estimated. Then several viewpoint candidates are calculated, which look perpendicular to the surface and overlap with previous sensor data. A NBV is selected with the goal to fill areas which are occluded. This approach focuses on the completion of a 3D model of an unknown object. Thereby, the search space for the viewpoints is not restricted to a cylinder or sphere. Our NBV determination proves to be very fast, and is evaluated in an experiment on test objects, applying an industrial robot and a laser range scanner.

In the context of 3-D scene modeling, this work aims at the accurate estimation of the pose of a close-range 3-D modeling device, in real-time and passively from its own images. This novel development makes it possible to abandon using inconvenient, expensive external positioning systems. The approach comprises an ego-motion algorithm tracking natural, distinctive features, concurrently with customary 3-D modeling of the scene. The use of stereo vision, an inertial measurement unit, and robust cost functions for pose estimation further increases performance. Demonstrations and abundant video material validate the approach.

The development of applications for multi-sensor data fusion typically faces heterogeneous hardware components, a variety of sensing principles and limited computational resources. We present a concept for synchronization and communication which tackles these challenges in multi-sensor systems in a unified manner. Here, a combination of hardware synchronization and deterministic software signals is promoted for global synchronization. Patterns of event-driven communication ensure that sensor data processing and evaluation are not bound to runtime constraints induced by data acquisition anymore. The combination of unified range and pose data description, event-driven communication, and global synchronization allows to build 3D sensing applications for various tasks. The proposed concept is implemented and evaluated for a variety of applications based on the DLR multisensory 3D-modeller. Extendability to other range and pose sensors is straightforward.

This paper presents the DLR Laser Stripe Profiler as a component of the DLR multisensory Hand-Guided Device for 3D modeling. After modeling the reconstruction process, we propose a novel method for laser plane self-calibration based on the assessment of the deformations the miscalibration leads to. In addition, the requirement for absence of optical filtering implies the development of a robust stripe segmentation algorithm. Experiments demonstrate the validity and applicability of the approaches.