We present SceneFactory, a workflow-centric and unified framework for incremental scene modeling, that conveniently supports a wide range of applications, such as (unposed and/or uncalibrated) multi-view depth estimation, LiDAR completion, (dense) RGB-D/RGB-L/Mono/Depth-only reconstruction and SLAM. The workflow-centric design uses multiple blocks as the basis for constructing different production lines. The supported applications, i.e., productions avoid redundancy in their designs. Thus, the focus is placed on each block itself for independent expansion. To support all input combinations, our implementation consists of four building blocks that form SceneFactory: (1) tracking, (2) flexion, (3) depth estimation, and (4) scene reconstruction. The tracking block is based on Mono SLAM and is extended to support RGB-D and RGB-LiDAR (RGB-L) inputs. Flexion is used to convert the depth image (untrackable) into a trackable image. For general-purpose depth estimation, we propose an unposed & uncalibrated multi-view depth estimation model (U2 -MVD) to estimate dense geometry. U 2 -MVD exploits dense bundle adjustment to solve for poses, intrinsics, and inverse depth. A semantic-aware ScaleCov step is then introduced to complete the multi-view depth. Relying on U 2 -MVD, SceneFactory both supports user-friendly 3D creation (with just images) and bridges the applications of Dense RGB-D and Dense Mono. For high-quality surface and color reconstruction, we propose Dual-purpose Multi-resolutional Neural Points (DM-NPs) for the first surface accessible Surface Color Field design, where we introduce Improved Point Rasterization (IPR) for point cloud based surface query. We implement and experiment with SceneFactory to demonstrate its broad applicability and high flexibility. Its quality also competes or exceeds the tightly-coupled state of the art approaches in all tasks. We contribute the code to the community

In this paper, we present a local, robot-centric navigation map optimized for autonomous mobile robots operating in unknown environments, enhancing their onboard perception systems for collision-free operation with far look-ahead distances. Utilizing a novel converging covariance cell representation, our approach effectively analyzes hazards such as obstacles and hazardous slopes in both terrestrial and aerial navigation contexts. The new technique specifically targets mapping from stereo scenarios with ultra short baseline and highly oblique viewpoints close to the ground.Our methodology surpasses traditional window-based hazard analysis by resolving sub-cell size obstacles and terrain gradients at the individual cell level, thereby avoiding the computational overhead typically associated with such analyses. It leverages a multi-resolution strategy adaptive to the range errors common in stereo vision systems, making it particularly suitable for embedded systems with computational limitations.Functionality includes constant-time queries for height, obstacle presence, and slope details, boasting improvements in run time, memory usage, precision, and resolvable obstacle size compared to existing grid-based mapping algorithms. We validate our approach through rigorous simulation and real-world testing. This technique will be used for the local mapping and collision avoidance on NASA’s CADRE lunar rovers.

We study the motion model of a sensor rigidly mounted inside a ball. Due to the rigid placement inside the ball, the geometry of the sensor trajectory resembles a 3D curate trochoid. A new calibration method for spherical systems estimates the extrinsic parameters of the sensor with respect to the balls center of rotation. We deploy the calibration and motion model on our spherical mobile mapping platform to estimate the trajectory of a LiDAR sensor and compare it to trajectories of state-of-the-art LiDAR-Inertial odometry (LIO) methods. The motion model, which is solely based on IMU measurements, produces comparable results to the LIO methods, sometimes even outperforming them in positional accuracy. Although the LIO methods provide better rotational accuracy due to the utilization of LiDAR data, they struggle to reproduce the trochoidal nature of the trajectory and only provide pose estimations at the LiDAR frequency, whereas the motion model produces a more consistent trochoidal trajectory at the much higher IMU frequency. The results demonstrate the difficulty that current LIO methods have on spherical systems and indicate that our motion model is suitable for overcoming these issues.

We present Uni-Fusion, a universal continuous mapping framework for surfaces, surface properties (color, infrared, etc.) and more (latent features in contrastive language-image pretraining (CLIP) embedding space, etc.). We propose the first universal implicit encoding model that supports encoding of both geometry and different types of properties (RGB, infrared, features, etc.) without requiring any training. Based on this, our framework divides the point cloud into regular grid voxels and generates a latent feature in each voxel to form a latent implicit map (LIM) for geometries and arbitrary properties. Then, by fusing a local LIM frame-wisely into a global LIM, an incremental reconstruction is achieved. Encoded with corresponding types of data, our LIM is capable of generating continuous surfaces, surface property fields, surface feature fields, and all other possible options. To demonstrate the capabilities of our model, we implement three applications: incremental reconstruction for surfaces and color, 2-D-to-3-D transfer of fabricated properties, and open-vocabulary scene understanding by creating a text CLIP feature field on surfaces. We evaluate Uni-Fusion by comparing it in corresponding applications, from which Uni-Fusion shows high-flexibility in various applications while performing best or being competitive.

The field of human-robot interaction has been rapidly advancing in recent years, as robots are increasingly being integrated into various aspects of human life. However, for robots to effectively collaborate with humans, it is crucial that they have a deep understanding of the environment in which they operate. In particular, the ability to predict traversability and detect tactile information is crucial for enhancing the safety and efficiency of human-robot interactions. To address this challenge, this paper proposes a method called “Feel the Point Clouds” that use point clouds to predict traversability and detect tactile terrain information for a tracked rescue robot. This information can be used to adjust the robot’s behavior and movements in real-time, allowing it to interact with the environment in a more intuitive and safe manner. The experimental results of the proposed method are evaluated in various scenarios and demonstrate its effectiveness in improving human-robot interaction and visualization for a more accurate and intuitive understanding of the environment.

Space robotics applications, such as Active Space Debris Removal (ASDR), require representative testing before launch. A commonly used approach to emulate the microgravity environment in space is air-bearing based platforms on flat-floors, such as the European Space Agency's Orbital Robotics and GNC Lab (ORGL). This work proposes a control architecture for a floating platform at the ORGL, equipped with eight solenoid-valve-based thrusters and one reaction wheel. The control architecture consists of two main components: a trajectory planner that finds optimal trajectories connecting two states and a trajectory follower that follows any physically feasible trajectory. The controller is first evaluated within an introduced simulation, achieving a 100% success rate at finding and following trajectories to the origin within a Monte-Carlo test. Individual trajectories are also successfully followed by the physical system. In this work, we showcase the ability of the controller to reject disturbances and follow a straight-line trajectory within tens of centimeters.

A spherical robotic probe has several advantages in rough environments and has therefore raised interest for application in planetary exploration. A sphere is well-suited to protect high-sensitive payloads, however, the locomotion system for planetary surfaces raises several challenges. This paper presents a novel locomotion system consisting of linear actuators which are usable in a multi-functional fashion. Apart from pushing and bringing leverage for locomotion the extendable rods enable a tripod mode for improved sensing. The developed solutions offer a mathematical-physical system description, simple algorithms for the control of locomotion and balancing as well as general calculations for determining the maximum achievable performance parameters of such a robot. The first built prototype shows the basic suitability of the system and reveals directions for further research.

In this paper, we present a user-selectable control unit to make complex robot systems easier to operate. Search and rescue robots are primarily used for obtaining information and manipulating dangerous objects or supporting emergency forces in dealing with crisis situations. Based on hazardous mission scenarios in which mobile robot systems are confronted with complex manipulation and manoeuvring tasks (e.g., leakage of contaminants), new methods and concepts in the area of robot decision making, sensor data presentation and control concepts were developed to facilitate the handling and operation of assistance robots. The flexibility of the control concept approach is intended to increase the confidence of the emergency services and to provide intuitive operation of the assistance robots for every user. An intuitive control and stable communication for both control commands and feedback from the robot itself increase trust in the robotic system and its acceptance by the operators. These developments have been investigated in field trials with different types of robots and with network communication constraints.

This work in progress paper presents an automated approach for network coverage prediction in real-world environments by combining mobile mapping, 3D mesh generation, and a ray launching based network simulator. We identify the challenges and demonstrate the functionality of such a pipeline. We preview an empirical evaluation in a realistic real-world environment.

In industrial applications continuous wireless connectivity of mobile clients can rarely by guaranteed. Lack of communication negatively impacts the performance of industrial automation systems, e.g. Automated Guided Vehicle (AGV) fleets. Utilizing industrial Mobile Ad-hoc NETworks (MANETs) and adaptive positioning systems can reduce the number of disconnections in these AGV fleets. Therefore the performance of the mobile systems (e.g. AGV fleet) is improved and factory efficiency increased.

We present an algorithm for explicit change detection on 3D point cloud data from a mobile mapping scenario, namely the KITTI dataset. Our method is able to partition a 3D point cloud into static and dynamic points using ray traversal of a 3D voxel grid. We are thus not using a machine learning approach or RGB camera data but instead compute the intersections of the scene volume with the lines-of-sight between the sensor and the measured points. Our approach does thus not require any object detection or tracking and has comparatively low requirements on the hardware. While our earlier work focused on dense point clouds from terrestrial 3D laser scans, here we investigate its application on the sparse 3D point clouds produced by a Velodyne laser range finder in a mobile mapping scenario and compare our results to two competing implementations using the ground truth annotation from FuseMODNet for a quantitative analysis. We also introduce spherical quadtree point cloud reduction as a way to only work on less than 1% of the original data, making processing multiple times faster while at the same time producing results with equivalent F1scores.

Modern photogrammetric methods as well as laser measurement systems make it easy to collect large 3D point clouds that sample objects or environments. As the recorded point clouds can be used to render computer-generated images and models, they are of particular interest in the domains of geographical and architectural engineering, as well as for computer graphics (e.g., games or virtual reality). However, point clouds have a huge storage demand, thus, point clouds shall be reduced by removing some of the points. This will inevitably also reduce the Quality of Experience (QoE) of media, which is rendered from the reduced point clouds. In this work, the impact of two different reduction methods on the QoE of rendered images is investigated from two point of views, i.e., based on ratings from both naive crowdworkers as well as point cloud experts.

The detection of markers or reflectors within point cloud data (PCD) is often used for 3-D scan registration, mapping, and 3-D environmental modeling. However, the reliable detection of such artifacts is diminished when PCD is sparse and corrupted by detection and spatial errors, for example, when the sensing environment is contaminated by high dust levels, such as in mines. In the radar literature, constant false alarm rate (CFAR) processors provide solutions for extracting features within noisy data; however, their direct application to sparse, 3-D PCD is limited due to the difficulty in defining a suitable noise window. Therefore, in this article, CFAR detectors are derived, which are capable of processing a 2-D projected version of the 3-D PCD or which can directly process the 3-D PCD itself. Comparisons of their robustness, with respect to data sparsity, are made with various state-of-the-art feature detection methods, such as the Canny edge detector and random sampling consensus (RANSAC) shape detection methods.

This paper presents the positioning, navigation and awareness (PNA) system developed for the Underwater Robotic Mining System of the !VAMOS! project [1]. It describes the main components of the !VAMOS! system, the PNA sensors in each of those components, the global architecture of the PNA system, and its main subsystems: Position and Navigation, Realtime Mine Modeling, 3D Virtual reality HMI and Real-time grade system. General results and lessons learn during the first mining field trial in Lee Moor, Devon, UK during the months of September and October 2017 are presented.

Detecting moving objects is of great importance for autonomous unmanned vehicle systems, and a challenging task especially in complex dynamic environments. This paper proposes a novel approach for the detection of moving objects and the estimation of their motion states using consecutive stereo image pairs on mobile platforms. First, we use a variant of the semi-global matching algorithm to compute initial disparity maps. Second, assisted by the initial disparities, boundaries in the image segmentation produced by simple linear iterative clustering are classified into coplanar, hinge, and occlusion. Moving points are obtained during ego-motion estimation by a modified random sample consensus) algorithm without resorting to time-consuming dense optical flow. Finally, the moving objects are extracted by merging superpixels according to the boundary types and their movements. The proposed method is accelerated on the GPU at 20 frames per second. The data which we use for testing and benchmarking is released, thus completing similar data sets. It includes 812 image pairs and 924 moving objects with ground truth for better algorithms evaluation. Experimental results demonstrate that the proposed method achieves competitive results in terms of moving-object detection and their motion state estimation in challenging urban scenarios.

Since the advent of thermal imaging, devices with a high optical resolution that use detector arrays to capture the emitted radiance in the thermal infrared range of an entire scene simultaneously have developed as a standard in monitoring energy related aspects. They have had a huge impact on the building industry and in manufacturing, where they are commonly used to monitor proecces that require stable temperature conditions. As beneficial as contactless measurements are, the subsequent localization of points of interest in the environment is often difficult. To overcome this problem we propose a portable system that combines thermal imaging with Augmented Reality (AR). The idea of the approach is to project the gathered temperature information back into the scene to facilitate visual inspection.

This paper concerns the an intelligent mobile application for spatial design support and security domain. Mobility has two aspects in our research: The first one is the usage of mobile robots for 3D mapping of urban areas and for performing some specific tasks. The second mobility aspect is related with a novel Software as a Service system that allows access to robotic functionalities and data over the Ethernet, thus we demonstrate the use of the novel NVIDIA GRID technology allowing to virtualize the graphic processing unit. We introduce Complex Shape Histogram, a core component of our artificial intelligence engine, used for classifying 3D point clouds with a Support Vector Machine. We use Complex Shape Histograms also for loop closing detection in the simultaneous localization and mapping algorithm. Our intelligent mobile system is built on top of the Qualitative Spatio-Temporal Representation and Reasoning framework. This framework defines an ontology and a semantic model, which are used for building the intelligent mobile user interfaces. We show experiments demonstrating advantages of our approach. In addition, we test our prototypes in the field after the end-user case studies demonstrating a relevant contribution for future intelligent mobile systems that merge mobile robots with novel data centers.

This paper describes the application of a weight estimation method for emergency patients in clinical environments. The approach applies established algorithms for point cloud processing and filtering to data from a low-cost, structured light sensor. A patient's volume is estimated on the basis of their visible front surface. The approach is currently being tested in the workflow of the emergency room at the Universitätsklinikum Erlangen, Germany. Preliminary results show the accuracy of the approach in relation to other conservative means of weight measurements, for example, by physicians and anthropometric measurements.

In this video, we illustrate how one of the classical areas of computational geometry has gained in practical relevance, which in turn gives rise to new, fascinating geometric problems. In particular, we demonstrate how the robot platform IRMA3D can produce high-resolution, virtual 3D environments, based on a limited number of laser scans. Computing an optimal set of scans amounts to solving an instance of the Art Gallery Problem (AGP): Place a minimum number of stationary guards in a polygonal region P, such that all points in P are guarded.

Heat and air conditioning losses in buildings and factories lead to a large amount of wasted energy. The Action Plan for Energy Efficiency [4] of the European Commission estimates that the largest cost-effective energy savings potential lies in residential (≈ 27%) and commercial (≈ 30%) buildings. Imagine a technology that creates a precise digital 3D model of heat distribution and heat flow enabling one to detect all sources of wasted energy and to modify buildings to reach these savings. This video presents our approach to this task. Methods for creating a consistent laser scan model enhanced with information from thermal and optical cameras are presented.

The terrestrial acquisition of 3D point clouds by laser range finders has recently moved to mobile platforms. Measuring the environment while simultaneously moving the vehicle demands a high level of accuracy from positioning systems such as the IMU, GPS and odometry. We present a novel semi-rigid SLAM algorithm that corrects the global position of the vehicle at every point in time, while simultaneously improving the quality and accuracy of the entire acquired map. Using the algorithm the temporary failure of positioning systems or the lack thereof can be compensated for. We demonstrate the capabilities of our approach on a wide variety of systems and data sets.

Finding road intersections in advance is crucial for navigation and path planning of moving autonomous vehicles, especially when there is no position or geographic auxiliary information available. In this paper, we investigate the use of a 3D point cloud based solution for intersection and road segment classification in front of an autonomous vehicle. It is based on the analysis of the features from the designed beam model. First, we build a grid map of the point cloud and clear the cells which belong to other vehicles. Then, the proposed beam model is applied with a specified distance in front of autonomous vehicle. A feature set based on the length distribution of the beam is extracted from the current frame and combined with a trained classifier to solve the road-type classification problem, i.e., segment and intersection. In addition, we also make the distinction between +-shaped and T-shaped intersections. The results are reported over a series of real-world data. A performance of above 80% correct classification is reported at a real-time classification rate of 5 Hz.

The ICP (Iterative Closest Point) algorithm is the de facto standard for geometric alignment of three-dimensional models when an initial relative pose estimate is available. The basis of the algorithm is the minimization of an error function that takes point correspondences into account. While four closed-form solution methods are known for minimizing this function, linearization seems necessary for solving the global scan registration problem. This paper presents such linear solutions for registering n-scans in a global and simultaneous fashion. It studies parameterizations for the rigid body transformations of the n-scan registration problem.

Three dimensional point clouds acquired by range scanners often do not represent the environment precisely due to noise and errors in the acquisition process. These latter systematical errors manifest as deformations of different kinds in the 3D range image. This paper presents a novel approach to correct deformations by an analysis of the structures present in the environment and correcting them by non-rigid transformations. The resulting algorithms are used for creating high-accuracy 3D indoor maps.

The recently introduced Minimum Uncertainty Maximum Consensus (MUMC) algorithm for 3D scene registration using planar-patches is tested in a large outdoor urban setting without any prior motion estimate whatsoever. With the aid of a new overlap metric based on unmatched patches, the algorithm is shown to work successfully in most cases. The absolute accuracy of its computed result is corroborated for the first time by ground-truth obtained using reflective markers. There were a couple of unsuccessful scan-pairs. These are analyzed for the reason of failure by formulating two kinds of overlap metrics: one based on the actual overlapping surface-area and another based on the extent of agreement of range-image pixels. We conclude that neither metric in isolation is able to predict all failures, but that both taken together are able to predict the difficulty level of a scan-pair vis-à-vis registration by MUMC.

We propose a new approach to appearance based loop detection from metric 3D maps, exploiting the NDT surface representation. Locations are described with feature histograms based on surface orientation and smoothness, and loop closure can be detected by matching feature histograms. We also present a quantitative performance evaluation using two real-world data sets, showing that the proposed method works well in different environments.

To advance robotic science it is important to perform experiments that can be replicated by other researchers to compare different methods. However, these comparisons tend to be biased, since re-implementations of reference methods often lack thoroughness and do not include the hands-on experience obtained during the original development process. This paper presents a thorough comparison of 3D scan registration algorithms based on a 3D mapping field experiment, carried out by two research groups that are leading in the field of 3D robotic mapping. The iterative closest points algorithm (ICP) is compared to the normal distributions transform (NDT). We also present an improved version of NDT with a substantially larger valley of convergence than previously published versions.

Time-of-flight cameras constitute a smart and fast technology for 3D perception but lack in measurement precision and robustness. The authors present a comprehensive approach for 3D environment mapping based on this technology. Imprecision of depth measurements are properly handled by calibration and application of several filters. Robust registration is performed by a novel extension to the Iterative Closest Point algorithm. Remaining registration errors are reduced by global relaxation after loop-closure and surface smoothing. A laboratory ground truth evaluation is provided as well as 3D mapping experiments in a larger indoor environment.

The great majority of path following control laws for either kinematical or dynamical mobile robot models are designed assuming ideal actuators, i.e. assuming that any commanded velocity or torque (in the kinematical and dynamical cases respectively) will be instantly implemented regardless of its value. Real actuators are far from being ideal. In particular, only bounded velocities and torques can be realized for any given command. With reference to the kinematical model of a differential drive mobile robot, a known path following control law is modified to account for actuator velocity saturation. The proposed solution is experimentally shown to be particularly useful for high speed applications where accounting for actuator velocity saturation may have a large influence on performance.

In the past many solutions for simultaneous localization and mapping (SLAM) have been presented. Recently these solutions have been extended to map large environments with six degrees of freedom (DoF) poses. To demonstrate the capabilities of these SLAM algorithms it is common practice to present the generated maps and successful loop closing. Unfortunately there is often no objective performance metric that allows to compare different approaches. This fact is attributed to the lack of ground truth data. For this reason we present a novel method that is able to generate this ground truth data based on reference maps. Further on, the resulting reference path is used to measure the absolute performance of different 6D SLAM algorithms building a large urban outdoor map.

In this paper, we present a new combination of a biologically inspired attention system (VOCUS – Visual Object detection with a CompUtational attention System) with a robust object detection method. As an application, we built a reliable system for ball recognition in the RoboCup context. Firstly, VOCUS finds regions of interest generating a hypothesis for possible locations of the ball. Secondly, a fast classifier verifies the hypothesis by detecting balls at regions of interest. The combination of both approaches makes the system highly robust and eliminates false detections. Furthermore, the system is quickly adaptable to balls in different scenarios: The complex classifier is universally applicable to balls in every context and the attention system improves the performance by learning scenario-specific features quickly from only a few training examples.

We demonstrate how one of the classical areas of computationalgeometry has reached practical application, which in turngives rise to new, fascinating geometric problems.In particular, we discuss the problem of developing a goodonline strategy for anautonomous mobile robot to locate an object that is hidden behinda corner or door.

To create with an autonomous mobile robot a 3D volumetric map of a scene it is necessary to gage several 3D scans and to merge them into one consistent 3D model. This paper provides a new solution to the simultaneous localization and mapping (SLAM) problem with six degrees of freedom. Robot motion on natural surfaces has to cope with yaw, pitch and roll angles, turning pose estimation into a problem in six mathematical dimensions. A fast variant of the Iterative Closest Points algorithm registers the 3D scans in a common coordinate system and relocalizes the robot. Finally, consistent 3D maps are generated using a global relaxation. The algorithms have been tested with 3D scans taken in the Mathies mine, Pittsburgh, PA. Abandoned mines pose significant problems to society, yet a large fraction of them lack accurate 3D maps.

This paper presents a robust and real-time capable recognition system for the fast detection and classification of objects in spatial 3D data. Depth and reflection data from a 3D laser scanner are rendered into images and fed into a saliency-based visual attention system that detects regions of potential interest. Only these regions are examined by a fast classifier. The time saving of classifying objects in salient regions rather than in complete images is linear with the number of trained object classes. Robustness is achieved by the fusion of the bi-modal scanner data; in contrast to camera images, this data is completely illumination independent. The recognition system is trained for two different object classes and evaluated on real indoor data.

This paper describes a novel, laser-based approach for tracking the pose of a high-speed mobile robot. The algorithm is outstanding in terms of accuracy and computational time, being 33 times faster than real time. The efficiency is achieved by a closed form solution for the matching of two lasers scans, the use of natural landmarks and fast linear filters. The implemented algorithm is evaluated with the high-speed robot Kurt3D (4 m/s), and compared to standard scan matching methods in indoor and outdoor environments.