Light Detection and Ranging (LiDAR) technology has become crucial in robotics and autonomous systems for generating precise 3D environmental representations. However, challenges persist in achieving high accuracy and precision, especially in indoor environments. This paper rigorously analyzes the performance of various indoor Li DAR systems under different conditions. We present a novel experimental methodology, which quantifies LiDAR accuracy and precision, examining factors such as sensor type, environmental conditions, and target characteristics. Using an extensive dataset collected from nine distinct locations with more than 36000 LiDAR scans, combined with high-precision reference data from a FARO laser scanner, our analysis reveals significant insights into the accuracy and precision across different LiDAR models. The resulting public dataset, which include detailed point clouds and groundtruth labels, are expected to serve as a valuable resource for developing and validating advanced LiDAR processing techniques and benchmarks for various applications. The dataset will be publicly available at http://lidaraccuracy.github.io.

Quadrupedal animals can perform agile and playful tasks while interacting with real-world objects. For instance, a trained dog can track and catch a flying frisbee before it touches the ground, while a cat left alone at home may leap to grasp the door handle. Successfully grasping an object during high-dynamic locomotion requires highly precise perception and control. However, due to hardware limitations, agility and precision are usually a trade-off in robotics problems. In this work, we employ a perception-control decoupled system based on Reinforcement Learning (RL), aiming to explore the level of precision a quadrupedal robot can achieve while interacting with objects during high-dynamic locomotion. Our experiments show that our quadrupedal robot, mounted with a passive gripper in front of the robot’s chassis, can perform both tracking and catching tasks similar to a real trained dog. The robot can follow a mid-air ball moving at speeds of up to 3m/s and it can leap and successfully catch a small object hanging above it at a height of 1.05m in simulation and 0.8m in the real world.

Traditional robot navigation systems primarily utilize occupancy grid maps and laser-based sensing technologies, as demonstrated by the popular move_base package in ROS. Unlike robots, humans navigate not only through spatial awareness and physical distances but also by integrating external information, such as elevator maintenance updates from public notification boards and experiential knowledge, like the need for special access through certain doors. With the development of Large Language Models (LLMs), which possesses text understanding and intelligence close to human performance, there is now an opportunity to infuse robot navigation systems with a level of understanding akin to human cognition. In this study, we propose using osmAG (Area Graph in OpenStreetMap textual format), an innovative semantic topometric hierarchical map representation, to bridge the gap between the capabilities of ROS move_base and the contextual understanding offered by LLMs. Our methodology employs LLMs as an actual copilot in robot navigation, enabling the integration of a broader range of informational inputs, while maintaining the robustness of traditional robotic navigation systems. Our code, demo, map, and experiment results can be accessed at https://github.com/xiexiexiaoxiexie/Intelligent-LiDAR-Navigation-LLM-as-Copilot.

Humans exhibit diverse and expressive whole-body movements. However, attaining human-like whole-body coordination in humanoid robots remains challenging, as conventional approaches that mimic whole-body motions often neglect the distinct roles of upper and lower body. This oversight leads to computationally intensive policy learning and frequently causes robot instability and falls during real-world execution. To address these issues, we propose Adversarial Locomotion and Motion Imitation (ALMI), a novel framework that enables adversarial policy learning between upper and lower body. Specifically, the lower body aims to provide robust locomotion capabilities to follow velocity commands while the upper body tracks various motions. Conversely, the upper-body policy ensures effective motion tracking when the robot executes velocity-based movements. Through iterative updates, these policies achieve coordinated whole-body control, which can be extended to loco-manipulation tasks with teleoperation systems. Extensive experiments demonstrate that our method achieves robust locomotion and precise motion tracking in both simulation and on the full-size Unitree H1-2 robot. Additionally, we release a large-scale whole-body motion control dataset featuring high-quality episodic trajectories from MuJoCo simulations. The project page is https://almi-humanoid.github.io.

Reflective surfaces present a persistent challenge for reliable 3D mapping and perception in robotics and autonomous systems. However, existing reflection datasets and benchmarks remain limited to sparse 2D data. This paper introduces the first large-scale 3D reflection detection dataset containing more than 50,000 aligned samples of multi-return Lidar, RGB images, and 2D/3D semantic labels across diverse indoor environments with various reflections. Textured 3D ground truth meshes enable automatic point cloud labeling to provide precise ground truth annotations. Detailed benchmarks evaluate three Lidar point cloud segmentation methods, as well as current state-of-the-art image segmentation networks for glass and mirror detection. The proposed dataset advances reflection detection by providing a comprehensive testbed with precise global alignment, multi-modal data, and diverse reflective objects and materials. It will drive future research towards reliable reflection detection. The dataset is publicly available at http://3dref.github.io

Given a building floorplan, humans can localize themselves by matching the observation of the environment with the floorplan using geometric, semantic, and topological clues. Inspired by this insight, this paper proposes a learning- based topometric robot localization method FloorplanNet, which implements a match between a metric robot map and the potentially inaccurate building floorplan in nonuniform scales and different shapes by semantic information. The method uses a novel Graph Neural Network to learn descriptors of nodes from topometric graphs generated from the input maps. We demonstrate that our method can match the 3D point cloud sub-map generated by the robot during the SLAM process with the 2D map. Furthermore, we apply our map-matching algorithm for real-world robot localization. We evaluate our method on several publicly available real-world datasets. Even though our network is solely trained using simulation data, our method demonstrates high robustness and effectiveness in real- world indoor environments and outperforms the existing SOTA map-matching algorithms. We further develop a simulator that automatically creates and annotates the required training data to train our neural networks. The method and simulator are released at: https://github.com/fengdelin/FloorplanNet.git

Benchmarking Simultaneous Localization and Mapping (SLAM) algorithms is important to scientists and users of robotic systems alike. But through their many configuration options in hardware and software, SLAM systems feature a vast parameter space that scientists up to now were not able to explore. The proposed SLAM Hive Benchmarking Suite is able to analyze SLAM algorithms in 1000's of mapping runs, through its utilization of container technology and deployment in a cluster. This paper presents the architecture and open source implementation of SLAM Hive and compares it to existing efforts on SLAM evaluation. Furthermore, we highlight the function of SLAM Hive by exploring some open source algorithms on public datasets in terms of accuracy. We compare the algorithms against each other and evaluate how parameters effect not only accuracy but also CPU and memory usage. Through this we show that SLAM Hive can become an essential tool for proper comparisons and evaluations of SLAM algorithms and thus drive the scientific development in the research on SLAM.

With the increasing application of robots, stable and efficient Visual Odometry (VO) algorithms are becoming more and more important. Based on the Fourier Mellin Transformation (FMT) algorithm, the extended Fourier Mellin Transformation (eFMT) is an image registration approach that can be applied to downward-looking cameras, for example on aerial and underwater vehicles. eFMT extends FMT to multi-depth scenes and thus more application scenarios. It is a visual odometry method which estimates the pose transformation between three overlapping images. On this basis, we develop an optimized eFMT algorithm that improves certain aspects of the method and combines it with back-end optimization for the small loop of three consecutive frames. For this we investigate the extraction of uncertainty information from the eFMT registration, the related objective function and the graph-based optimization. Finally, we design a series of experiments to investigate the properties of this approach and compare it with other VO and SLAM (Simultaneous Localization and Mapping) algorithms. The results show the superior accuracy and speed of our o-eFMT approach, which is published as open source.

Multi-perspective cameras are quickly gaining importance in many applications such as smart vehicles and virtual or augmented reality. However, a large system size or absence of overlap in neighbouring fields-of-view often complicate their calibration. We present a novel solution which relies on the availability of an external motion capture system. Our core contribution consists of an extension to the hand-eye calibration problem which jointly solves multi-eye-to-base problems in closed form. We furthermore demonstrate its equivalence to the multi-eye-in-hand problem. The practical validity of our approach is supported by our experiments, indicating that the method is highly efficient and accurate, and outperforms existing closed-form alternatives.

Spatiotemporal calibration of sensors, especially of those which do not share their fields of view, is becoming increasingly important in the fields of autonomous driving and robotics. This paper presents a general sensor calibration method, named Multical, that makes use of multiple planar calibration targets whose poses will be estimated alongside spatiotemporal calibration. Multical exploits continuous-time curves to represent the state of the sensor platform during data collection, and thus is a general framework to calibrate different kinds of sensors and deal with both spatial as well as temporal offsets. Multical includes algorithms to estimate the initial guesses of spatial transformations between sensors, and also the relative poses between calibration targets. Users do not need to provide any extrinsic priors. We apply the proposed calibration approach to both simulated and real-world experiments, and the results demonstrate the high fidelity of the proposed method.

Spectral methods such as the improved Fourier Mellin Invariant (iFMI) transform have proved to be faster, more robust and accurate than feature based methods on image registration. However, iFMI is restricted to work only when the camera moves in 2D space and has not been applied on omni-cameras images so far. In this work, we extend the iFMI method and apply a motion model to estimate an omni-camera's pose when it moves in 3D space. In the experiment section, we compare the extended iFMI method against ORB and AKAZE feature based approaches on three datasets, showing different types of environments: office, lawn and urban scenery (MPI-omni dataset). The results show that our method reduces the error of the camera pose estimation two to three times with respect to the feature registration techniques, while offering lower processing times.

Underwater robot interventions require a high level of safety and reliability. A major challenge to address is a robust and accurate acquisition of localization estimates, as it is a prerequisite to enable more complex tasks, e.g. floating manipulation and mapping. State-of-the-art navigation in commercial operations, such as oil& gas production (OGP), rely on costly instrumentation. These can be partially replaced or assisted by visual navigation methods, especially in deep-sea scenarios where equipment deployment has high costs and risks. Our work presents a multimodal approach that adapts state-of-the-art methods from on-land robotics, i.e., dense point cloud generation in combination with plane representation and registration, to boost underwater localization performance. A (a) two-stage navigation scheme is proposed that initially generates a coarse probabilistic map of the workspace, which is used to filter noise from computed point clouds and planes in the second stage. Furthermore, an adaptive decision-making approach is introduced that determines which perception cues to incorporate into the localization filter to optimize accuracy and computation performance. Our approach is investigated first in simulation and then validated with data from field trials in OGP monitoring and maintenance scenarios.

We propose a novel pose estimation method for geometric vision of omni-directional cameras. On the basis of the regularity of the pixel movement after camera pose changes, we formulate and prove the sinusoidal relationship between pixels movement and camera motion. We use the improved Fourier-Mellin invariant (iFMI) algorithm to find the motion of pixels, which was shown to be more accurate and robust than the feature-based methods. While iFMI works only on pin-hole model images and estimates 4 parameters (x, y, yaw, scaling), our method works on panoramic images and estimates the full 6 DoF3D transform, up to an unknown scale factor. For that we fit the motion of the pixels in the panoramic images, as determined by iFMI, to two sinusoidal functions. The offsets, amplitudes and phase-shifts of the two functions then represent the 3D rotation and translation of the camera between the two images. We perform experiments for 3D rotation, which show that our algorithm outperforms the feature-based methods in accuracy and robustness. We leave the more complex 3D translation experiments for future work.

In this paper, we demonstrate our work on Gaussian Process Occupancy Mapping (GPOM). We concentrate on the inefficiency of the frame computation of the classical GPOM approaches. In robotics, most of the algorithms are required to run in real time. However, the high cost of computation makes the classical GPOM less useful. In this paper we dont try to optimize the Gaussian Process itself, instead, we focus on the application. By analyzing the time cost of each step of the algorithm, we find a way that to reduce the cost while maintaining a good performance compared to the general GPOM framework. From our experiments, we can find that our model enables GPOM to run online and achieve a relatively better quality than the classical GPOM.

Hand-eye calibration is a well-known calibration problem. The problem assumes that a camera (eye) is rigidly mounted to the gripper (hand) of a robot arm and aims to find the transformation between them. In this paper, we propose a novel pipeline for hand-eye calibration without the use of a calibration target. First we employ feature extraction and matching, followed by an initial hand-eye calibration step using 2-view matches. In an iterative process, we then alternately employ triangulation and bundle adjustment to optimize the reconstruction and the hand-eye calibration result. Unlike in structure from motion and traditional hand-eye calibration, during this process we always determine the global camera poses using the hand poses and the estimated hand-eye transformation. Synthetic-data and real-data experiments are performed to evaluate the proposed approach, and the results indicate that the accuracy of our approach is superior to state-of-the-art approaches. Moreover, the speed of our algorithm is faster than existing methods.

Mapping is an important task for mobile robots. But assessing the quality of maps in a simple, efficient and automated way is not trivial and an ongoing research topic. A new approach on map evaluation is presented here. It is based on Topology Graphs as a topological, abstracted representation of 2D grid maps. The Topology Graphs are derived from Voronoi Diagrams that get post-processed to capture the high-level spatial structures. Based on a similarity metric on vertices in Topology Graphs, the vertices can be matched across maps and spatial (dis)similarities and hence errors in the mapping can be identified and measured. More precisely, the vertex-similarity is the basis to match the structures of Topology Graphs up to the identification of subgraph isomorphisms through wave-front propagation. This allows to determine important map quality attributes up to very challenging structural elements like brokenness, i.e., the number of locally correct partitions in the candidate map and their relative placement towards each other. Experiments with real robot generated maps including examples from various teams in the RoboCup Rescue competition are used to validate the usefulness of this method for map quality assessment.

A core challenge in probabilistic mapping is to extract meaningful uncertainty information from data registration methods. While this has been investigated in ICP-based scan matching methods, other registration methods have not been analyzed. In this paper, an uncertainty analysis of a Fourier Mellin based image registration algorithm is introduced, which to our knowledge is the first of its kind involving spectral registration. A covariance matrix is extracted from the result of a Phase-Only Matched Filter, which is interpreted as a probability mass function. The method is embedded in a pose graph implementation for Simultaneous Localization and Mapping (SLAM) and validated with experiments in the underwater domain.

This article addresses fast 3D mapping by a mobile robot in a predominantly planar environment. It is based on a novel pose registration algorithm based entirely on matching features composed of plane-segments extracted from point-clouds sampled from a 3D sensor. The approach has advantages in terms of robustness, speed and storage as compared to the voxel based approaches. Unlike previous approaches, the uncertainty in plane parameters is utilized to compute the uncertainty in the pose computed by scan-registration. The algorithm is illustrated by creating a full 3D model of a multi-level robot testing arena.

This paper presents a new technique for the update of a probabilistic spatial occupancy grid map using a forward sensor model. Unlike currently popular inverse sensor models, forward sensor models can be found experimentally and can represent sensor characteristics better. The formulation is applicable to both 2D and 3D range sensors and does not have some of the theoretical and practical problems associated with the current approaches which use forward models. As an illustration of this procedure, a new prototype 3D forward sensor model is derived using a beam represented as a spherical sector. Furthermore, this model is used for fusion of point-clouds obtained from different 3D sensors, in particular, time-of-flight sensors (Swiss-ranger, laser range finders), and stereo vision cameras. Several techniques are described for an efficient data-structure representation and implementation. The range beams from different sensors are fused in a common local Cartesian occupancy map. Experimental results of this fusion are presented and evaluated using Hough-transform performed on the grid.