LiDAR point cloud is essential for autonomous vehicles, but motion distortions from dynamic objects degrade the data quality. While previous work has considered distortions caused by ego motion, distortions caused by other moving objects remain largely overlooked, leading to errors in object shape and position. This distortion is particularly pronounced in high-speed environments such as highways and in multi-LiDAR configurations, a common setup for heavy vehicles. To address this challenge, we introduce HiMo, a pipeline that repurposes scene flow estimation for non-ego motion compensation, correcting the representation of dynamic objects in point clouds. We further propose SeFlow++, a real-time scene flow estimator that achieves state-of-the-art performance on both scene flow and motion compensation. We validate HiMo through extensive experiments on Argoverse 2, ZOD and a newly collected real-world dataset featuring highway driving and multi-LiDAR-equipped heavy vehicles.

Accurate ground truth annotations are critical to supervised learning and evaluating the performance of autonomous vehicle systems. These vehicles are typically equipped with active sensors, such as LiDAR, which scan the environment in predefined patterns. 3D box annotation based on data from such sensors is challenging in dynamic scenarios, where objects are observed at different timestamps, hence different positions. Without proper handling of this phenomenon, systematic errors are prone to being introduced in the box annotations. Our work is the first to discover such annotation errors in widely used, publicly available datasets. Through our novel offline estimation method, we correct the annotations so that they follow physically feasible trajectories and achieve spatial and temporal consistency with the sensor data. For the first time, we define metrics for this problem; and we evaluate our method on the Argoverse 2, MAN TruckScenes, and our proprietary datasets. Our approach increases the quality of box annotations by more than 17% in these datasets. Furthermore, we quantify the annotation errors in them and find that the original annotations are misplaced by up to 2.5 m, with highly dynamic objects being the most affected. Finally, we test the impact of the errors in benchmarking and find that the impact is larger than the improvements that state-of-the-art methods typically achieve w.r.t. the previous state-of-the-art methods; showing that accurate annotations are essential for correct interpretation of performance. Our code is available at https://github.com/alexandre-justo-miro/annotation-correction-3D-boxes.

Scene flow enables an understanding of the motion characteristics of the environment in the 3D world. It gains particular significance in the long-range, where object-based perception methods might fail due to sparse observations far away. Although significant advancements have been made in scene flow pipelines to handle large-scale point clouds, a gap remains in scalability with respect to long-range. We attribute this limitation to the common design choice of using dense feature grids, which scale quadratically with range. In this paper, we propose Sparse Scene Flow (SSF), a general pipeline for long-range scene flow, adopting a sparse convolution based backbone for feature extraction. This approach introduces a new challenge: a mismatch in size and ordering of sparse feature maps between time-sequential point scans. To address this, we propose a sparse feature fusion scheme, that augments the feature maps with virtual voxels at missing locations. Additionally, we propose a range-wise metric that implicitly gives greater importance to faraway points. Our method, SSF, achieves state-of-the-art results on the Argoverse2 dataset, demonstrating strong performance in long-range scene flow estimation. Our code is open-sourced at https://github.com/KTH-RPL/SSF.git.

LiDAR point cloud is essential for autonomous vehicles, but motion distortions from dynamic objects degrade the data quality. While previous work has considered distortions caused by ego motion, distortions caused by other moving objects remain largely overlooked, leading to errors in object shape and position. This distortion is particularly pronounced in high-speed environments such as highways and in multi-LiDAR configurations, a common setup for heavy vehicles. To address this challenge, we introduce HiMo, a pipeline that repurposes scene flow estimation for non-ego motion compensation, correcting the representation of dynamic objects in point clouds. During the development of HiMo, we observed that existing self-supervised scene flow estimators often produce degenerate or inconsistent estimates under high-speed distortion. We further propose SeFlow++, a real-time scene flow estimator that achieves state-of-the-art performance on both scene flow and motion compensation. Since well-established motion distortion metrics are absent in the literature, we introduce two evaluation metrics: compensation accuracy at a point level and shape similarity of objects. We validate HiMo through extensive experiments on Argoverse 2, ZOD and a newly collected real-world dataset featuring highway driving and multi-LiDAR-equipped heavy vehicles. Our findings show that HiMo improves the geometric consistency and visual fidelity of dynamic objects in LiDAR point clouds, benefiting downstream tasks such as semantic segmentation and 3D detection. See https://kin-zhang.github.io/HiMo for more details.

3D object detection at long-range is crucial for ensuring the safety and efficiency of self-driving vehicles, allowing them to accurately perceive and react to objects, obstacles, and potential hazards from a distance. But most current state-of-the-art LiDAR based methods are range limited due to sparsity at long-range, which generates a form of domain gap between points closer to and farther away from the ego vehicle. Another related problem is the label imbalance for faraway objects, which inhibits the performance of Deep Neural Networks at long-range. To address the above limitations, we investigate two ways to improve long-range performance of current LiDAR-based 3D detectors. First, we combine two 3D detection networks, referred to as range experts, one specializing at near to mid-range objects, and one at long-range 3D detection. To train a detector at long-range under a scarce label regime, we further weigh the loss according to the labelled point’s distance from ego vehicle. Second, we augment LiDAR scans with virtual points generated using Multimodal Virtual Points (MVP), a readily available image-based depth completion algorithm. Our experiments on the long-range Argoverse2 (AV2) dataset indicate that MVP is more effective in improving long range performance, while maintaining a straightforward implementation. On the other hand, the range experts offer a computationally efficient and simpler alternative, avoiding dependency on image-based segmentation networks and perfect camera-LiDAR calibration.

To deploy Micro Aerial Vehicles (MAVs) in real-world applications, there is a need for online methods to cope with uncertainties in localization and external disturbances. In this article, we propose a set of novel real-time embedded Nonlinear Model Predictive Control (NMPC) and Nonlinear Moving Horizon Estimation (NMHE) modules for MAV based external disturbance rejection. The NMPC and NMHE are based on the dynamic model of the MAV, thus, avoiding the need for system identification and creating specific aerodynamic models, a benefit that results in a generic solution capable of being independent of the type of the MAVs. As it will be presented, the NMHE estimates the external forces, while the NMPC generates thrust and attitude commands for the low-level controller to compensate the various disturbances that could occur, such as wind gusts, tethered payload, and varying center of gravity. The proposed method is evaluated extensively in multiple experimental results that include the scenarios of position hold against an actuating wind-wall, adding payload, and changing the MAV’s arm configurations.

Robots’ simultaneous relative pose estimation has become an essential step in most robotic-oriented problems, such as map merging, collision avoidance, path planning, and multi-Simultaneous Localization and Mapping (SLAM). This article addresses the problem of 3D and ego-centric relative pose estimation for a team of robots equipped with Ultra WideBand (UWB) nodes. More specifically, the article introduces a novel optimization framework to obtain pose information based on the embodiment of UWB ranges, without relying on any fixed external infrastructure configuration of UWB anchors on the surrounding environment. In the proposed method, we demonstrate the validity through the utilization of a Micro Aerial Vehicle (MAV) and a ground vehicle that are equipped with multiple UWB transceivers, and each platform simultaneously acts as a based anchor for the other platform for extracting an ego-centric position estimation of the UWB nodes. Additionally, for the pose estimation, the obtained information is fused with the onboard Inertial Measurement Unit (IMU) measurements on each of the considered robotic platforms. Finally, the efficacy of the proposed theoretical framework is evaluated in multiple experiments, where the aerial and ground platforms are simultaneously and separately navigating, and the ego-centric collaborative pose-estimation is compared with a VICON ground truth positioning system.

This article focuses on a spacecraft trajectory planning algorithm that allows observation of multiple site locations on the asteroid surface, while avoiding any collision with debris objects trapped in the asteroid's gravity field. Asteroids provide a challenging target for satellite based visual coverage missions, since they are partially illuminated, rotating, irregular shaped celestial bodies with a low but also irregular gravity field. For addressing this problem, an optimization approach for visual coverage is proposed with an objective to determine the sequence of the imaging site locations and the associated safe and fuel efficient trajectories, while considering rotational dynamics of the asteroid, changing illumination condition for each site, irregular gravity constraints of the asteroid and the safe separation distance from the moving debris object. Numerical simulations are performed to demonstrate the ability of the trajectory planner to ensure successful optimal coverage of all the desired asteroid site locations. letterpaper, 10 pt.

Robust localization is a fundamental capability to increase the autonomy levels of robotic platforms. A core processing step in vision based odometry methods is the extraction and tracking of distinctive features in the image frame. Nevertheless, when deploying robots in challenging environments like underground tunnels, the sensor measurements are noisy with lack of information due to low light conditions, introducing a bottleneck for feature detection methods. This paper proposes a deep classifier Convolutional Neural Network (CNN) architecture to retain detailed and noise tolerant feature maps from RBG images, establishing a novel feature tracking scheme in the context of localization. The proposed method is feeding the RGB image into the AlexNet or VGG-16 network and extracts a feature map at a specific layer. This feature map consists of feature points which are then paired between frames resulting in a discrete vector field of feature change. Finally, the proposed method is evaluated with RGB camera footage of the Micro Aerial Vehicle (MAV) flights in dark underground mines and the performance is compared with existing feature extraction methods, while the noise is added to the images.

Degraded Subterranean environments are an attractive case for miniature aerial vehicles, since there is a constant need to increase the safety operations in underground mines. The starting point for integrating aerial vehicles in the mining process is the capability to reliably navigate along tunnels. Inspired by recent advancements, this paper presents a collection of different, experimentally verified, methods tackling the problem of MAVs heading regulation while navigating in dark and textureless tunnel areas. More specifically, four different methods are presented in this work with the common goal to identify open space in the tunnel and align the MAV heading using either visual sensor in methods a) single image depth estimation, b) darkness contour detection, c) Convolutional Neural Network (CNN) regression and 2D Lidar sensor in method d) range geometry. For the works a)‐c) the dark scene in the middle of the tunnel is considered as open space and is processed and converted to yaw rate command, while d) examines the geometry of the range measurements to calculate the yaw rate command. Experimental results from real underground tunnel demonstrate the performance of the methods in the field, while setting the ground for further developments in the aerial robotics community.

This article proposes a novel visual framework for detecting tunnel crossings/junctions in underground mine areas towards the autonomous navigation of Micro Aerial Vehicles (MAVs). Usually mine environments have complex geometries, including multiple crossings with different tunnels that challenge the autonomous planning of aerial robots. Towards the envisioned scenario of autonomous or semi-autonomous deployment of MAVs with limited Line-of-Sight in subterranean environments, the proposed module acknowledges the existence of junctions by providing crucial information to the autonomy and planning layers of the aerial vehicle. The capability for a junction detection is necessary in the majority of mission scenarios, including unknown area exploration, known area inspection and robot homing missions. The proposed novel method has the ability to feed the image stream from the vehicles on-board forward facing camera in a Convolutional Neural Network (CNN) classification architecture, expressed in four categories: 1) left junction, 2) right junction, 3) left & right junction, and 4) no junction in the local vicinity of the vehicle. The core contribution stems for the incorporation of AlexNet in a transfer learning scheme for detecting multiple branches in a subterranean environment. The validity of the proposed method has been validated through multiple data-sets collected from real underground environments, demonstrating the performance and merits of the proposed module.

This article presents a Convolutional Neural Network (CNN) method to enable autonomous navigation of low-cost Micro Aerial Vehicle (MAV) platforms along dark underground mine environments. The proposed CNN component provides online heading rate commands for the MAV by utilising the image stream from the on-board camera, thus allowing the platform to follow a collision-free path along the tunnel axis. A novel part of the developed method consists of the generation of the data-set used for training the CNN. More specifically, inspired from single image haze removal algorithms, various image data-sets collected from real tunnel environments have been processed offline to provide an estimation of the depth information of the scene, where ground truth is not available. The calculated depth map is used to extract the open space in the tunnel, expressed through the area centroid and is finally provided in the training of the CNN. The method considers the MAV as a floating object, thus accurate pose estimation is not required. Finally, the capability of the proposed method has been successfully experimentally evaluated in field trials in an underground mine in Sweden.

This work presents an aerial tool towards the autonomous cooperative coverage and inspection of a large scale 3D infrastructure using multiple Unmanned Aerial Vehicles (UAVs). In the presented approach the UAVs are relying only on their onboard computer and sensory system, deployed for inspection of the 3D structure. In this application each agent covers a different part of the scene autonomously, while avoiding collisions. The autonomous navigation of each platform on the designed path is enabled by the localization system that fuses Ultra Wideband with inertial measurements through an Error- State Kalman Filter. The visual information collected from the aerial team is collaboratively processed to create the 3D model. The performance of the overall setup has been experimentally evaluated in realistic wind turbine inspection experiments, providing dense 3D reconstruction of the inspected structures.

In this article, we propose a planning algorithm for coverage of complex structures with a network of robotic sensing agents, with multi-robot surveillance missions as our main motivating application. The sensors are deployed to monitor the external surface of a 3D structure. The algorithm controls the motion of each sensor so that a measure of the collective coverage attained by the network is nondecreasing, while the sensors converge to an equilibrium configuration. A modified version of the algorithm is also provided to introduce collision avoidance properties. The effectiveness of the algorithm is demonstrated in a simulation and validated experimentally by executing the planned paths on an aerial robot.