The practical application of aerial-aquatic robots is hindered by severe impact loads during water entry. Existing load reduction methods are inefficient for robots requiring rapid, high-frequency aerial-aquatic transitions. Therefore, this study proposes an active learning based joint optimization framework for nose profiles and water-entry strategies, which is fully automated. Specifically, an 8-dimensional parameter space is defined for generating dataset inputs through Latin Hypercube Sampling (LHS). Furthermore, a Deep Kernel Learning (DKL) surrogate model is trained under different water-entry strategies, in order to predict peak impact loads and quantify prediction uncertainty for varying nose profiles. Within each active learning loop, the Non-dominated Sorting Genetic Algorithm III (NSGA-III) guides the selective labelling of samples to expand the dataset, and the DKL model is iteratively retrained until convergence. Compared against state-of-the-art methods, the proposed approach reduces the number of required high-fidelity Computational Fluid Dynamics (CFD) simulations to 65% of that of the comparison methods, while achieving maximum reductions of 66.9% in axial and 72.8% in normal impact loads across the parameter space, respectively. Notably, under the water-entry strategies employed by the hunting behavior of gannets, the approach yields profiles closely matching the skull morphology of the northern gannet. In this case, this work not only delivers an efficient and reliable impact loads reduction solution for aerial-aquatic robots, but also reveals the role of natural selection in minimizing such loads.

Semi-supervised learning (SSL) has gained significant attention in the domain of sea state estimation (SSE) due to its capacity to alleviate the reliance of deep learning models on extensive labeled datasets. While existing semi-supervised SSE methodologies leveraging pseudo-labeling have achieved promising results, they often overlook the challenges posed by high class imbalance and the prevalence of missing data in ship motion datasets, which restricts their broader applicability. In this article, we propose a novel SSL approach BalanceSSE based on the class-imbalanced ship motion data for SSE. This approach consists of three main modules: 1) the dynamic imputation (DIT); 2) the imbalance temporal-frequency learning (ITFL); and 3) the ClusterProx classifier (CL). The DIT module dynamically imputes incomplete ship motion data by assigning different weights to various dimensions data. The ITFL module employs time-frequency collaborative learning to generate pseudo-labels and integrate an adaptive confidence strategy to select high confidence pseudo-labels. This process is further enhanced by the CL module to produce better estimates. Experimental tests on UCR datasets and ship motion datasets demonstrate that BalanceSSE outperforms state-of-the-art methods. Ablation studies highlight the critical role of each module in BalanceSSE.

Semantic segmentation is a key technique that enables mobile robots to understand and navigate surrounding environments autonomously. However, most existing works focus on segmenting known objects, overlooking the identification of unknown classes, which is common in real-world applications. In this paper, we propose a feature-oriented framework for open-set semantic segmentation on LiDAR data, capable of identifying unknown objects while retaining the ability to classify known ones. We design a decomposed dual-decoder network to simultaneously perform closed-set semantic segmentation and generate distinctive features for unknown objects. The network is trained with multi-objective loss functions to capture the characteristics of known and unknown objects. Using the extracted features, we introduce an anomaly detection mechanism to identify unknown objects. By integrating the results of close-set semantic segmentation and anomaly detection, we achieve effective feature-driven LiDAR open-set semantic segmentation. Evaluations on both SemanticKITTI and nuScenes datasets demonstrate that our proposed framework significantly outperforms state-of-the-art methods. The source code will be made publicly available at https://github.com/nubot-nudt/DOSS.

In recent years, the engineering application value of hybrid aquatic-aerial robots (HAARs) has been increasingly validated. However, during the hydrofoil planing process, time-varying environmental disturbances can lead to significant model uncertainties in the system. To address this challenge, this paper proposes an error-oriented active disturbance rejection control (ADRC) approach. The method employs an extended state observer (ESO) to uniformly estimate and compensate for both internal and external disturbances based on real-time error signals. On this foundation, We propose an innovative time-series compensation mechanism, which uses historical characteristic information of the system to enhance suppression of time-varying uncertain interferences. Simulation results demonstrate that the proposed control strategy outperforms conventional methods in terms of anti-interference performance and robustness.

The hybrid aquatic-aerial robot (HAAR) enables efficient water-air transition through surface-skimming maneuvers, but the complex free-surface interactions and unsteady hydrodynamic effects presents formidable challenges in attitude control. To address this issue, this paper proposes an innovative HAAR configuration incorporating a shallowly submerged hydrofoil system to facilitate stable surface-skimming locomotion. Through systematic computational fluid dynamics (CFD) investigations, we quantitatively characterize the influence of hydrofoil geometric parameters on the robot’s hydrodynamic performance during surface-skimming operations. The results show that optimized hydrofoil configurations can enhance the lift-drag ratio and improve longitudinal stability margins. To address the attitude instability caused by complex near-surface flow disturbances, this paper validates the effectiveness of the innovative HAAR configuration in surface-skimming attitude stabilization using an error-driven active disturbance rejection control (ADRC) algorithm with time-series compensation. This approach ensures the attitude stability of the HAAR in complex surface environments, contributing to its practical application in cross-medium operations.

Multimodal 3D object detectors leverage the strengths of both geometry-aware LiDAR point clouds and semantically rich RGB images to enhance detection performance. However, the inherent heterogeneity between these modalities, including unbalanced convergence and modal misalignment, poses significant challenges. Meanwhile, the large size of the detection-oriented feature also constrains existing fusion strategies to capture long-range dependencies for the 3D detection tasks. In this work, we introduce a fast yet effective multimodal 3D object detector, incorporating our proposed Instance-level Contrastive Distillation (ICD) framework and Cross Linear Attention Fusion Module (CLFM). ICD aligns instance-level image features with LiDAR representations through object-aware contrastive distillation, ensuring fine-grained cross-modal consistency. Meanwhile, CLFM presents an efficient and scalable fusion strategy that enhances cross-modal global interactions within sizable multimodal BEV features. Extensive experiments on the KITTI and nuScenes 3D object detection benchmarks demonstrate the effectiveness of our methods. Notably, our 3D object detector outperforms state-of-the-art (SOTA) methods while achieving superior efficiency. The implementation of our method has been released as open-source at: https://github.com/nubot-nudt/ICD-Fusion.

In sensitive scenarios, such as meetings, negotiations, and team sports, messages must be conveyed without detection by noncollaborators. Previous methods, such as encrypting messages, eye contact, and micro-gestures, had problems with either inaccurate information transmission or leakage of interaction intentions. To this end, a novel gesture-free hand intention recognition scheme was proposed, that adopted surface electromyography (sEMG) and isometric contraction theory to recognize hand intentions without any gesture. Specifically, this work includes four aspects: first, the experimental system, consisting of the self-conducted myoelectric wristband, the matched host computer software, and the sports platform, is built to get sEMG signals and simulate multiple usage scenarios; second, the paradigm is designed to standard prompt and collect the gesture-free sEMG datasets. Eight-channel signals of ten subjects were recorded twice per subject at about 5–10 days intervals; third, the toolbox integrates preprocessing methods (data segmentation, filter, normalization, etc.), widely used sEMG classification methods, and various plotting functions, to facilitate future research based this dataset; fourth, the benchmark results of widely used methods are provided. The results involve single-day, cross-day, and cross-subject experiments of six-class and 12-class gesture-free hand intention when subjects have different time windows.

Visual Simultaneous Localization and Mapping (vSLAM) is a cornerstone technology in computer vision and robotics, underpinning applications such as autonomous vehicles and robot navigation. While traditional vSLAM systems have shown significant progress in indoor or outdoor environments, their performance often degrades in complex scenes, limiting their adaptability and robustness. Semantic vSLAM, which integrates high-level semantic information into vSLAM systems, has emerged as a promising solution to address these limitations by enabling a richer understanding of the environment. In this paper, we provide a comprehensive review of semantic vSLAM, offering a critical analysis of its evolution, methods, and challenges. We begin by revisiting the development of traditional vSLAM, emphasizing its limitations and the motivation for incorporating semantic information. Subsequently, we delve into the core modules of semantic vSLAM, including semantic extraction, object association, semantic loop closing, back-end optimization, and semantic mapping. Then, we present a performance comparison of semantic vSLAM systems under two different datasets, indoor and outdoor, respectively. Furthermore, we also provide a comparative analysis of widely used SLAM datasets to provide guidance for performance testing and validation. To further enrich the discussion, we identify unresolved challenges in semantic vSLAM, such as long-term semantic perception and association, open and unstructured environments. We propose future research directions, including balancing computational resources and quantifying system risk, large model-based navigation and mapping, and embodied AI SLAM. By providing key insights and forward-looking perspectives, this work aims to stimulate future research and improve the capabilities of semantic vSLAM in real-world applications.

Loop closing and relocalization are crucial techniques to establish reliable and robust long-term SLAM by addressing pose estimation drift and degeneration. This article begins by formulating loop closing and relocalization within a unified framework. Then, we propose a novel multi-head network LCR-Net to tackle both tasks effectively. It exploits novel feature extraction and a pose-aware attention mechanism to precisely estimate similarities and 6-DoF poses between pairs of LiDAR scans. In the end, we integrate our LCR-Net into a SLAM system and achieve robust and accurate online LiDAR SLAM in outdoor driving environments. We thoroughly evaluate our LCR-Net through three setups derived from loop closing and relocalization, including candidate retrieval, closed-loop point cloud registration, and continuous relocalization using multiple datasets. The results demonstrate that LCR-Net excels in all three tasks, surpassing the state-of-the-art methods and exhibiting a remarkable generalization ability. Notably, our LCR-Net outperforms baseline methods without using a time-consuming robust pose estimator, rendering it suitable for online SLAM applications. To our best knowledge, the integration of LCR-Net yields the first LiDAR SLAM with the capability of deep loop closing and relocalization. The implementation of our methods is open-sourced athttps://github.com/nubot-nudt/LCR-Net.

Fast and accurate path planning is important for ground robots to achieve safe and efficient autonomous navigation in unstructured outdoor environments. However, most existing methods exploiting either 2D or 2.5D maps struggle to balance the efficiency and safety for ground robots navigating in such challenging scenarios. In this paper, we propose a novel hybrid map representation by fusing a 2D grid and a 2.5D digital elevation map. Based on it, a novel path planning method is proposed, which considers the robot poses during traversability estimation. By doing so, our method explicitly takes safety as a planning constraint enabling robots to navigate unstructured environments smoothly. The proposed approach has been evaluated on both simulated datasets and a real robot platform. The experimental results demonstrate the efficiency and effectiveness of the proposed method. Compared to state-of-the-art baseline methods, the proposed approach consistently generates safer and easier paths for the robot in different unstructured outdoor environments. The implementation of our method is publicly available at https://github.com/nubot-nudt/T-Hybrid-planner.

Point cloud registration is an important task in robotics and autonomous driving to estimate the ego-motion of the vehicle. Recent advances following the coarse-to-fine manner show promising potential in point cloud registration. However, existing methods rely on good superpoint correspondences, which are hard to be obtained reliably and efficiently, thus resulting in less robust and accurate point cloud registration. In this paper, we propose a novel network, named RDMNet, to find dense point correspondences coarse-to-fine and improve final pose estimation based on such reliable correspondences. Our RDMNet uses a devised 3D-RoFormer mechanism to first extract distinctive superpoints and generates reliable superpoints matches between two point clouds. The proposed 3D-RoFormer fuses 3D position information into the transformer network, efficiently exploiting point clouds’ contextual and geometric information to generate robust superpoint correspondences. RDMNet then propagates the sparse superpoints matches to dense point matches using the neighborhood information for accurate point cloud registration. We extensively evaluate our method on multiple datasets from different environments. The experimental results demonstrate that our method outperforms existing state-of-the-art approaches in all tested datasets with a strong generalization ability.

This article presents a sliding widow-based visual-inertial odometry to deal with the micro air vehicle (MAV) pose estimation problem. Errors caused by inertial measurement unit (IMU) preintegration, visual landmarks reprojection, and marginalization, are unified into a nonlinear residual minimization framework. Furthermore, a dual-step marginalization method has been proposed to increase the computational efficiency. Experiments have been conducted on publicly available datasets, as well as customized handheld and MAV platform, where state-of-the-art approaches have served as the baselines for comparison. According to the results, the proposed method has a comparative accuracy, which can run in real-time on an onboard minicomputer.

In this paper, we address the problem of combining 3D laser scanner and camera information to estimate the motion of a mobile platform. We propose a direct laser-visual odometry approach building upon photometric image alignment. Our approach is designed to maximize the information usage of both, the image and the laser scan, to compute an accurate frame-to-frame motion estimate. To deal with the sparsity of the range measurements, our approach identifies planar point sets within individual point clouds and subsequently extract their corresponding pixel patches from the camera image. The extracted planar image patches are used together with the non-planar pixels to estimate the frame-to-frame motion using a homography formulation capable of incorporating both types of pixel alignments. To achieve high estimation accuracy, we explicitly predict possible occlusions caused by observations taken from different locations. We evaluate our proposed approach using the KITTI dataset as well as data recorded with a Clearpath Husky platform. The experiments suggest that our approach can achieve competitive estimation accuracy and produce consistently registered, colored point clouds.

Category‐independent object detection and localisation plays an important role in many computer vision tasks. In this study, an efficient method is proposed for generic objectness ranking by fusing two dimension (2D) or 3D information. A novel Bayesian model is designed to integrate multimodal cues and global cues to estimate object location, scale and number. In the pure trichannel colour space, the authors employ global spatial information as new global cues. From the colour+depth (red, green and blue+D) aspect, the authors compute multimodal saliency and oversegments to find two new multimodal cues. Local and regional depth cues are also explored and combined with them together so that a reliable objectness ranking scheme can be implemented. The proposed method is evaluated on web‐public common 2D and RGB+D datasets. In RGB+D cases, the experimental results show that the proposed method achieves an average 5% improvement over state‐of‐the‐art methods. Furthermore, for achieving the similar recall rates, the authors’ method only needs 30% amounts of sampled windows with respect of other available methods.

The scan registration is the cornerstone to Mobile Robotic Mapping, and the majority of existing global registration methods are dependent on specific features. This paper presents a global feature-less scan registration strategy based on the ground surface, which is extremely common in Mobile Robotic Mapping scenarios. The 3D rotation is decoupled from 3D translation by transforming the input scans into the Hough domain, wherein Phase Only Matched Filtering (POMF) is adopted for the partially overlapped signal registration. No particular features in the input data are prerequisite to our algorithm. The algorithm is validated by the challenging scans captured by our custom-built platform and a public dataset. The result illustrates the reliability of this algorithm to align feature-less, partially overlapped and noisy scans.

This paper presents a novel algorithm capable of generating multiple next best views (NBVs), sorted by achievable information gain. Although being designed for way-point generation in autonomous airborne mapping of outdoor environments, it works directly on raw point clouds and thus can be used with any sensor generating spatial occupancy information (e.g. LIDAR, kinect or Time-of-Flight cameras). To satisfy time-constraints introduced by operation on UAVs, the algorithm is implemented on a highly parallel architecture and benchmarked against the previous, CPU-based proof of concept. As the underlying hardware imposes limitations with regards to memory access and concurrency, necessary data structures and further performance considerations are explained in detail. Open-source code for this paper is available at http://www.github.com/benadler/.

To detect and localize objects is an indispensable step for many computer vision tasks. Most of the state-of-the-art methods of object detection and localization are category-dependent. These methods can achieve a significant performance. However, they are useless for detecting and localizing objects belonging to an unknown category when applying them to an unknown environment. In this paper, a method is proposed for detecting and localizing generic objects without specifying their categories. The proposed method combines diverse cues, including multi-scale saliency, superpixels straddling, intensity, depth and global information, into a uniform Bayesian framework to obtain accurate detection and localization. By comparison to state-of-the-art methods, our experiments show the promising performance of the proposed method based on the PASCAL VOC 08 dataset and our indoor scene dataset.