Environment description-based vehicle-scene interaction for vehicle localization in large-scale point cloud maps constructed through multi-sensor systems is critically significant for the advancement of large-scale intelligent transportation systems, such as delivery vehicles operating in the ‘last mile’. However, current vehicle-scene interaction encounter challenges due to the inability of point cloud encoders to effectively capture local details and long-range spatial relationships and a significant modality gap between text and point cloud representations. To address these challenges, we present Des4Pos, a novel two-stage text-driven 3D Lidar place recognition framework. In the coarse stage, the point-cloud encoder utilizes the Multi-scale Fusion Attention Mechanism (MFAM) to enhance local geometric features, followed by a bidirectional Long Short-Term Memory (LSTM) module to strengthen global spatial relationships. Concurrently, the Stepped Text Encoder (STE) integrates cross-modal prior knowledge from CLIP and aligns text and point-cloud features using this prior knowledge, effectively bridging modality discrepancies. In the fine stage, we introduce a Cascaded Residual Attention (CRA) module to fuse cross-modal features and predict relative localization offsets, thereby achieving greater localization precision. Experiments on the KITTI360Pose test set demonstrate that Des4Pos achieves state-of-the-art performance in text-to-point-cloud place recognition at the expense of a slight increase in the runtime. Specifically, it attains a top-1 accuracy of 40% and a top-10 accuracy of 77% under a 5-meter radius threshold, surpassing the best previous sota method Text2Loc by 8% and 7%, respectively. Our code and datasets are publicly available athttps://github.com/nuozimiaowu/Des4Pos

Precision navigation and positioning are essential for lunar landing exploration missions. Terrain-relative navigation based on crater matching provides an effective means for lander position estimation as craters are distinguishing features on lunar. However, challenges arise from the lack of a one-to-one correspondence between image-detected craters and the crater database, as well as the inconsistency of the coordinate system of craters in the image and those in the database, which complicates the matching process. This article has proposed a lunar crater matching method with triangle-based global second-order similarity for precision navigation. First, craters are constructed as triangles as the basic matching primitives, and the topological relationships between craters are transformed into a graph structure. Then, geometric constraints and triangle removal rules are designed to retain high-quality triangles that satisfy the first-order similarity. Next, a second-order similarity metric is introduced to evaluate the consistency of the topology of crater distributions from a global perspective. The global optimal crater matching is determined by constructing a second-order similarity score matrix. The proposed method is validated by comprehensive experiments using both simulation data and Chang’E-6 landing phase data. The experimental results show that the proposed method has achieved the highest accuracy and robustness among the comparison methods, and the average position estimation accuracies are 0.44% and 0.41% of flight altitude for orbiting and landing scenarios.

Recent years have witnessed a fast evolution and promising performance of the vision transformer (ViT)-based place recognizer, which aims at building a general system. State-of-the-arts (SOTAs) can hardly carry on their superiority at low light so far, thereby considerably blocking the broadening of visual place recognition-related mobile robot applications. To perform robust visual place recognition in low-light scenes, this article proposes an end-to-end trainable dark-enhanced Net, which tries to alleviate the impact of poor illumination and environmental noise. Specifically, a lightweight dark enhancement module, i.e.,$\sf ResEM$, is firstly trained to efficiently improve image illumination quality by residual-based adversarial learning. A dual-level sampling pyramid transformer, i.e.,$\sf DSPFormer$, is then constructed to extract discriminative features through aggregating reconstructed descriptors. Moreover, to improve the performance and reliability of place recognition, a reranking method based on cross-entropy loss is used for final place matching. To provide a comprehensive evaluation, we also build two challenging place benchmarks, namely,$\sf SimPlace$and$\sf DarkPlace$. Evaluations of both the public benchmarks and the newly built benchmarks show that the task-inspired design enables the recognizer to achieve significant performance improvements in the nighttime for robot place recognition compared to other top-ranked place recognizers.

The motivation of visual place recognition (VPR) is to enable robots to identify and localize specific places within an environment using visual cues, facilitating navigation, mapping, and context-aware applications. On the other hand, deeper networks impose an extra computing strain on robots and greatly impede the development of real-time robot applications. Most groundbreaking studies either focus on learning from only one teacher in their distilled approach to learning, ignoring the possibility of students learning from multiple teachers, or failing to reveal teachers place varying importance on specific examples. To cope with the above issues, we propose a novel adaptive soft-hard label teaching feature-level knowledge distillation learning framework, namely ASHT-KD, for all-day mobile robot VPR tasks. This framework learns a compact and quick all-day place recognizer through knowledge transfer from several teachers to a limited number of students. Specifically, depending on the complexity of the environments, teachers can impart knowledge to two types of students in two teaching modes: soft-label teaching and hard-label teaching, which corresponds to one type of student being required to learn a new and uncomplicated environment (query image), while the other type of students are forced to learn a more complex environment (database images). To balance computational memory and performance, the teacher network is designed to be a two-level sampling ViT pipeline, while the Siamese student network is constructed to be a lightweight pipeline consisting only of one-level down-sampling ViT for place matching. In addition, a cross-entropy loss network is introduced to further improve the VPR performance by strengthening the correlation of feature representations from the Siamese network. Extensive experiments demonstrate the effectiveness and superiority of ASHT-KD. The practicability of ASHT-KD is also verified through outdoor testing.

Abstract Unmanned aerial vehicle (UAV) image stitching is a key technology for aerial remote sensing applications. Most existing image stitching methods based on optimal seamline searching algorithms can eliminate defects such as ghosting and distortion in stitched images, which unfortunately suffer from the problem that the seamline may cross those regions with significant geometric misalignment between different images. Therefore, a novel image stitching method based on multi‐region image segmentation is proposed. The algorithm starts by performing a multi‐scale morphological reconstruction in the overlapping regions between UAV images to obtain superpixel images with precise contours. Then, the fast density peaks clustering based on K‐nearest neighbours is applied to automatically determine the clustering centres and the number of clusters. By constructing a cost function, an energy map is generated in the overlapping regions between UAV images. Finally, the optimal seamline can be determined with a graph‐cut method. Compared to several popular image stitching algorithms in real experiments, the proposed method can essentially prevent the seamline from crossing significant ground objects to ensure the integrity of structural objects while achieving satisfactory accuracy and efficiency during the UAV image stitching process.

Abstract Image stitching is an important way to achieve large‐field high‐resolution imaging. The inconsistencies in brightness and structure and defects in ghosting, blurring and misalignment between images, which are inevitable and difficult to eliminate, make a challenge to image stitching, due to the external lighting environment and changes in camera pose and parameters. Here, a novel method is proposed to search for the optimal seamline based on the fast marching method, which can stitch large parallax images with high quality. A feature weight map is first formed based on the similarity in colour, edge, texture and saliency of the images. Then it is used as the cost value of the seamline to search for the optimal seamline by fast marching method. The results show that this new method is more efficient to reduce defects, such as ghosting, misalignment and chromatic aberration, and realize high quality image stitching compared with traditional stitching tools and methods, which provides a new perspective for image stitching technology.