Plenty of effort has been devoted to solving the nonlinear radiation distortions (NRDs) in planetary image matching. The mainstream solutions convert multimodal images into “single” modal images, which requires building the intermediate modalities of images. Phase congruency (PC) features have been widely used to construct intermediate modalities due to their excellent structure extraction capabilities and have proven their effectiveness on Earth remote sensing images. However, when dealing with large-scale planetary remote sensing images (PRSIs), traditional PC features constructed based on the log-Gabor filter take considerable time, counterproductive to global topographic mapping. To address the efficiency issue, this work proposes a fast planetary image-matching method based on efficient PC-based feature transform (EPCFT). Specifically, we introduce a method to calculate PC using Gaussian first- and second-order derivatives, called efficient PC (EPC). Different from the log-Gabor filter, which is sensitive to structures in a single direction,$\rm EPC$uses circularly symmetric filters to equally process changes in all directions. The experiments with 100 image pairs show that compared with other methods, the efficiency of our method is nearly doubled without loss of accuracy.

In planetary environments, conducting autonomous exploration tasks requires rovers to autonomously navigate the scene and achieve a detailed understanding of the terrain. Vision-based simultaneous localization and mapping (SLAM), which utilizes compact and low-power visual sensors for autonomous exploration, offers significant advantages in hardware deployment. While several methods have been proposed to apply visual navigation in planetary scenarios, they often rely on aerial imagery from orbiters and high-resolution DEMs for assistance. Additionally, accurate camera poses are typically required for dense matching during scene reconstruction, and the inability to perform loop closure significantly limits the performance of visual SLAM. Here, we propose a monocular visual SLAM approach combined with an adjusted neural radiance field for autonomous navigation and 3D reconstruction in planetary environments. Our approach solely relies on visual images as input and leverages the powerful learning capabilities of neural radiance fields to adapt to unseen scenes while simultaneously regressing both camera poses and scene representations. The estimated depth maps and poses can be further used for 3D reconstruction, assisting planetary exploration missions. The proposed method was tested on the Devon and MADMAX datasets that simulate planetary environments and achieved remarkable results. Even under the fixed rover navigation perspective, our pose estimation accuracy outperforms classical visual SLAM and other deep learning-based SLAM methods. Additionally, our novel view synthesis results exhibit quality comparable to those in terrestrial scenes. Comparisons with MVS techniques in terms of 3D reconstruction demonstrate that our approach recovers finer surface details. We also applied our method to the Perseverance rover dataset and achieved satisfactory positioning and reconstruction results in a real Martian environment, proving the practical feasibility of our method.

Digital Surface Models (DSMs) extracted from multi-view satellite images have extensive applications in the filed of photogrammetry. Although Neural Radiance Fields (NeRF) has shown significant potential in 3D reconstruction, most existing NeRF methods for satellite scenes adopt an end-to-end single-branch network structure, making it difficult to achieve fine-grained modeling of multiple complex terrain simultaneously, and performs poorly in weak-texture regions. Meanwhile, deep MLP structures are prone to information degradation during feature transmission, further affecting the completeness and accuracy of DSMs. To address these challenges, we propose RDS-NeRF, a novel NeRF framework integrating residual feature enhancement and depth supervision. The method introduces a residual feature enhancement structure to alleviate the problem of information degradation during feature transmission in the network and improve the model’s ability to model local details and low-texture regions. Additionally, estimated depth maps are incorporated as global geometric priors to guide the network in constructing more accurate and complete 3D structures. Experiments on the WorldView-3 satellite imagery datasets across multiple typical land cover types (building, road, water body, and vegetation) and complex scenes integrating multiple land features demonstrate that RDS-NeRF outperforms mainstream methods in terms of DSM accuracy, completeness, and novel view synthesis quality. Ablation experiments further validate the complementarity and effectiveness of the residual enhancement and depth supervision mechanisms across different scene types. In conclusion, RDS-NeRF provides a new and effective solution for generating high-quality DSMs from satellite imagery with adaptability to multiple scenes. Code will be available at https://github.com/dfsvdgf/RDS-NeRF.

Digital elevation models (DEMs) play a crucial role in scientific research and exploration mission of the lunar south pole regions, as they offer vital topographic and morphological information of terrain surfaces. Surface points processed from various remote sensors provide a discrete but accurate measurements of lunar terrains. However, reconstructing high-resolution terrains from surface points poses significant challenges for neural networks primarily due to the lack of grid connectivity inherent in unordered surface points. Therefore, this study develops a point-based terrain reconstruction framework that incorporates learnable neural shape priors to better capture complex terrain geometry. To model structural characteristics inherent in three-dimensional (3-D) surface points, signed distance field (SDF) is introduced as a continuous representation of lunar surfaces. Moreover, we train a 3-D point-based encoder–decoder network that allows terrain modeling from unstructured surface points with encoded planar features. Additionally, a dynamic planar features strategy is proposed to optimize the framework’s performance to the characteristics of terrain flatness. Experiments were conducted on photoclinometry DEMs of the lunar polar region to demonstrate the advantages of our proposed framework. The qualitative and quantitative results highlight that our framework reconstructs accurate terrain geometry, outperforming traditional methods in terms of elevation, slope, aspect and roughness. Furthermore, the model exhibits a generalization ability when applied to both Lunar Orbiter Laser Altimeter (LOLA) measurements and unseen regions, underscoring its effectiveness in lunar exploration tasks.

The accurate matching of planetary remote sensing images (PRSIs) is the premise of accurate planetary terrain mapping. However, PRSIs often lack apparent man-made structures such as buildings or roads, leading to difficulties in feature description. In addition, the PRSIs collected by different sensors are affected by the imaging mechanism and the solar illumination, and there are obvious nonlinear radiation differences (NRDs). These problems make the matching of PRSIs difficult. To address the above issues, this article proposes a PRSI matching method based on optimized bandpass filtering and local topological prior, divided into two stages: coarse matching and fine matching. In the coarse matching stage, we first use the bandpass filtering to calculate the phase congruency (PC). Then, the feature block descriptors are constructed, and the local topology consensus is used to achieve the coarse alignment of feature blocks. Finally, we extract the point features and use the matching results of block features to narrow the matching range of point features. Based on the coarse matching results, the precision and reliability of the results are further improved through fine matching. The experimental results achieved with a PRSI dataset with 75 image pairs demonstrate that our method is superior to other recent methods, the matching accuracy of the proposed method is improved by more than 2.367 pixels, and the success rate is improved by over 22.667%. The source code will be publicly available athttps://github.com/WGY-RS/OFLP.

Mapping of planetary surface using stereo orbital imagery is a basic task of planetary exploration. As the number of orbiter images increases, excessive images will lead to unnecessary time costs and accuracy loss during the photogrammetric mapping process. However, existing studies focus more on the factors that affect the mapping accuracy and often ignores the redundancy and distribution rationality of images in the mapping process. Therefore, optimal data selection from extremely redundant orbiter images has become a challenging and urgent issue for planetary mapping. This paper proposes an effective optimal selection method of off-track stereo pairs, which extends the selection operation from a local to a global perspective, and reduces the redundancy of image pair subset covering the region of interest (ROI) based on multi-constraints such as quality constraints and global distribution constraints. Specifically, the ROI is rasterized and gridded, and the optimal and candidate stereo pairs are selected for each grid based on the quantified quality constraints. By combining global distribution constraints and consistency constraints between adjacent stereo pairs, an optimization model is constructed and the optimal subset of stereo pairs is determined using belief propagation algorithm. Experiments conducted on the selected ROIs of Moon and Mars indicate that our method significantly reduces data redundancy while ensuring the quality and rational distribution of stereo image pairs. Qualitative and quantitative evaluations demonstrate that the stereo pairs obtained using the proposed method perform exceptionally well in terms of intersection angles, redundancy rates, and other factors, showing overall superior performance compared to other methods. Additionally, the stereo pairs obtained by the proposed method were employed for photogrammetric mapping within the ROI, and the results validate the mapping products also exhibit satisfactory accuracy.

Deep learning-based feature matching methods have been actively explored for satellite image bundle block adjustment, leveraging their inherent robustness to large geometric distortions and radiometric differences—key challenges in multi-source optical image processing. However, their practical utility remains limited by two critical bottlenecks: poor adaptability to extreme scale variations and prohibitive computational costs for large-format satellite data. To address these limitations, we propose a learning-based feature representation method enhanced by nonlinear diffusion filtering, with two targeted innovations: (1) Nonlinear diffusion filtering with terrain-adaptive parameters is integrated into a novel image tiling strategy, which preserves local feature integrity while enabling consistent correspondence across heterogeneous satellite data sources; (2) A top-down pyramid construction mechanism that incorporates local continuity constraints and an adaptive matching strategy selection protocol optimizes scale space exploration efficiency while safeguarding matching quality. Experiments on Earth observation and Martian image datasets validate the method’s superiority: it achieves the highest matching success rate (97.25%) and the highest BBA accuracy(1.65 pixel) among competing approaches, alongside competitive efficiency. This performance advantage is particularly pronounced under extreme scale differences and challenging imaging conditions, confirming its suitability for high-precision remote sensing applications.

The 3D reconstruction of the lunar surface is a fundamental task, with stereo matching algorithms play a crucial role. The cornerstone of stereo matching is the computation of the matching cost, a step that significantly influences the ultimate result. In this paper, an improved census cost computation method is proposed and integrated in the Semi-Global Matching(SGM), which incorporates two-bit string to encode the relationship between pixels and ternary operator combined with an adaptive threshold. In this way, the matching robustness at depth discontinuities can be effectively improved and is particularly suitable for the case of multiple craters on the lunar surface. Experimental results show that the proposed method can greatly improve the matching results at lunar craters.

A rational function model (RFM) has long been a versatile tool in image block adjustment when building the global control network in lunar topographic mapping. By unifying the physical imaging model with mathematical expressions, the RFM facilitates using multisource images from various missions. However, the scarcity of ground control points (GCPs) on the lunar surface and the imaging void of Lunar Reconnaissance Orbiter Camera (LROC) Narrow Angle Camera (NAC) limits the application of RFM in large-scale mapping tasks. This work proposed a time-based RFM generation approach for strip-based image block adjustment using merely sparse control points. The proposed time-based RFM integrates separated image pieces in the same orbit into a cohesive whole for strip-based block adjustment, requiring merely sparse control points when mapping and geopositioning all the image pieces. Besides, we employ the elevation control information extracted from Lunar Orbit Laser Altimeter (LOLA) data in the strip-based block adjustment. Experiment results indicate that under the same sparse control conditions, the elevation accuracy using strip-based adjustment is improved by an average of 12.8% compared with single-based adjustment, and our method achieves a similar adjustment accuracy using sparse elevation controls as that in the densely controlled single-based block adjustment.

Matching multimodal remote sensing images (MRSIs) is a challenging task. Due to significant nonlinear radiation differences (NRDs), traditional image-matching methods cannot achieve satisfactory results. This article shows that structural information can get more robust matching results compared with texture information (i.e., gradient features) from images. In order to better explore the structural information of images, this article proposes an MRSI matching method using structure saliency features, called weighted structure saliency feature (WSSF). Two strategies are investigated and integrated into WSSF to improve the matching performance. The scale space is constructed based on the pointwise shape-adaptive texture scale filtering, which can better retain the structure features, and the second-order Gaussian steerable filtering, edge confidence map, and phase features are combined to establish the structural saliency map combined with second-order Gaussian steerable filtering, which is much more robust to NRD than traditional gradient map. The performance of the proposed method was evaluated on a total of 120 image pairs from two MRSI datasets and compared with the state-of-the-art matching methods, including the histogram of the orientation of weighted phase (HOWP), locally normalized image feature transform (LNIFT), co-occurrence filter space matching (CoFSM), radiation-variation insensitive feature transform (RIFT), local phase sharpness orientation (LPSO), and position-scale-orientation scale-invariant feature transform (SIFT) (PSO-SIFT). The experimental results indicate that WSSF obtains satisfactory and reliable results in terms of success rate (SR) and matching accuracy. Compared with the above six methods, the matching accuracy of WSSF is improved by more than 20.275%, and the SR is improved by over 5.833%. The source code will be publicly available athttps://github.com/WGY-RS/WSSF.

3D point cloud semantic segmentation plays an essential role in fine-grained scene understanding from photogrammetry to autonomous driving. Although recent efforts have been made to push the 3D semantic segmentation forward, many solutions cannot generalize well to new data with different sensor configurations. For example, when transferring the segmentation model learned from terrestrial laser scanning (TLS) data to mobile laser scanning (MLS) data, the performance drops dramatically. Besides, rich-labeled data is usually required. However, labeling point cloud data is time-consuming and label-intensive in practice. In light of this, we propose SegTrans, an unsupervised domain adaption method for the point cloud semantic segmentation task, which largely improves the generalization performance from one labeled dataset (source domain) to another unlabeled dataset (target domain). Specifically, we first introduce a data selection module (DSM) to tackle the discrepancy between different datasets at the data level. Then an adversarial learning module (ALM) with an adversarial loss is iteratively implemented to align the domain-specific feature in both the source and target domains, which only consists of two fully connected layers. Experiments show the overall accuracy of the proposed method achieves 88% OA on the TUM City Campus dataset (MLS dataset) when trained on the Semantic3D dataset (TLS dataset).

Registration is a fundamental but critical task in point cloud processing, which usually depends on finding element correspondence from two point clouds. However, the finding of reliable correspondence relies on establishing a robust and discriminative description of elements and the correct matching of corresponding elements. In this letter, we develop a coarse-to-fine registration strategy, which utilizes rotation-invariant features in frequency domain and a new graph matching (GM) method for iteratively searching correspondence. In the GM method, the similarity of both nodes and edges in the Euclidean and feature space is formulated to construct the optimization function. The proposed strategy is evaluated using two benchmark datasets and compared with several state-of-the-art methods. Regarding the experimental results, our proposed method can achieve a fine registration with rotation errors of less than 0.2° and translation errors of less than 0.1 m.

We tackle the problem of place recognition from point cloud data and introduce a self-attention and orientation encoding network (SOE-Net) that fully explores the relationship between points and incorporates long-range context into point-wise local descriptors. Local information of each point from eight orientations is captured in a PointOE module, whereas long-range feature dependencies among local descriptors are captured with a self-attention unit. Moreover, we propose a novel loss function called Hard Positive Hard Negative quadruplet loss (HPHN quadruplet), that achieves better performance than the commonly used metric learning loss. Experiments on various benchmark datasets demonstrate superior performance of the proposed network over the current state-of-the-art approaches. Our code is released publicly at https://github.com/Yan-Xia/SOE-Net.

The high dimensionality of hyperspectral images often results in the degradation of clustering performance. Due to the powerful ability of potential feature extraction and nonlinear representation, deep clustering algorithms have become a hot topic in hyperspectral remote sensing. Different tasks often need different features. However, the current deep clustering algorithms generally separate feature extraction from clustering, which results in the extracted features that are not constrained by clustering tasks. Therefore, the features extracted by these algorithms may not be suitable for clustering. To address this issue, we adopt intraclass distance as a constraint condition and proposed an intraclass distance constrained deep clustering algorithm for hyperspectral images. The proposed algorithm propagates the clustering error back to the feature mapping process of the autoencoder network, so as to realize the constraint of clustering objective on feature extraction and make the extracted features more suitable for clustering tasks. In addition, the proposed algorithm simultaneously completes network optimization and clustering, which is more efficient. Experimental results demonstrate the intense competitiveness of the proposed algorithm in comparison with state-of-the-art clustering methods for hyperspectral images.

The semantic interpretation using point clouds, especially regarding light detection and ranging (LiDAR) point cloud classification, has attracted a growing interest in the fields of photogrammetry, remote sensing, and computer vision. In this letter, we aim at tackling a general and typical feature learning problem in 3-D point cloud classification- how to represent geometric features by structurally considering a point and its surroundings in a more effective and discriminative fashion? Recently, enormous efforts have been made to design the geometric features, yet it is less investigated to fully explore the potentials of the features. For that, there have been many filter-based studies proposed by selecting a subset of the whole feature space for better representing the local geometry structure. However, such a hard-threshold selection strategy inevitably suffers from information loss. In addition, the construction of the geometric features is relatively sensitive to the size of the neighborhood. To this end, we propose to extract multi-scaled feature representations and locally embed them into a low-dimensional and robust subspace where a more compact representation with the intrinsic structure preservation of the data is expected to be obtained, thereby further yielding a better classification performance. In our case, we apply a popular manifold learning approach, that is, locality-preserving projections, for the task of learning low-dimensional embedding. Experimental results conducted on one LiDAR point cloud data set provided by the 2018 IEEE Data Fusion Contest demonstrate the effectiveness of the proposed method in comparison with several commonly used state-of-the-art baselines.

Light Detection and Ranging (LiDAR) techniques is an efficient way of obtaining 3D information of complex urban scenes. However, automatically and efficiently interpreting acquired 3D points is still a challenging task. For achieving an excellent semantic interpretation of point clouds, the extraction of distinctive and reliable geometric features often plays a vital role. In this paper, we propose a method generating features from the local vicinity of different sizes and combine them for a better feature representation. To evaluate the proposed method, experiments were conducted using Li-DAR point cloud dataset and compared with that using single scale feature extraction methods.

We propose a workflow for extracting geometric primitives, including linear, planar, and cylindrical objects, from point clouds of the construction site, using a novel segmentation- and recognition-based strategy. The entire point cloud is first organized by an octree-based voxel structure. The proposed voxel- and graph-based segmentation is conducted by aggregating connected adjacent voxels in a fully connected local affinity graph, the weighted edges of which consider their saliencies simultaneously, including the spatial distance, the shape similarity, and the surface connectivity. After the segmentation, an improved efficient RANSAC algorithm is tailored to recognize and extract geometric primitives from segments. The synthetic, laser scanned, and photogrammetric point clouds are tested in our experiments, and qualitative and quantitative results reveal that our method can outperform the representative segmentation algorithms for our application having the precision and recall better than 0.77. It also shows a good performance with a correctness value better than 0.7 in primitive extraction.

In this letter, an improved phase correlation (PC) method based on 2-D plane fitting and the maximum kernel density estimator (MKDE) is proposed, which combines the idea of Stone's method and robust estimator MKDE. The proposed PC method first utilizes a vector filter to minimize the noise errors of the phase angle matrix and then unwraps the filtered phase angle matrix by the use of the minimum cost network flow unwrapping algorithm. Afterward, the unwrapped phase angle matrix is robustly fitted via MKDE, and the slope coefficients of the 2-D plane indicate the subpixel shifts between images. The experiments revealed that the improved method can effectively avoid the impact of outliers on the phase angle matrix during the plane fitting and is robust to aliasing and noise. The matching accuracy can reach 1/50th of a pixel using simulated data. The real image sequence tracking experiment was also undertaken to demonstrate the effectiveness of the proposed PC method with a registration accuracy of root-mean-square error better than 0.1 pixels.

Platform oscillation is a crucial error source that undermines the geometric performance of satellite imagery. In this paper, an approach for oscillation detection that utilizes the parallax observation between multispectral bands is proposed. Due to the parallax observation configuration of the multispectral sensors, the attitude oscillation of the ZY-3 satellite can be detected and estimated by the parallax disparities between adjacent band images. The parallax disparities between bands are obtained through a high-accuracy image matching method based on phase correlation at the subpixel level. The pixel displacements caused by the satellite oscillation are then retrieved from the parallax disparities by the use of two proposed transformation models. Experiments using both single-scene and long-strip images were conducted in order to retrieve the frequencies and amplitudes of the oscillation, as well as its changing trend. The experimental results for the ZY-3 satellite demonstrate the following findings: 1) the oscillation components obtained contain a distinct frequency of around 0.65 Hz; 2) the amplitude of the oscillation displacement on the image plane ranges from 0.5 to 1.5 pixels in the cross-track direction and from 0.2 to 0.6 pixels in the along-track direction, respectively; and 3) the oscillation frequency detected from the images is in agreement with that from the original attitude data.

Subpixel translation estimation using phase correlation is a fundamental task for numerous applications in the remote sensing community. The major drawback of the existing subpixel phase correlation methods lies in their sensitivity to corruption, including aliasing and noise, as well as the poor performance in the case of practical remote sensing data. This paper presents a novel subpixel phase correlation method using singular value decomposition (SVD) and the unified random sample consensus (RANSAC) algorithm. In the proposed method, SVD theoretically converts the translation estimation problem to one dimensions for simplicity and efficiency, and the unified RANSAC algorithm acts as a robust estimator for the line fitting, in this case for the high accuracy, stability, and robustness. The proposed method integrates the advantages of Hoge's method and the RANSAC algorithm and avoids the corresponding shortfalls of the original phase correlation method based only on SVD. A pixel-to-pixel dense matching scheme on the basis of the proposed method is also developed for practical image registration. Experiments with both simulated and real data were carried out to test the proposed method. In the simulated case, the comparative results estimated from the generated synthetic image pairs indicate that the proposed method outperforms the other existing methods in the presence of both aliasing and noise, in both accuracy and robustness. Moreover, the pixel locking effect that commonly occurs in subpixel matching was also investigated. The degree of pixel locking effect was found to be significantly weakened by the proposed method, as compared with the original Hoge's method. In the real data case, experiments using different bands of ZY-3 multispectral sensor-corrected images demonstrate the promising performance and feasibility of the proposed method, which is able to identify seams of the image stitching between sub-charge-coupled device units.