Multimodal image-matching success rates (SRs) are often low due to nonlinear radiation differences. Furthermore, when geometric transformations such as rotation and resolution exist between images, the matching SR between multimodal images decreases even further. (It is worth noting that experiments have shown the impact of scale to be relatively small; therefore, this discussion focuses only on the influence of resolution differences on multimodal image matching.) To tackle these challenges, we have enhanced the feature point extraction, description, and association processes in image matching. This has resulted in a robust multimodal image-matching framework that is invariant to rotation and resolution, with a high SR. Specifically, inspired by the concept of image pyramids, we designed a strategy for extracting feature points in multiple resolution dimensions. This enables assigning resolution dimension information to feature points and expanding the set of points to be matched. Building upon feature point extraction, we enhanced the Log-Gabor filter and designed a novel feature descriptor. This descriptor can work robustly in scenarios with modal differences and rotational variances ranging from 0° to 360°. Applying this descriptor to the matching framework helps to eliminate the influence of angle differences on feature point associations between images. Furthermore, to further improve the matching SR, we adopted a resolution dimension traversal retrieval strategy for the association of feature points. Based on this strategy, the number of correct matches (NCM) can be increased under the condition of the same feature points, thereby increasing the inlier rate of the matching results and enhancing the SR of the matching results. To evaluate the performance of this matching framework, we created a testing dataset containing 42496 pairs of images using publicly available datasets. These images cover six categories including optical, synthetic aperture radar (SAR), digital elevation model (DEM), infrared, map, and nighttime light, with three types of transformations between images: translation, rotation, and scaling. We conducted comparative experiments using the multimodal image rotation and resolution invariant feature transformation (MIRRIFT) method against five advanced multimodal feature matching methods with publicly available source code, namely, radiation-invariant feature transform (RIFT), locally normalized image feature transform (LNIFT), histogram of absolute phase consistency gradients (HAPCG), histogram of the orientation of weighted phase (HOWP), and Log-Gabor histogram descriptor (LGHD). The results demonstrate that the MIRRIFT method proposed in this article exhibits robustness to rotational, resolution, and modal differences in images. Specifically, it achieved an average SR improvement of 59%, an increase in average correct matching points by 12%, and an average matching accuracy of 1.97. The executable program and sample data will be made available at:https://github.com/Geng-Zemin/MIRRIFT

The goal of image stitching is to generate high-quality panoramic images with minimal computational cost. However, variations in viewpoint or scene depth can cause parallax effects in UAV images, complicating precise alignment and leading to artifacts such as ghosting, blurring, and misalignment. While advanced seamline detection algorithms reduce ghosting and blurring, structural distortions and misalignments near the seamline often remain, negatively affecting stitching quality. Moreover, these algorithms typically face challenges in balancing computational efficiency with alignment accuracy. In this paper, we propose a robust and flexible UAV image stitching method based on global optimal seamline detection and local alignment with seamline constraint. Our approach ensures precise alignment while maintaining processing efficiency. First, a global transformation-based alignment algorithm is used to pre-align the images to a common coordinate system. Then, an efficient weighted fast sweeping (WFS) algorithm is proposed to detect the globally optimal seamline, minimizing artifacts in overlapping regions caused by alignment errors and dynamic objects. Finally, an optical flow-guided local alignment method with seamline constraint is developed to correct residual misalignments along the seamline, reducing global structural distortion. Extensive experiments on a range of challenging datasets demonstrate that the proposed method outperforms existing approaches, producing more natural-looking stitching results.

Accurate multi-source remote sensing image matching is the foundation of many applications. However, the complex geometric and nonlinear intensity distortion between remote sensing images makes it challenging. This work selects 11 representative ones: OS-SIFT, PSO-SIFT, LGHD, LINIFT, RIFT, NISR, POS-GIFT, RedFeat, MatchFormer, and SemLA, and compares their performance on various remote sensing images. At last, we summarize the work by giving the research bottleneck and potential research directions toward high-precision multisource remote image matching.

To increase the observation performance of optical remote sensing satellites (ORSSs), onboard camera systems usually perform multislice and multiband imaging. The resulting multiband or multislice subimages require correction into an aligned and stitched complete image, to provide users with standard image products. In a linear array sensor, multiband and multislice sensor correction processing is achieved by mapping subimages on the same virtual linear array based on the determined geometric parameters and the principle of object space consistency. However, a planar array sensor has the same geometric parameters outside the camera; therefore, object space consistency is essentially equivalent to image space consistency, which simplifies coordinate mapping calculations. In this study, we developed a novel sensor correction method based on image space consistency for planar array sensors and combined it with a method to determine the relative geometric parameters between bands. The internal geometric parameters of multiple bands were determined under connection constraints between bands established by consistent image space pointing. Next, the coordinate mapping model between the virtual image point and physical subimage point was established, and the angular resolution was introduced for coordinate solving. Based on strict geometric correspondence between the subimages and virtual images, the whole image with interband registration and interslice stitching was generated from the subimages. We experimentally verified the accuracy and effectiveness of our method using real multiband and simulated multislice plane array sensor images from GF-4 satellites. The corrected whole image showed satisfactory interband registration accuracy and internal geometric accuracy.

Relative radiometric calibration (RRC) is a crucial step in enhancing the quality of satellite images, and it serves as a fundamental technology to ensure the reliability of information extracted from these images. Traditional side-slither RRC methods typically use homogeneous scenes as imaging sites to ensure approximately identical input for detectors of multiple non-collinear time-delay integration-charge-coupled devices (TDI-CCDs). However, as spatial resolution improves, the demand for site uniformity is increasing, limiting the application of these traditional methods. Therefore, we propose a push-broom data-aided side-slither RRC method, which involves sequential calibration of a single TDI-CCD and an entire field of view (FOV). The first step is designed to eliminate response differences within each TDI-CCD, while the latter incorporates push-broom data to determine the adjacent radiometric relationship via adjustment with maximum standard deviation preservation (MSDP). Experiments show that our method has achieved better results in both visual effect and quantitative assessment, compared with the other two advanced RRC methods.

The accurate matching of multisource, multi-temporal remote sensing images is challenging because of significant nonlinear intensity differences (NIDs) and severe geometric distortions. To address these problems, we developed a robust image matching method: oriented filter-based matching (OFM). OFM is insensitive to NIDs, while exhibiting scale and rotational invariance. First, salient feature points with multiscale attributes were detected in the Gaussian-scale space of the input images. Then, the images were convoluted using multi-oriented filters, and unified feature maps were constructed by the extraction of orientation indices using effective data pooling operations. The constructed feature maps were highly resistant to NIDs. Five filters were integrated into the OFM framework to investigate their applicabilities in different application scenarios. Next, a novel rotation-invariant feature descriptor was constructed, using a dominant direction determination approach and a descriptor-grouping strategy. The dominant direction determination approach enables accurate dominant direction estimation, whereas the descriptor-grouping strategy improves the stability of the method under different rotational angles. Finally, brute-force matching was implemented to obtain initial matches; an improved mismatch elimination method was used to identify reliable putative matches. To evaluate the performance of OFM, we created a large dataset comprising 4,427 pairs of multitemporal optical–optical, optical–synthetic aperture radar (SAR), optical–infrared, and optical–depth images. OFM outperformed state-of-the-art methods in terms of number of correct matches, recall, inlier ratio, root mean square error and success rate. Our implement is publicly available1.

Cloud detection plays an important role in remote sensing image quality evaluation, information acquisition, and analysis. For most optical satellites, the acquired multispectral bands include only three visible bands and one near-infrared band. How to achieve fast and accurate cloud detection with this limited number of bands is a problem worth studying. In this paper, a cloud detection algorithm (SMFCD) based on a saliency-based mixed feature map of images is proposed. This algorithm first calculates the saliency characteristics from the band mean and haze optimized transformation (HOT) map. The transmission map obtained from dark channel prior theory is used to combine the mixed feature map for subsequent detection. The proposed algorithm generates the initial thick cloud mask using a segmentation method based on the Otsu algorithm later. Guided filtering is then used to refine the cloud mask. The final cloud detection result is acquired through post processing. The algorithm is tested on six different datasets. It is shown that the algorithm can obtain good detection results for most images, with overall accuracies from 0.860 to 0.969 on these datasets. The average time consumption on these datasets reaches 4.4~42.1 seconds, and a fast version of the proposed algorithm reduces these times by about 50%. The proposed algorithm can be used for remote sensing image quality evaluation and subsequent application preprocessing with greater adaptability and flexibility than existing algorithms, since it does not depend on specific satellite radiometric calibration coefficients.

On-orbit geometric calibration (GC) is always performed to compensate for geometric distortion from the satellite’s camera. However, the traditional GC method is complex and difficult to apply broadly due to its reliance on a rigorous physical model (RPM), which involves not only complex processing of attitude, orbit, and time, but also the transformation among multiple coordinate systems. Additionally, the RPM is closely related to designs of satellites and cameras, which increases the complexity of the GC, and thus reduces generalizability. This paper proposes a practical and robust GC method to prevent camera distortion based on a standardized rational polynomial coefficient (RPC) model. Through a series of innovations, including stepwise optimization, a priori gross error elimination, the adjustment model with angular resolution, and the correction for the bias field-of-view (FOV) distortion, we were able to achieve robust GC for camera distortion, as well as accurate splicing and registration among segmented images. Method validation using data from the linear-array camera of the ZiYuan3-02 satellite, and the area-array camera of the GaoFen-4 satellite, produced satisfactory results, indicating that our method effectively compensates for systematic geometric distortion such that consistent GC and accuracy comparable with that of traditional RPM-based methods can be obtained.

The corrections in both[1]and[2]are similar. The corrections are as follows.

Independent methods for geometric calibration (GC) have become an important research direction in the field of optical satellite technology. The main purpose of this research is to eliminate dependence on ground calibration sites using relative constraints between images. Based on a systematic analysis of these relative constraints, we found that it was difficult, if not impossible, to completely eliminate ground constraints, although the number of ground control points (GCPs) required can be greatly reduced. To achieve practical GC with high accuracy and low cost, we proposed a new method to compensate for systematic errors in linear optical satellite data acquisition using only the relative constraints between two overlapped images, namely, the corresponding elevation constraints and sparse GCPs. We first demonstrated the feasibility of GC with relative constraints and established an optimized GC model suitable for these relative constraints. We then presented a global iterative method to eliminate inaccuracies in internal calibration caused by the different distributions of GCPs within two images. The nadir (NAD) linear camera on board the Zi-Yuan 3 (ZY-3) satellite was used to evaluate the feasibility of the presented GC method; the results indicated that the present method effectively compensated for systematic errors. Thus, this article demonstrated the feasibility of GC without calibration sites.

Geometric calibration (GC) is a technique to compensate for the systematic errors in the imaging model of an optical satellite. The independent GC method without use of ground calibration site has been studied in recent years. In this paper, an integrated independent method aiming to the GC of stereo cameras aboard an optical satellite is presented. Supported by the satellite's stereo imaging ability, this method can overcome the strong correlation between the elevation error and camera GC accuracy, and get rid of the dependency on the constraints of ground elevation further. The real data collected by the stereo three-linear camera (TLC) of ZY-3 satellite was used to verify this method. The satisfactory GC accuracy better than 1 pixel indicated that the presented method effectively compensated for systematic errors, and improved the geometric quality of TLC images together.

GaoFen1 is the first high-resolution earth observation satellite built in China, and carries four wide field-of-view (WFV) cameras to achieve large-scale monitoring and mapping. However, the unstable attitude measurement accuracy of the satellite generally imparts low geopositioning accuracy and inconsistent geometric error in overlapping areas of WFV images. A feasible and effective large-scale planar block adjustment (PBA) method is presented that corrects the geometric errors of the vast WFV images integrally, further improving the geometric accuracy of these images. In addition, whether ground control points (GCPs) are needed, and the effect of different numbers of GCPs on PBA accuracy is also investigated. Two key technologies are used in this paper. First, a universal PBA error equation based on the virtual control points is presented to allow PBA with or without GCPs. Second, an adjustment method aided by a digital elevation model is adopted to overcome the weak convergence geometry among WFV images, further ensuring stable estimation of PBA. The effectiveness of the proposed method was verified by 664 WFV images covering most of mainland China. The satisfactory experimental results indicate that the method presented herein is reasonable and effective, but that a certain number of GCPs is needed to ensure the accuracy of large-scale PBA results for WFV images.

Patch imaging is an important capability of GaoFen4, which is the first Chinese high-resolution planar array satellite in geosynchronous orbit. When the satellite collects images in the patch-imaging mode, overlapping images can be successively obtained. As these images are captured at different times and from a very high orbit, the initial geometric accuracy of the overlapping area between images is inconsistent, which directly affects image mosaicking. In this paper, a novel bundle block adjustment method based on the rational function model without ground control points (GCPs) is proposed to eliminate the relative geometric error among the images and to generate refined rational polynomial coefficients (RPCs). In this method, the average elevation is used instead of the true elevation of the ground points to solve the problem of weak convergence of corresponding bundles, which would result in unstable calculation in the height direction. Virtual control points (VCPs) generated from the original RPCs are used to restrain the freedom of the entire block in the horizontal direction, thereby ensuring stable calculation without the use of GCPs. A successive RPC regeneration method based on VCPs is also presented. To verify the effectiveness of the proposed method, four experiments were performed using real data to assess the geometric accuracy of the method, and the satisfactory experimental results indicate that the presented method is both practical and effective.

Due to the limitations in the geometric accuracy of reference data and matching accuracy between images from different sensors, the conventional method of calibration may not perform well in terms of accuracy for an agile optical satellite (AOS) with submeter resolution. Due to the high mobility of an AOS, a cross-image pair (CIP) including a push-broom image and a swing scanning image for the same area could be collected. This letter developed a novel calibration method for an AOS, which is characterized using the constraints from the CIP instead of reference images covering a calibration site. In this method, the viewing angle of a charge-coupled device detector fit by two polynomials is introduced into a rigorous imaging model to establish the calibration model, and a bundle adjustment under the constraint of an aided digital surface model (DSM) is applied to solve the unstable calculation stemming from the strong correlation between the parameters of the coupled images. To verify the effectiveness of the method, experiments were conducted using the CIP simulated according to the rigorous imaging process of an AOS. In tests, the method achieved a high theoretical accuracy of better than 0.1 pixels.