Stereo matching in remote sensing has recently garnered increased attention, primarily focusing on supervised learning. However, datasets with ground truth generated by expensive airbone Lidar exhibit limited quantity and diversity, constraining the effectiveness of supervised networks. In contrast, unsupervised learning methods can leverage the increasing availability of very-high-resolution (VHR) remote sensing images, offering considerable potential in the realm of stereo matching. Motivated by this intuition, we propose a novel unsupervised stereo matching network for VHR remote sensing images. A light-weight module to bridge confidence with predicted error is introduced to refine the core model. Robust unsupervised losses are formulated to enhance network convergence. The experimental results on US3D and WHU-Stereo datasets demonstrate that the proposed network achieves superior accuracy compared to other unsupervised networks and exhibits better generalization capabilities than supervised models. Our code will be available at https://github.com/Elenairene/CBEM.

Image registration and change detection are two important tasks in synthetic aperture radar (SAR) image processing. Conventional researches considered them as two individual problems, where change detection requires pixel-wise registration first. However, they can be integrated into one processing framework and benefit from each other. This letter proposes a unified algorithm to address the change detection problem in multitemporal SAR images considering ambiguous co-registration errors. Based on the framework of optical flow, we designed an advanced objective function by introducing three individual features of heterogeneous, homogeneous, and changed areas in multitemporal SAR images. Afterward, the objective function is iteratively optimized, resulting in pixel-wise correspondences and a changed map. Experimental results on multitemporal SAR images show that the proposed algorithm achieves good performances both on registration accuracy and change detection discrimination.

Optical and synthetic aperture radar (SAR) image matching presents a formidable challenge due to their pronounced geometric and radiometric distinctions arising from multimodality. The distinct imaging mechanisms of optical and SAR sensors make it challenging to identify essentially homologous points in the physical sense, raising concerns about the accuracy and repeatability of correspondences in current feature matching methods. In this study, we introduce a detector-free feature matching algorithm specifically designed to match optical and SAR images through a two-step strategy. In the initial phase, our proposed method conducts pixelwise matching (PM) using downsampled feature descriptors, eliminating the necessity to identify repeatable keypoints. To mitigate complexity, we enforce a pseudo-epipolar constraint (PEC) to reduce computational costs by constraining the search range. Subsequently, refined matching is performed on the initial correspondences to rectify inaccuracies in the PM localization of the first step. Both matching steps are implemented on a graphics processing unit (GPU) to ensure high efficiency. The proposed algorithm attains an average matching accuracy of 2.39 pixels and operates with an efficiency of 1.09 s for 1108 image pairs, underscoring its superior comprehensive performance compared to various state-of-the-art algorithms, including handcrafted methods and deep learning networks.

The development of Earth Observation technology makes it possible for clearly understanding of our planet. Very high resolution (VHR) satellite remote sensing images with sub-meter level has been applied in many fields including land monitoring, urban construction and others. Traditional models focus on the improvement of the over-all accuracy of processed images, while local distortions leading pixel-level geolocation error has not attracted attention for VHR images. Hence, this paper focus on the calibration of local distortions between images with a coarse-to-fine framework. Benifited from the widespread of the rational function model (RFM), the affine transformation model was utilized for the overall accuracy improvement using sparse matching points. Then, dense matching relationships are filled to build a fine calibration model based on the error distribution between images. Images obtained from the Chinese Gaofen-2 (GF-2) are experimented. Results indicated that our proposed method can significantly improvement the geometric consistency between VHR images, providing a generic way for geometric process of VHR images.

On-orbit processing is becoming more prevalent due to its ability to efficiently exploit satellite resources. On-orbit geometric rectification improves positioning accuracy for follow-up tasks such as object detection or geometric calibration, while avoiding heavy burden on downlinking bandwidth and time delay. However, existing rectification methods faces some challenges. The hardware resources onboard satellites are restricted, and geographic positioning is often inaccurate. In this article, we propose a novel method designed for on-orbit rectification. The proposed method introduces a two-step registration framework to overcome large initial offsets and also a feature-compressing strategy to reduce the storage space of reference patches. Quantitative and practical experiments demonstrate that the proposed method performs well in terms of storage space, time efficiency as well as registration accuracy.

Multi-sensor remote sensing applications require the registration of optical and Synthetic Aperture Radar (SAR) images, which presents challenges due to significant radiometric and geometric differences resulting from distinct imaging mechanisms. Although various algorithms have been proposed, including hand-crafted features and deep learning networks, most of them focus on matching radiometric-invariant features while ignoring geometric differences. Furthermore, these algorithms often achieve promising results on datasets that use manually labeled ground truths that may be less reliable for high-resolution SAR images affected by speckle noise. To address these issues, we propose a robust global-to-local registration algorithm consisting of four modules: geocoding, global matching, local matching, and refinement. We generate a geometry-invariant mask in the geocoding module to help the local matching module focus on valid areas, introduce a fast global matching method to solve large offsets, and use matching confidence to guide subsequent local matching based on the accuracy of global matching. We propose a feature based on multi-directional anisotropic Gaussian derivatives (MAGD) and embed it into the confidence-aware local matching with the geometry-invariant mask to reduce the effect of geometric differences. Finally, we refine correspondence positions and remove outliers. We also build a high-accuracy evaluation dataset with hundreds of image pairs, where the ground truth is obtained by meta poles, which have clear and reliable structures in both optical and SAR images. Experimental results on this dataset demonstrate the superiority of our proposed algorithm compared to several state-of-the-art methods.

Optical satellite remote sensing has become an important means of large-scale targets detection. However, due to the small perspective of satellite remote sensing, most of the structural information of slender targets, such as power transmission towers, is compressed during the imaging process. Experiments have found that the existing network does not work well in this situation. In order to tackle this problem, we proposed a shadow information-based slender targets detection (SI-STD) method. First, the shadow of the tower is used to compensate for the structure information lost during imaging. Second, the candidate regions containing towers and shadows are proposed by a modified regional proposal network (MRPN), which also classifies these regions at the same time. Third, the tower targets are regressed from candidate regions by TowerHead. Finally, by fusing the results of MRPN and TowerHead, we get the final results. The framework can comprehensively use the tower and its shadow to complete the detection task, and it has also shown excellent performance in the test.

Land cover classification is the basic task of remote sensing image interpretation. Related methods have developed rapidly, especially the branch based on deep learning (DL). For high-resolution remote sensing images, the smaller inter-class difference and greater intra-class difference are two obstacles to improving the classification accuracy. For the former, the DL models generally use a deeper encoder to extract more powerful classification features. Considering that the scale of different land cover categories varies greatly, multi-scale feature extraction modules are also used to improve the classification accuracy. While the latter is always overlooked, and thus we propose a dual-output model, which uses a dense spatial pyramid pooling (DSPP) module to generate both the pixel-level and region-level predictions, to reduce the influence of intra-class differences. To further increase the classification accuracy, we investigate the band selection technique to apply the pre-trained encoder from the natural red green blue (RGB) dataset to multi-spectral remote sensing images. Extensive experiments on two datasets demonstrate the effectiveness of our model.

Radiometric normalization minimizes the radiometric inconsistencies between images in synthetic aperture radar (SAR) images mosaic. However, the radiometric principle of SAR have not been fully considered by the existing methods. To this issue, a radiometric principle-based radiometric normalization method is proposed. First, image areas with consistent coverage areas in object space and approximate rough ground surface between images are extracted as adjustment candidates based on imaging principle. Then, considering the radiometric characteristic of SAR image, all images are taken as a whole to solve the radiometric adjustment model, which transfers radiometric normalization into the least square optimization. Finally, a global quantization strategy is used to ensure radiometric consistency during orthorectification and mosaicking. The experimental results of Gaofen-3 SAR images demonstrate that the proposed method has the best performance, and can effectively and stably eliminate the radiometric differences between images.

Unlike geocoded images, slant-range (SR) synthetic aperture radar (SAR) images vary from imaging resolution to angles, which are difficult to be registered directly using the traditional SAR image registration methods. A possible way is to match their corresponding geocoded images and to project the correspondences to SR images. However, this way is time consuming and suffers from both registration and projection errors. In this letter, an automatic and efficient method is proposed to directly match multiview SR SAR images. We first estimate the scale and rotation differences between two SR images from the metadata delivered by vendors alongside the image file. Specifically, the scale differences of the range and azimuth directions are estimated by transforming the range and azimuth pixel intervals into a uniform geographical resolution, and the rotation differences are estimated by comparing the azimuth angles of an image-pair. A global-to-local framework is then implemented to accelerate the registration process. In the global stage, we fix the scale and rotation parameters in SAR-scale-invariant-feature-transform (SAR-SIFT) method to avoid mismatches. In the local stage, the phase correlation of cropped patches is parallelized to generate accurate matches. Experimental results on 13 multiview SAR images of the Omaha city show that the proposed method can provide accurate and efficient registration results for each pair of the 13 images, and outperforms the state-of-the-art methods both in accuracy and in efficiency.

Accurate optical and synthetic aperture radar (SAR) image registration is crucial to multisensor remote sensing applications. Though several algorithms have been proposed, the practical implementation that directly matches large and high-resolution optical and SAR images remains underexplored by the community. Equipped with the satellite positioning parameters, optical and SAR images can be roughly registered based on geographic coordinates. However, the relative positioning accuracy is still dozens, even hundreds of pixels due to the inaccuracies of sensor parameters and elevation. Consequently, we propose a robust registration algorithm, which consists of two stages, where the horizontal positioning errors can be reduced in the first stage, and then, we fine-tune the correspondences in the second stage. Specifically, we propose a novel template matching method based on the dilated convolutional feature (DCF) and epipolar-oriented phase correlation. DCF is constructed by a depthwise-separable dilated convolution with multichannel gradients, which are generated by the Sobel operator for optical images and a ratio of exponentially weighted averages (ROEWA) operator for SAR images. Due to the large reception field of dilated convolution, DCF can retain invariance even for large relative positioning errors. The epipolar-oriented rectangle template, which stretches along the epipolar line, is then proposed to capture more overlapping areas compared to square templates. Furthermore, the outlier removal is implemented in the coordinate system of optical images to avoid the effect of range compression in SAR images. Inliers are finally used to refine the rational polynomial coefficients (RPCs) based on the bundle adjustment technique. Experimental results on high-resolution optical and SAR image products of various scenarios demonstrate the effectiveness of the proposed registration framework. The relative positioning errors of the refined RPCs can be reduced from hundreds of pixels to the subpixel level.

Despite impressive progress in the past decade, accurate and efficient multiview synthetic aperture radar (SAR) image registration remains a challenging task due to complex imaging mechanisms and various imaging conditions. Especially, for rugged areas, SAR images obtained from the opposite-side view reflect different characteristics, making popular SAR image registration methods no longer applicable. To this end, we propose a geometry-aware image registration method by extracting inherent orientation features and concentrating on geometry-invariant areas. First, slant range images are terrain-corrected using a digital elevation model (DEM) to reduce large relative positioning errors caused by elevation. Second, the Gabor-ratio detector is introduced to obtain multiscale orientation features, which are more robust under various imaging conditions. Then, a geometry-aware mask is produced by intersecting the 3-D space ray with DEM, and thus, SAR images can be divided into three categories, layover, shadow, and geometry-invariant areas. The geometry-aware matching method, which focuses on geometry-invariant areas and masks out misleading caused by geometric and radiometric distortions, is proposed to realize accurate matching. The rational polynomial coefficients (RPCs) are refined to achieve relative correction. Extensive results on dozens of SAR images demonstrate the effectiveness and universality of the proposed algorithm by quantitative evaluation using man-made and natural corner reflectors. An analysis of the factors affecting registration accuracy is also discussed.

As the first C-band full-polarization spaceborne synthetic aperture radar (SAR) satellite in China, Gaofen-3 (GF-3) is of great significance in scientific research and economic development. Thanks to the advantage of all-weather and all-day observation, spaceborne SAR imagery is widely used in digital elevation model (DEM) generation, target tracking, and so on. And, all these applications are based on the high 3-D geolocation accuracy of SAR imagery. Therefore, a new slant range error update model and an error-source-based weight strategy which can effectively improve the 3-D geolocation accuracy of multiobserved data set from GF-3 satellite are proposed in this letter. The slant range error update model is proposed to improve the 3-D geolocation accuracy using the virtual control information in height direction. At the same time, an error-source-based weight strategy is introduced as an appendant to the multiobservation block adjustment model based on the rational function model (RFM). The experimental results demonstrate that our proposed method can achieve a much higher 3-D geolocation accuracy than the traditional methods.

Phase correlation (PC), an efficient frequency-domain registration method, has been extensively used in remote sensing images owing to its subpixel accuracy and robustness to image contrast, noise, and occlusions. However, its performance becomes poor when applied to the registration between optical and synthetic aperture radar (SAR) images, which are two typical multisensor images. Inspired by the recently proposed feature-based methods, we present a novel subpixel registration method that combines robust feature representations of optical and SAR images and the 3-D PC (OS-PC). The robust feature representations, which capture the inherent property of the two images and retain their structural information, form two dense image cubes. The 3-D PC utilizes the image cubes as a substitute of two raw images to estimate 2-D translations, either by locating peak in the spatial domain or by directly working in the Fourier domain. Furthermore, we investigate two techniques to improve the accuracy of the 3-D PC both in the spatial domain and Fourier domain: the first is the constrained energy minimization method to seek the Dirac delta function after 3-D inverse Fourier transform and the second is the fast sample consensus fitting to estimate phase difference after high-order singular value decomposition of the PC matrix. Experiments with both simulated and satellite optical-to-SAR pairs were carried out to test the proposed method. Compared with state-of-the-art PC methods and optical-to-SAR registration methods, the proposed method presents a superior performance in both accuracy and robustness. Moreover, we verify the adaptability of the proposed method.

Geo-positioning accuracy improvement is one of the most important step of remote sensing image preprocessing. Traditional methods require a large number of ground control points(GCPs) which consuming lots of manpower and financial resources. With the resolution up to 0.8m, the original geo-positioning accuracy of the Chinese Gaofen(GF-2) multi-angle imagery is about 90m which means a limited application in geometric processing. In this paper, we propose a new method to improve the geometric performance of the multi-angle satellite imagery based on the geometric error sources of this experimental dataset without GCPs. Under the condition of weak intersection of our test dataset, we use a DEM-assisted approach to acquire a more accurate initial position accuracy of all tie points, and all extracted data is clustered by the Density based spatial clustering of applications with noise(DBSCAN) algorithm in order to eliminate points or impages with large positioning error automatically. Then, the error-based block adjustment model are proposed and investigated to improved the geometric performance of the experimental dataset. Based on our proposed method, 142 multi-angle GF-2 satellite images covering the western Beijing area are experimented and the root mean square error(RMSE) of the geometric accuracy is improved up to about 12m in plane and 6m in height, which shows a significantly improvement in geo-positioning accuracy of these multi-angle GF-2 remote sensing imagery.

The GF-3 satellite is the first full-polarization SAR satellite in China, which operates in the C band with a resolution of 1m. This paper presents a dual-aspect geometric terrain correction method to overcome the inherent shortages of SAR image such as foreshortening, shadow and layover. The results using GF-3 satellite data show that the method can effectively eliminate layover and shadow distortions in SAR images. This method solves the geometric correction problem that cannot be solved with a single SAR image．

The automatic geometric rectification of space-borne SAR remote sensing images is generally achieved by automatic matching with the optical control base map. However, the optical map has the problem that its coverage is limited and the data cannot be updated in time. With the continuous development of digital maps, the coverage and accuracy of vector maps continue to increase. In order to expand the source of data for correction processing of SAR remote sensing images, an automatic SAR image correction method based on vector map is proposed. Application experiments show that the geometric positioning accuracy of SAR images can be improved significantly based on vector map.

Microsatellites, which have become research hotspots, can obtain remote sensing video data of the target area through gaze imaging mode. However, at this stage, there is no dedicated video playback software and solution suitable for remote sensing video data display. Conventionally, remote sensing video data can only be played through ordinary video playback software. This could not fully display the rich information contained in remote sensing video data. In view of the above problems, this paper analyzes and designs a software solutions suitable for remote sensing video data playback. The proposed software solution can better display the geographical related information contained in the remote sensing video and describe the target dynamic information in the video.

Owing to the property of being constant to image contrast and the identification of various types of features, phase congruency (PC) model has been widely used in remote sensing applications. However, when the PC is directly applied to optical and synthetic aperture radar (SAR) image registration, it fails to handle large radiometric and geometric differences. In this paper, we propose an automatic algorithm to solve this problem. First, evenly-distributed keypoints are extracted from the optical images via the block harris method. Complementary grid points are selected in image regions with poor structure and texture information. Then a robust similarity metric based on the improved PC model is proposed. Since the two images show diverse properties, we utilize two different PC models, the traditional PC and the SAR-PC. The PC values of several directions are aggregated to construct the feature descriptors on the basis of which, as a result, a similarity metric using the normalized correlation coefficient (NCC) is obtained. We compare the proposed metric with two baselines (mutual information and NCC) and a state-of-the-art method (histogram of the oriented phase congruency, HOPC) in the case of various scenarios, the results show that our method outperforms the baselines and show comparable performance with HOPC in regions with abundant structure information and better performance in untextured regions.

Coregistration of high-resolution optical and synthetic aperture radar (SAR) images is still an ongoing problem due to different imaging mechanisms of two kinds of remote sensing images. In this paper, we propose an optical flow-based algorithm to solve the dense registration problem [optical-to-SAR (OS)-flow]. Unlike parametric registration methods that estimate a transformation model, OS-flow aims to find pixelwise correspondences between optical and SAR images. Specifically, two frameworks of OS-flow, a global method and a local method, are proposed. Due to the drastic differences between SAR and optical images, two dense feature descriptors, rather than the raw intensities, are utilized to retain the constancy assumption in optical flow estimation. Considering the inherent properties of the two images, two dense descriptors are constructed using consistent gradient computation. After satisfying the constancy assumption, the global method estimates the flow map by optimizing an objective function, and the local method iteratively estimates the flow vector in a local neighborhood. Both methods use the coarse-to-fine matching strategy to address large displacements and reduce the computational cost. Experiments on several optical-to-SAR image pairs in various scenarios show that the proposed methods have a strong ability to match across optical and SAR images and outperform other state-of-the-art methods in terms of registration accuracy.

China is a flood disaster-prone country, floods occur almost every year, especially in July and August. Rapid detection and assessment for floods affected areas are of great significance. The Chinese GF-3 SAR satellite, which uses active ground observation technology, has obvious advantages in flood disaster monitoring owing to its all-day, all-weather imaging characteristics. For the purpose of rapid water detection in flooding area, an automatic detection method of flood area based on GF-3 single-polarization SAR data is proposed. The proposed method consists of image preprocessing and water extraction. The experimental results show that the proposed method can realize rapid and accurate extraction of waters in flood disaster area.

The conventional shape similarity measurements of remote sensing data face problems in the situation of noise interference, partial information occlusion and missing. A method of shape similarity measurement based on principal curvature enhancement distance transformation is proposed. The distance transformation is carried out to extend the range of the shape contour, improving the robustness of the similarity measure. Besides, to ensure the accuracy of measurement results, the distance map is enhanced by the principal curvature of the shape contour, improving the response of contours with rich information. Application experiments of road vectors with GPS data and optical remote sensing images show that the method is effective in practical application.

As an indispensable part of remote sensing image preprocessing, geometric correction has a great influence on the application of remote sensing images. Traditional geometric correction requires lots of well-distributed ground control points(GCPs). However, GCPs are difficult to obtain GCPs in some places such as mountain and desert areas. For this reason, we propose a Multi-Observation Block Adjustment(MOBA) method to get accurate geo-position without control points. The method improves the geometric positioning accuracy by the error compensation parameters of the remote sensing images, which caculated from block adjustment error equations. Different from tranditional method, error equations are built depending on multi-projection in image space of a single object. In this way, error equation can converge to more accurate solution. Experiments of ZY-3 with wide baseline and GF-2 with narrow baseline show that the proposed method can significantly improve geometric positioning accuracy of both kinds of remote sensing images.

Although the scale-invariant feature transform (SIFT) algorithm has been successfully applied to both optical image registration and synthetic aperture radar (SAR) image registration, SIFT-like algorithms have failed to register high-resolution (HR) optical and SAR images due to large geometric differences and intensity differences. In this paper, to perform optical-to-SAR (OS) image registration, we proposed an advanced SIFT-like algorithm (OS-SIFT) that consists of three main modules: keypoint detection in two Harris scale spaces, orientation assignment and descriptor extraction, and keypoint matching. Considering the inherent properties of SAR images and optical images, the multiscale ratio of exponentially weighted averages and multiscale Sobel operators are used to calculate consistent gradients for the SAR images and optical images on the basis of which, as a result, two Harris scale spaces can be constructed. Keypoints are detected by finding the local maxima in the scale space followed by a localization refinement method based on the spatial relationship of the keypoints. Moreover, gradient location orientation histogram-like descriptors are extracted using multiple image patches to increase the distinctiveness. The experimental results on simulated images and several HR satellite images show that the proposed OS-SIFT algorithm gives a robust registration result for optical-to-SAR images and outperforms other state-of-the-art algorithms in terms of registration accuracy.

The ratio of averages is a robust edge detector which provides the property of constant false alarm rate for synthetic aperture radar (SAR) imagery. However, the rectangular window used in the calculation of local mean may cause numerous false maxima. The size of the processing window also has a significant effect on the detection performance, but it is difficult to determine the optimum window size. In this letter, we first propose a new ratio-based detector that is constructed by the Gabor odd filter. The scale of the proposed detector is related to the size of the processing window. Then, edge strength maps extracted by multiscale detectors are combined using an edge tracking algorithm to form a final response. We used the receiver operating characteristic curves to evaluate the performance of the proposed detector. The experimental results on simulated and real-world SAR images show that the proposed multiscale edge detector yields an accurate and consecutive edge response.

In this letter, we propose an adapted anisotropic Gaussian scale-invariant feature transform (AAG-SIFT) method to find feature matches for synthetic aperture radar (SAR) image registration. First, features are detected and described in an AAG scale space. The scale space is built adaptively to local structures. Noises are blurred, but details and edges remain unaffected in this scale space. Compared with traditional SIFT-based matching methods, features extracted by AAG-SIFT are more stable and precise. Then, the dominant orientation consistency (DOC) property is analyzed and adopted to improve the matching stability. The correct matching rate is significantly increased by DOC matching. Experiments on various SAR images demonstrate the applicability of AAG-SIFT to find stable and precise feature matches for SAR registration.