Optical and Synthetic Aperture Radar (SAR) matching produces spatial and semantic correspondences of the input images, playing a pivotal role in the registration process. However, due to the difference in radiation characteristics and geometric properties, even the same target may manifest distinctive morphological and feature expressions in the cross-modal images. Consistent feature extraction remains a challenge for optical-SAR image matching. Therefore, based on the salient image patterns (keypoint, line, and block), a Visual Mixture-of-Experts method for optical-SAR image Matching (MV-MOE) is proposed. It facilitates adaptive graphical representation of multi-modal images across various scenes through the multi-task learning framework. With the aid of the attention mechanism, the task-related basic features are reconstructed into matching-related features, yielding similarity along with spatial offset vectors. Additionally, we employ a multi-level feature extraction backbone based on the visual retentive block, enhancing local feature perception with the preservation capability of the recurrent network structure. Experiments demonstrate the advantages of the proposed method on multi-modal image matching and its contribution to the subsequent registration.

Keypoint detection aims to automatically locate the most significant and informative points in remote sensing images (RSIs), which directly affects the accuracy of matching and registration. In contrast to the handcrafted keypoint detectors that heavily rely on the morphological gradient of corner, line, and ridge, the learning-based detectors emphasize obtaining reliable keypoints from deep features. However, the limited accuracy of semantics undermines the reliability of keypoints, especially in challenging scenarios characterized by repeated textures and boundaries. Therefore, a novel trustworthy semantic keypoint (TSK) detector is proposed for RSIs. It utilizes a lightweight multiscale feature extraction and fusion network, along with a saliency keypoint localization mechanism, to facilitate keypoint detection. Notably, the TSK detector employed explicit semantics, which is refined with multiple learning strategies about repeatability and representability across the multigranularity reasoning spaces, namely, pixel window, neighbor window, and existence entity. Finally, several metrics about repeatability, matching, and registration are used to evaluate the performance of the TSK detector and other competitive methods. Four RSI datasets, including MICGE, HRSCD, OSCD, and SZTAKI, are used to verify performances. TSK detector achieves competitive performance against existing methods.

Realistic rendering results are available to be generated by volume-based neural rendering methods like NeRF. However, the existing schemes take the color vector as the unique supervision information, which leads to ambiguous prediction results of the volume density in the same spatial position through different rendering rays. This problem is common in large-scale scene reconstruction based on remote sensing images taken by UAVs and other equipment. To this end, we contrive to integrate spatial point cloud and multi-view image information, making the sparse point cloud the calibration for key features and regions. Therefore, the CloudLoc-NeRF is proposed. For volume information extraction, the multi-resolution hash coding and voxel are adopted to estimate the district for ray marching and extract volume features efficiently. For the point cloud, annular sampling and plane coding are used to combine image features of the training views and the point cloud. The regions with high feature response in multiple modal data should correspond to the regions with high volume density. In addition, an optimization method based on point cloud density is proposed. The weight parameter of volume density confidence is constructed to symbolize the correlation between density distribution and point cloud density. We verified the performance of our method on NVSF and the wide-area scene reconstruction dataset. Experiments showed that CloudLoc-NeRF accurately expresses the details of the rendered scene and produces better view synthesis results.

Ship detection in synthetic aperture radar (SAR) images is a basic and challenging task in marine monitoring. There has been made remarkable achievements in recent years. However, the existing detection methods still have the problem of ambiguous location information. It leads to the lack of model pertinence, and the poor discrimination of foreground and neighboring background. In this paper, we propose a ship detection method based on FSAF. In particular, we design a Gaussian Leading Block (GLB) capable to eliminate the interference of neighboring background information in the ground truth. Experiments on the HRSID dataset show that the proposed method achieves a 3.6% Average Precision (AP) improvement over the baseline at a low cost.

Few-shot object detection (FSOD) in remote sensing images, which aims to detect never-seen objects with few training samples, has attracted wide attention. Some recent works leverage meta-learning to tackle this challenging task and achieve promising performance. However, information attenuation during feature extraction and simple prototype representation hamper further improvement in detecting novel classes. In this article, we propose a novel meta-learning-based FSOD approach named DPL-Net. Specially, within the meta-learning-based framework, performing role-specific feature extraction in query and support branches, DPL-Net adopts a Fine-grained Information Fusion (FIF) module to capture scale-aware information within query regions of interest (RoIs) and a Multi-frequency Information Enhancement (MIE) module to retain the spectral information of support samples, respectively. Moreover, considering the variability of remote sensing objects, a Discriminative Prototype Learning (DPL) strategy is developed to rectify the ambiguous distribution of support samples for more representative class-aware prototypes. Experiments on two benchmark datasets (NWPU VHR-10 and DIOR) demonstrate that our method effectively improves the performance of meta-learning in detecting remote sensing images with limited training data.

Target fine-grained classification has been the research hotspot in remote sensing image interpretation, which has received general attention in application fields. One challenge of the fine-grained classification task is to learn the most discriminative feature using the deep convolutional neural network (DCNN). At present, many works of fine-grained image classification obtain target features by optimizing the feature extraction and enhancement, which are not accurate enough in remote sensing images. In this paper, we propose an essential feature mining network (EFM-net for short) based on DCNN to address this issue. Its major motivation is to obtain the essential feature which is fine enough to distinguish between similar instances. The proposed pipeline includes the Miner for purifying the essential feature and the Refiner for data augmentation. These two modules can work in a mutually reinforcing way, and extract the essential feature of targets. We evaluate EFM-Net on two public fine-grained classification datasets in remote sensing, FGSC-23 and FGSCR-42, and our Aircraft-16. The results show that the proposed method consistently outperforms existing alternatives. We have released our source code in Github https://github.com/JACYI/EFM-Net-Pytorch.git.

As the basics of 3D vision, pose estimation with 2D images is of significance in 3D reconstruction, UAV positioning, and other fields. However, the related works focus on the natural images and pay less attention to the wide-coverage UAV remote sensing (RS) images. In fact, the relationship between objects in UAV images can benefit pose estimation. Therefore, aiming at the outdoor scene captured by the UAV monocular camera, a novel pose estimation network that emphasizes the association between objects is proposed. The multi-scale visual features extracted by the convolutional neural network (CNN) are manipulated by the object-agnostic segmentation model to indicate the existing space of all possible objects in the whole scene. The features of all possible objects are embedded into vectors, and then processed with a graph convolution network (GCN) for relationship analysis. Based on the known sparse point cloud and the optimized features of 2D images, the camera pose is regressed iteratively by 3D visual geometry. To verify the feasibility of the network, experiments are conducted on the Extended CMU Seasons and the simulation UAV dataset. Results prove that our network emphasizes more features on the small objects and obtains superior pose estimation results.

In the field of remote sensing image object detection, the popular CNN-based methods need a large-scale and diverse dataset that is costly, and have limited generalization abili-ties for new categories. The few-shot object detection can be driven using only a few annotated samples. Existing few-shot detection methods are mainly designed for natural images, which ignore multi-scale objects and complex environments in remote sensing images. To tackle these challenges, we pro-pose a two-stage multi-scale method based on context mech-anism. Guided by the context-aware module, the multi-scale contextual information around the object is effectively extract and adaptively is combined into the ROI features to enhance the classification ability of the detector, which can reduce the classification confusion. Comparative experiments on public remote sensing image dataset RSOD show the effectiveness of our method.

Aircraft fine-grained recognition of remote sensing image is widely exploited in both military and civilian fields, which the similar physical structure and the changeable attitude between variable aircraft types makes this task challenge. Great progress has been made by proposal models based on convolutional neural network. However, previous works did not focus on local and multiple hierarchy features. In this paper, we propose a framework based on multiple hierarchy network and attention module for aircraft recognition. Compared to the existing methods, the extraction and enhancement of features proposed by us has been greatly improved. Remarkable results have been achieved on our dataset Aircraft-16 and the MTARSI dataset.

Multitemporal optical remote sensing image registration is still a challenging problem for current feature-based image registration algorithms due to the complex nonlinear discrepancies arising from diverse factors, including illumination, weather, and surface condition changes. To address the issue, this article attempts to combine the dual receptive field descriptors (DRFDs) constructed by a novel deep convolutional network. In addition, a novel inner loss function (ILF) that imposes constraints on the intermediate descriptors is adopted in order to consolidate the distinguishability of the descriptors when the overlapping areas of the input image patches are large. Subsequently, the dual feature distance maps (DFDMs) are built on the basis of the DRFDs and combined with features from accelerated segment test (FAST) key points for efficient and accurate correspondence establishment across the source image and the target image. Eventually, an iterative algorithm is proposed to remove the possible outliers. Experiments show that the combination of DRFDs trained with the ILF performs better than current learnable local descriptors, such as L2-Net, HardNet, and SOSNet. The image registration results using our method are more accurate than the methods based on learnable descriptors, such as L2-Net, HardNet, and SOSNet, and handcrafted descriptors, such as scale-invariant feature transform (SIFT), SURF, and ORB.

Multi-scale ship detection for remote sensing images is always a popular research field in civil and military application. In this paper, in order to solve the feature information single pathway flow in feature layer causes the lack of detailed information in the deep feature layer, we propose a bidirectional pathway feature pyramid networks (BP-FPN) method, which enables the deep feature layer to have strong semantic information as well as rich detailed information. At the same time, the Reverse Scale-transfer Layer down-sampling method is proposed to reduce the information loss of feature layer in the process of downsampling. It ensures that the feature layer maintain information during the down-sampling process. Experimental results on a dataset collected from Google Earth have quantitatively and qualitatively demonstrated the effectiveness of our approach

Image registration is an important technique that has been widely used in many areas. It is an indispensable premise for remote sensing image tasks like change detection and image fusion. In this paper, we propose a deep learning framework to generate descriptors for key points and then combine the descriptors constructed with FAST key points for accurate image registration. During the process of training, we adopt a novel loss function named Moat Loss (ML) to train our model, which is accordingly called MoatNet. Experiments show that our method is more robust than traditional algorithms like SIFT and is more accurate than the end-to-end deep learning methods in much more complex cases.

Aiming at the closed water detection in remote sensing imagery at medium-low resolution, this paper proposes a novel method for closed water detection based on fusion detection which conducts detection via informative fused images blended by Synthetic Aperture Radar (SAR) and optical images. Firstly, it utilizes SAR and optical image pairs containing the same closed water object to generate aligned image pairs according to latitude and longitude information. Next, generative adversarial network (GAN) is adopted to fuse two categories of images. At last, a target detection network driven by optical image samples is used to detect the closed water on the fused image. The experiment result on Sentinel-1&2 shows that the proposed method can effectively make up for the shortage of SAR image in closed water detection and improve the detection performance.

Accurate prow detection (i.e., ship heading prediction) is important in many applications that rely on optical remote sensing imagery, such as track forecasting and maritime navigation. In recent years, many advanced methods based on deep convolution neural networks (DCNNs) have succeeded in detecting multidirectional ships. However, these methods are not effective at determining the prow orientation, primarily due to three limitations: weak adaptability to geometric transformations of ship targets, confusing the semantic information between the prow and other parts of ships, and the boundary discontinuity problem. To address these problems, we propose an omnidirectional prow detection network (OPD-Net) based on feature enhancement and an improved regression model. OPD-Net consists of a feature refinement network (FRN), a prow attention network (PAN), and a complex plane coordinates regression model (CPCRM). First, the FRN balances the low-level location information and high-level semantic information from multiscale feature maps and then fits various geometric transformations regarding ship targets through deformable blocks. Next, the PAN, which is based on supervised learning, is used to enhance the ship prow feature as well as suppress background noise, which improves the accuracy of ship heading predictions. Finally, the CPCRM is designed to effectively solve the boundary discontinuity problem and correctly achieve prow detection in arbitrary orientations. Experiments on optical remote sensing image data sets demonstrate the robustness and superiority of our method for prow detection. Moreover, our approach is also competitive when used only for ship detection.

The filter based on tensor decomposition is an effective method to remove the noise of the interferometric phase stack data. The key is to choose the definition of tensor rank and an appropriate tensor decomposition model. The Bayesian CP Factorization (BCPF) -InSAR framework proposed in this paper definites the rank of InSAR tensor by CP rank and decomposes InSAR tensor into low rank tensor, noise tensor and outlier tensor. Compared with several widespread filters, BCPF-InSAR is proved as an effective InSAR tensor filtering method on the simulation data and real data.

Ship change detection is of great significance for maritime safety supervision, wharf vessel management, and vessel life cycle analysis. Nowadays, the ship change information is obtained through the difference between the multiple independent object detection results of multi-temporal images. However, it neglects the temporal correlation of the concerned features, impeding further improvement of the detection accuracy of change detection. Therefore, based on the sequential network, namely convolution LSTM, we built an end-to-end ship change detection (SCD) R-CNN network. The network extracts abstract semantic information reflecting the changed features of ships, and the time correlation between features of the different phases is established. Specifically, the changed features are utilized to guide the judgement of ship change. It is verified from the RS images that the proposed network avoids the misjudgment caused by the errors of object detection in the conventional method. In addition, a higher efficiency is revealed, maintaining the accuracy of the change detection of ship targets.

Arbitrary-oriented ship detection is one of the main applications of high-resolution remote sensing images. The current target direction detection methods, based on deep convolution neural network (DCNN), can estimate most of the ship directions (i.e. represented by angles). However, the performance of these networks is limited by the boundary discontinuity problem due to the angle-based regression model. In this paper, we propose an improved regression model based on coordinates in the complex plane to solve the boundary discontinuity problem. Experiments on real remote sensing dataset verify the effectiveness and robustness of our method.

Ship detection is of great significance in both military and civilian application domains. Deep Convolutional Neural Network (DCNN) method with region proposal, e.g. Faster R-CNN, achieves ship detection well. However, for multi-scale target detection in high-resolution remote sensing image, the limitation of accuracy is induced by the region proposal restricted by the training set. Therefore, the mechanism of multi-scale ship detection based on saliency estimation is proposed in our work. Firstly, a saliency estimation algorithm is used to distinguish which image contains large ships and the image pyramid for each input one is established. Then, using a target detection network in different scales of images. The results are merged at the end of network. Finally, accuracy and validity are verified by real data processing.

Interferometric stack data acquired by multi-pass synthetic aperture radar interferometry (InSAR) techniques is subject to additive outliers. Therefore, interferometric stack data denoising is an essential prerequisite for elevation inversion and terrain deformation monitoring. However, similar-patch-based denoising methods, e.g. NL-InSAR, neglect the temporal similarity between different interferograms, and tensor-based methods, e.g. Higher order Robust PCA (HoRPCA) and its evolutionary versions, cannot adequately use the repeated similar fringe patterns existing in one interferogram. In our work, the limitations of NL-InSAR, HoRPCA, Weighted HoRPCA are summarized, and then a novel denoising method is proposed based on the combination of nonlocal(NL)-means and HoRPCA, named as NL-HoRPCA. This method further improves the estimation accuracy of low rank InSAR tensor. Consequently, one InSAR tensor is used to prove the superiority of NL-HoRPCA in the case of dense fringes and high outlier ratio. The accuracy results prove the robustness of the proposed method.

Multi - frequency interferometric synthetic aperture radar (InSAR) has drawn more attention in reconstructing height profile. In this paper, a novel unwrapped phase estimation method based on split-spectrum strategy and maximum likelihood (ML) criterion is proposed for multi-frequency InSAR. Firstly, multiple range subband interferograms are generated with the split-spectrum strategy to form a multifrequency configuration. Then, unwrapped phase of the reference frequency is estimated via ML method modeled by the proposed normalized probability density function (pdf) which is able to remove the phase ambiguity. Finally, simulation experiments are carried out to evaluate algorithm performance. Smaller unwrapped phase error standard deviation verifies the effectiveness and exactness of the proposed method.

Due to the low correlation of SAR data in Yunnan mountainous area, special interferometric processing method is proposed in this paper. Initially, two SLC images are analyzed and co-registrated by using different correlation functions. The best registration result is selected through correlation evaluation. Then, the noisy phase is multi-looked and smoothed by four phase filtering algorithms. The most appropriate filtering method is chosen by evaluating the filtered phase. Subsequently, the Goldstein's branch cut method guided by residues grouping is employed to derive the unwrapped phase. Ultimately, both the gradient-jump point number and its distribution are utilized to evaluate the unwrapped results and identify errors in unwrapped phase.

The cycle-property of probability density function (pdf) of the interferometric phase is demonstrated in brief. The normalized baseline pdf is presented in terms of reference baseline to conquer the constant 2π-cycle. Accordingly, a novel unwrapped phase estimation method, based on a maximum likelihood (ML) principle technique, is imported for multi-baseline SAR interferometry. Simulations corroborate the validity of maximum likelihood unwrapped phase estimation method proposed in this work.

In this paper, a novel multi-baseline phase unwrapping approach is proposed based on maximum likelihood estimation (MLE) method. Topography phase gradient is acquired by fusing multi-baseline InSAR data. It is convenient to obtain unwrapped phase by using gradient integral. Since search interval is critical for the uniqueness and accuracy of solution, low-precision digital elevation model (DEM) is imported to improve the estimation interval. Simulations corroborate the validity of maximum likelihood unwrapped phase estimation method proposed in our work.

Traditional radar utilizes Shannon-Nyquist theorem for high bandwidth signal sampling, which induces the complicated system. Compressive sensing (CS) indicates that the compressible signal using a few measurements can be reconstructed by solving a convex optimization problem. Thus, the huge amount of data according to high Shannon-Nyquist rate is significantly reduced by compressive sensing. Parallel frequency radar theoretically cannot degrade the resolution compared with a traditional radar system and effectively reduces the sampling rate. In this paper, we focus on the data processing of the novel radar system. Basing on a sufficient structure, an algorithm of target scene reconstruction in pursuance of compressive sensing applied to the novel radar is proposed. Several simulations demonstrate the feasibility and the superiority of parallel frequency radar via compressive sensing.