The remote sensing image scene classification continues to face significant challenges due to high intraclass diversity and interclass similarity. Existing methods mainly use semantic associations between images to establish deep semantic associations between classes, ignoring the rich high-level semantic knowledge contained in the label text. This high-level information are especially valuable for distinguishing between confusing categories, as it enables the model to capture both similarity and distinctive features effectively. In this article, we introduce a novel approach that incorporates label semantic information and proposes a plug-and-play framework to guide classification model learning of intraclass and interclass relationships. Specifically, our framework includes a dynamic soft label module (DSLM), which uses textual semantics to facilitate classification model learning of interclass relationships via soft labels at the target level. In addition, we design a coarse-to-fine contrastive module (CFCM) to integrate textual semantics into contrastive learning, guiding the model in capturing intraclass and interclass relationships at the feature level. Our framework is compatible with both convolutional neural network (CNN)-based and vision transformer (ViT)-based classification architectures and is employed solely during training to minimize computational overhead. Experimental results on four datasets validate the effectiveness of our approach.

We study the problem of exacting accurate correspondence pairs for point cloud registration. The existing correspondence methods focus on constructing point descriptors and then extracting correspondence point pairs. However, this process encounters two main issues: 1) point features are unstable and susceptible to noise, leading to a low inlier ratio (IR) for correspondence pairs and 2) the positional deviation of correspondence point pair results in accuracy errors when computing rigid transformations. To address these issues, we propose a robust point cloud registration framework based on patch matching, achieving high positional accuracy and high inlier-rate prediction of correspondence pairs. Specifically, we design a dual-branch point cloud registration network, with one branch dedicated to patch matching and the other branch to predicting the patch anchor, i.e., the coordinates used for patch matching. For patch matching, we integrate the topology of patches into the attention mechanism and adopt a multilevel patch-matching strategy to enhance the matching success rate. For coordinate prediction, we introduce graph convolutional network (GCN) and cross-attention mechanisms to explore local similar points through information interaction and feature correlation of patch pairs. Thanks to the stability of patch descriptors, our method demonstrates higher robustness compared to existing correspondence methods. Extensive experiments conducted on indoor, outdoor, synthetic, and deformable benchmarks validate the superiority of our method. Additionally, our method achieves certain effectiveness in cross-source point clouds.

Remote sensing scene classification is an important yet challenging task of remote sensing image interpretation. In recent years, the development of convolutional neural networks (CNNs) has significantly improved the accuracy of this study. However, most of the methods based on CNNs use discrete label representation, which ignores the potential correlations between scenes, resulting in insufficient generalization ability of the model. This paper proposes a model based on inter-class association information mining, named Correlation Label Embedding Network (CLENet). Specifically, we mine inter-class association information from the network’s predictions and continuously adjust them by back-propagation during the network training process to obtain continuous label representations. Unlike other methods, we distill the potential correlations between scenes as the supervisory signal of the network to guide the feature selection process of the network. This forces the output of the network to learn more scene-related features to adapt to complex application scenarios and improve the generalization performance of the model. Additionally, to enhance the discriminative nature of the model, we design regular metric terms based on the learned interclass association information. Compared with the state-of-the-art scene classification methods, the experimental results validate the potential of CLENet models on remote sensing image scene classification.

Vehicle detection in infrared aerial images is vital for both military and civilian applications, as infrared imaging remains effective under low-light conditions and various adverse weather scenarios. However, the longer wavelengths of long-wave infrared, compared to visible light, make diffraction more noticeable, leading to low-frequency degradation of vehicle information. Thermal radiation from the environment and optical system leads to higher noise in infrared images. Additionally, atmospheric transport models for various weather conditions can degrade infrared images to different extents. These factors lead to a reduced signal-to-noise ratio, which complicates the extraction of clear features. To overcome these challenges, we propose the Dual-Tuning Network (DTNet), an advanced framework for vehicle detection in infrared aerial images, developed based on the mechanisms of infrared imaging. Specifically, the core component of DTNet is the Dual-Tuning Block (DTBlock), which works alongside the Dynamic Guided Filtering Module (DGFM) and the Point Spread Recovery Module (PSRM) for feature extraction. DTBlock decomposes feature maps into low- and high-frequency components with learnable low-pass filters. DGFM eliminates disturbance from the optical system and background thermal radiation in the high-frequency component of feature maps, while preserving the details and texture of vehicles. PSRM aggregates vehicle features in the low-frequency component of feature maps, which is proposed with reference to diffraction and atmospheric models. The concept of Dual-Tuning refers to enhancing the signal and suppressing interference in the low and high frequency parts, respectively. Experimental results on the DroneVehicle public dataset for infrared vehicle detection indicate that our proposed approach achieves state-of-the-art (SOTA) performance. Moreover, extensive ablation studies confirm the superior capability of our DTNet in robust feature extraction from infrared images.

Point clouds obtained from 3D scanners are often noisy and cannot be directly used for subsequent high-level tasks. In this article, we propose a novel point cloud optimization method capable of denoising and homogenizing point clouds. Our idea is based on the assumption that the noise is generally much smaller than the effective signal. We perform noise perturbation on the noisy point cloud to get a new noisy point cloud, called self-variation point cloud. The noisy point cloud and self-variation point cloud have different noise distribution, but the same point cloud distribution. We compute the potential commonality between two noisy point clouds to obtain a clean point cloud. To implement our idea, we propose a Self-Variation Capture Network (SVCNet). We perturb the point cloud features in the latent space to obtain self-variation feature vectors, and capture the commonality between two noisy feature vectors through the feature aggregation and averaging. In addition, an edge constraint module is introduced to suppress low-pass effects during denoising. Our denoising method does not take into account the noise characteristics, and can filter the drift noise located on the underlying surface, resulting in a uniform distribution of the generated point cloud. The experimental results show that our algorithm outperforms the current state-of-the-art algorithms, especially in generating more uniform point clouds. In addition, extended experiments demonstrate the potential of our algorithm for point clouds upsampling.

Multi-frame matching decision is a key step of high-precision scene matching navigation. The traditional decision algorithms mainly adopts hierarchical filtering method. When dealing with multi-frame sequences with low matching quality, it is hard to meet the precision requirements. In view of this problem, an intelligent multi-frame matching consistency decision algorithm based on SVM_bagging is proposed. Firstly, the features of correlation plane formed by matching are split and combined to form a variety of training datasets and train multiple independent support vector machine (SVM) models based on the bagging idea of ensemble learning. Then, a hard voting mechanism is adopted to combine the decision results of all SVM models for consistency decision. The experimental results on Sentinel_1-2 optical/SAR data show that the proposed algorithm can further reduce the false positive rate and false negative rate by more than 60% on the basis of the traditional decision algorithm.

For ship targets in down-sight conditions, the aspect ratio is uneven. Therefore, horizontal box labeling can lead to the interference of too much background information, making the target modeling inaccurate. To address this issue, a Rotated Discriminative Scale Space Tracker (roDSST) based on DSST is proposed. Firstly, the sampling method has been refined. The rotated target area is sampled using bilinear interpolation to reduce background interference and avoid quantization errors. Secondly, an angle filter is introduced to predict the rotation angle of the bounding box. Finally, the template update mechanism is improved by always retaining a certain weight of the initial template. The experimental results show that roDSST has superior performance in tracking accuracy and robustness on downward-looking ship sequences.

Infrared sensors are highly valuable in both civil and military applications due to their immunity to weather and time conditions. Among the various targets in infrared remote sensing images, vehicle targets are the most prevalent and significant. However, Existing deep learning-based detection algorithms face challenges such as high false alarm rates and low efficiency when applied to infrared weak vehicle targets detection. To address the aforementioned issues, we propose a Multi-scale Adaptive Feature Enhancement Network, named MAFE-Net, which focuses on highlighting significant features of vehicles and the correlation between targets and their surrounding environments. It consists of two significant designs: the self-adaptive coordinate attention (SCA) for feature enhancement and the multi-scale self-attention module (MSM) for feature fusion. In the meantime, we created the IRSWV dataset, which comprises 4,770 infrared aircraft-captured images. Comparing our algorithm with other current mainstream target detection algorithms on the IRSWV dataset, our algorithm demonstrates significant advantages.

Remote sensing ship detection is of great importance for the applications of maritime transport and precision guidance. However, the background interference on land, the large similarity between ship classes, and the arbitrary orientation of ships all pose serious challenges for inshore ship detection. To solve the problem of complex and diverse land backgrounds and sea conditions, we use the contextual attention module to enhance the ship target features. In addition, a rotation-invariant feature extraction and fusion module is introduced to make the features directionally invariant and increase the inter-class distance by fusing global features with local features. Finally, the feature decoupled module is redesigned to resolve the conflict between target detection and fine-grained recognition. Experimental results show that our proposed integrated model for inshore ship detection and fine-grained recognition improves mAP by about 6% compared to the current baseline model.

Aircraft detection in synthetic aperture radar (SAR) image is a very important but challenging question. Due to the multi-scale characteristics of aircraft and the complex background of airports in SAR images, the detection process often encounters challenges of false alarms and missed detections. Meanwhile, the presence of SAR image noise and the discrete distribution characteristics of scatterers can also contribute to incomplete aircraft target detection results. To address these problems, we proposed a novel multi-scale approach based on Swin Transformer. It employs a shifted window-based self-attention mechanism to extract the correlated features between scatter points. Moreover, to better enhance and integrate the multi-scale information among various level features, we incorporated an enhanced neck network with a four-layer feature pyramid and proposed an Adaptive Feature Fusion Network (AFFN) in our approach. Experiments are conducted on the GaoFen-3 (GF3) SAR aircraft datasets, and the results show the effectiveness of the proposed method.

Infrared vehicle detection based on aerial images has significant applications in military and civilian fields for the perception ability under low-light and foggy conditions. However, it remains challenging due to the following characteristics. First, infrared textures and edges are blurred, implying a plenty of low-frequency signals and a shortage of detailed descriptions. Second, objects in infrared images present different patterns depending on their thermal radiations, which hamper the feature extraction of convolution kernels. Third, infrared images lack color information, which means fewer features for classification and regression can be used. Inspired by cognitive neuroscience that humans perceive the entirety from low-frequency information and discern details from high-frequency information, we devise a new framework for oriented infrared vehicle detection called I2MDet (Infrared Information Mining Detector) to tackle the above challenges. It consists of two significant designs: the kaleidoscope module and the semantic feature supplement module (SFSM). In the kaleidoscope module, we explore the effect of kernel sizes and dilation rates on frequency information mining with kaleidoscope-like equivalent kernels. Features in this module are extracted by adaptive involution operators instead of convolution kernels to deal with multiple patterns. The SFSM provides the network with features beneficial for classification and regression. On the one hand, the network is guided to learn more meaningful features under semantic supervision. On the other hand, features output by the SFSM supplement the detection head with semantic information. Experimental results on the public dataset DroneVehicle demonstrate that our proposed approach achieves outstanding performance on oriented infrared vehicle detection.

Aircraft detection in remote sensing images plays an essential role in military and civil applications. Many aircraft in remote sensing images are small targets and are covered by cloud and fog, which make it difficult to extract sufficient feature information for aircraft detection. In this paper, a single-stage object detection model YOLM (You can Look More) has achieved better remote sensing aircraft detection performance by extracting more features. YOLM includes an enhanced neck network with a four-layer feature pyramid and a path aggregation network. More layers enable the model to extract more detailed information. To use the background information around the aircraft as a supplement to aircraft features, an attention module which can obtain the image context information of aircraft is designed. Experiments are conducted on the aircraft subset of FAIR1M data set and some aircraft images we obtained, and the proposed model has out-performed many baseline methods such as YOLO V5s.

Aircraft is easily covered by clouds in optical remote sensing images. It is a challenge to detect the aircraft and recognize its sub-categories in this situation. However, the methods proposed by the current research are mainly applied to high-quality images, which do not perform well on cloudy images. In this paper, an adaptive feature aggregation network called AFA-Net is proposed to solve this problem. We design a mixed self-attention module that adaptively focuses on the uncovered parts of the aircraft and its neighborhood from space and channel in feature maps. Experiments were done on the Optical Image Aircraft Detection and Recognition Data Set of the 3rdTianzhibei Challenge. Compared with the most advanced object detection algorithms, the proposed approach achieves state-of-the-art performance.

As an important part of the air defense and anti-missile system, the surface-to-air missile sites (SAMSs) have important military application value. The existing deep learning-based detection algorithms have the problems of high false alarm rate and low efficiency when applied to large-scale remote sensing images. To address this issue, in this work we propose a multi-task detection and recognition network, including classification branch and detection branch. The classification branch selects the suspected target area from the large-scale remote sensing image, and the detection branch detects and recognizes the target in the suspected area, achieving precise positioning from coarse to fine. In addition, we propose a density clustering algorithm to post-process the detection results, which effectively reduces the false alarm rate of the algorithm. Finally, we propose a SAMS detection and recognition dataset (DSAMS), which divides the SAMSs into three parts: the launch fielding, the launch bunker and the control and guide plain. Comparing our algorithm with other current mainstream target detection algorithms on the DSAMS dataset, our algorithm has significant advantages.

Military-civilian attribute recognition of ships in synthetic aperture radar (SAR) imagery plays an important role in marine surveillance. However, high-quality labeled data are hard to obtain for SAR ships, which hinder the development of deep learning models. Considering that models directly transferred from labeled optical images cannot achieve satisfactory performance for SAR applications due to the great discrepancy of different modalities, we propose a two-stage transfer learning method by combining the data-level and feature-level knowledge transfer. First, CycleGAN is adopted in the first stage to transfer the labeled optical image domain to the intermediate SAR-like image domain with the attribute labels. Then, a novel network called Domain Transfer using Adversarial learning and Metric learning (DTAM) is proposed to realize the task of military-civilian ship recognition by the domain adaption of the intermediate domain and the target SAR domain with joint adversarial learning and metric learning. To validate the proposed method, we establish a high-resolution SAR ship recognition dataset (HRSSRD), containing SAR and optical images of military and civilian ships. The experimental results show that the proposed two-stage architecture exhibits promising performance on the problem of SAR military-civilian ship recognition.

Aircraft detection in remote sensing images has always been a research hotspot which has great significance in both civil and military applications. Due to the variations of aircraft types, poses, sizes and complex backgrounds, it is still difficult to effectively and accurately detect aircrafts in remote sensing images. This paper proposes DAFF-Net (Dual Attention Feature Fusion Network), which makes full use of the semantic information of the high-level feature map and the location information of the shallow feature map, and integrates the local features with its global dependency adaptively. Experiments on RSOD aircraft dataset have been implemented, and the results have proved that the detection accuracy of aircraft objects with different scales and densities can all be improved.

Image representation through local descriptors is a research hotspot in computer vision. In this letter, we propose a novel local image descriptor based on intensity permutation and zone division. The oFAST detector is first employed to detect keypoints with orientations, and then steered patterns are applied to sample rotation-invariant points within the local keypoint patch. In the step of local patch description, intensity permutation and zone division are implemented to construct our descriptor, with the advantages of inherent robustness and invariance to monotonic brightness changes. Our proposed algorithm performed well in the experiments on benchmark dataset for descriptor evaluation.

Traffic sign recognition (TSR) is an essential research issue in the design of driving support system and smart vehicles. In this paper, we propose a permutation-based image feature to describe traffic signs, which has an inherent advantage of illumination invariance and fast implementation. Our proposed feature LIPID (local image permutation interval descriptor) employs interval division and zone number assignment on order permutation of pixel intensities, and takes the zone numbers as the descriptor. A comprehensive performance evaluation on German Traffic Sign Recognition Benchmark (GTSRB) dataset is carried out, which reveals the great performance of our proposed method. Experiment results exhibit that our feature outperforms some state-of-the-art descriptors, showing a potential prospect in TSR applications.

To improve computational efficiency in synthetic aperture radar (SAR) image matching, a fast image matching method using a novel two-step searching strategy (coarse-to-fine) is proposed in this letter. This method is based on two-column histogram (TCH) hashing and improved random sample consensus (RANSAC). First, coarse matching is conducted using a novel TCH hashing, which is notable for its robustness and speed. Compared with the discrete cosine transform used in perceptual hashing, TCH describes SAR images more accurately and rapidly. Then, in the refining stage, key points are detected and described in the coarser scales using scale-invariant feature transform. The Euclidean distance strategy and the improved RANSAC based on prior energy function (P-RANSAC) are then employed to implement matching. On the basis of prior information, a model of energy function has been constructed to improve sampling strategy. Experimental results on various SAR images show that the proposed approach outperforms the state-of-the-art algorithms in SAR image matching.

Image representation through local descriptors is the basis of numerous computer vision applications. In the past decade, many local image descriptors such as SIFT and SURF have been proposed, yet algorithms requiring low memory and computation complexity are still preferred. Binary descriptors such as BRIEF have been suggested to satisfy this demand, showing a comparable performance but much faster computation speed. In this paper, we propose a novel local image descriptor, LIPID, which employs intensity permutation and interval division to yield an effective performance in terms of speed and recognition. Our method is inspired by LUCID, proposed by Ziegler and Christiansen [8]. An extensive evaluation on the well-known benchmark datasets reveals the robustness and effectiveness of LIPID as well as its capability to handle illumination changes and texture images.