Semantic segmentation under adverse imaging conditions such as low illumination and motion blur often breaks into holes, fragments, and disconnected regions. Naive smoothing can reduce these artifacts, but it also tends to leak labels across true boundaries. We address this coherence–leakage dilemma with a plug-and-play decode-head refinement composed of aMulti-scale Boundary Detector (MBD)and aBoundary-Barrier Coherence Refiner (BBCR). MBD predicts a boundary probability map, and BBCR turns it into a soft barrier for logit-space refinement through prototype-based global coherence and barrier-gated local affinity propagation. Targeting mixed-condition deployments, where favorable and adverse inputs are interleaved, the proposed modules attach to existing segmentation heads without modifying backbones or necks. Across eight architectures on a binary dairy-farm dataset, our method improves mIoU by 0.97 points on average and by up to 13.35 points on a manually annotated low-light/blur stress-test split without target-domain fine-tuning. On NightCity with 19 classes, it further yields a consistent average gain of 0.96 mIoU points while preserving performance under favorable conditions.

Heterogeneous change detection (HeCD) enables the identification of land-cover changes using remote sensing imagery obtained from different sensors. Most existing methods overly emphasize modality transformation and shared feature extraction to bridge the gap between heterogeneous images. While these strategies facilitate comparable representations, they tend to neglect the intrinsic characteristics of the changes themselves, which limits their effectiveness in complex scenarios. To overcome this limitation, we propose a change prior-guided image transformation model (CPIT) for unsupervised HeCD. Specifically, starting from the definition of change detection, we analyze the connections among pairwise object relationships, change labels, and change semantics, and then derive change semantic consistency and inconsistency rules solely from the inherent nature of the change detection problem, without relying on data-specific assumptions. These rules are subsequently encoded as change semantic consistency and inconsistency constraints, which, from the perspective of graph signal processing, correspond to low-pass and high-pass spectral properties of the change signals. Finally, by integrating these semantic constraints with sparsity priors and image transformation constraints, we formulate a more precise transformation model for HeCD. Solving this model produces change detection results that conform to the change priors, thereby improving the detection performance. The derivation, formulation, and utilization of change priors in this work offer valuable insights for broader change detection research. Extensive experiments on five datasets validate the effectiveness of CPIT. The code will be released at https://github.com/yulisun/CPIT.

The integration of Visual Transformers (ViTs) with Faster R-CNN has shown significant promise in computer vision tasks requiring both high accuracy and efficient object detection. However, the computational cost and resource requirements of these models often limit their application in real time, resource-constrained environments. This paper proposes a novel optimization strategy for integrating ViT with Faster R-CNN to enhance both performance and efficiency. We introduce an improved ViT-Tiny backbone with a hybrid attention mechanism, CS-attention, that combines high- and low-frequency attention to better capture local and global features while minimizing computational overhead. Additionally, a pyramid feature network (FPN) is incorporated to enhance multi-scale feature extraction, allowing the model to accurately detect objects at varying scales. Experimental results demonstrate that the optimized model achieves high accuracy and real-time processing capabilities, making it suitable for deployment in industrial and edge computing applications. The proposed approach is validated through extensive experiments, providing a general solution for efficient object detection across various domains.

Deep neural networks (DNNs), though highly effective in various Earth observation tasks with remote sensing images (RSIs), are vulnerable to adversarial attacks, threatening their reliability. Each maliciously attacked RSI potentially contains unique and critical information, but current defenses lack a unified framework for rapidly detecting adversarial RSIs and accurately restoring them to their natural state. To bridge this research gap, we propose a diffusion-based dual-stage adversarial defense (DiffDual-AD) framework. In the first stage, we propose a novel adversarial detection method based on the score expectation (AD-SE), which is integrated into the forward process of diffusion model with an improved denoiser constrained for adversarial defense. During the reverse process, the second stage introduces the distance and label-guided adversarial purification (DL-GAP) to restore adversarial RSIs to natural ones. With the help of two proposed guidelines and the specific denoiser, the DL-GAP effectively smooths out the adversarial perturbations from the detected adversarial RSIs while preserving semantic information and local key features. Finally, DL-GAP yields nonadversarial purified RSIs based on the results of AD-SE. Both stages are integrated within the diffusion models, complementing each other to form a pipelined operational mode. Extensive experiments across three RSI scene classification datasets have proven its efficacy in resisting adversarial RSIs in both whitebox and black-box scenarios, achieving an average adversarial detection accuracy of 87.02% with AD-SE and an average final classification accuracy of 81.50% with the entire DiffDual-AD. The performances of our proposed AD-SE and DL-GAP have surpassed other advanced adversarial detection and purification methods when applied to RSIs. Therefore, DiffDual-AD provides a unified and universal solution to preserve the utility and security of processing the adversarial RSIs, which advances the adversarial defense research in remote sensing.

Heterogeneous change detection (HeCD) is a highly valuable yet challenging task in remote sensing. To enable the comparison of heterogeneous images with different imaging mechanisms, some structural consistency-based image transformation methods have been proposed, which utilize graph models to represent image structures and constrain the transformed images and original images to have the same structural characteristics on the graph model. Consequently, these graph-based methods face two challenges: adequately characterizing the image structure and effectively utilizing the change information. To address these challenges, this article proposes a signed graph-based image transformation (SGIT) method for unsupervised HeCD. First, we analyze the limitations of previous unsigned graph-based methods in capturing the image structure, which leads to the failure to detect changes in some scenes. In light of this, we construct signed graph models that utilize positive/negative weights to represent the similarity/dissimilarity relationships within the image, respectively, and employ adaptive weighting, negative sampling, and neighborhood expansion strategies to bolster the structure representation capability of signed graphs. Second, we analyze how the change would induce a bimodal distribution of vertex feature distances in original and transformed images. Subsequently, a distribution-induced reweighted graph Laplacian regularization (RGLR) is proposed to exploit this prior change information. Finally, a more accuracy image transformation model is obtained by incorporating three types of constraints: signed graph-based structural consistency term, bimodal distribution-induced RGLR, and change sparsity-based penalty term. Extensive comparative experiments on five real datasets have demonstrated the effectiveness of the proposed SGIT.

When incorporating spatial priors (such as sparsity and low occurrence frequency) to identify anomalies, existing hyperspectral anomaly detection (HAD) methods typically operate on the original high-dimensional spectral bands or features obtained through dimensionality-reduction techniques. However, they often neglect the explicit modeling and exploitation of the internal discriminative structure inherent in bands, consequently diluting the discriminative spectral information crucial for effective anomaly-background separation. This limitation becomes particularly pronounced when processing hyperspectral data, where the rich spectral heterogeneity could provide valuable discriminative cues for HAD. To address this issue, we propose a super-band quaternion-guided sparse network (SuperB-QSNet), which explicitly encodes the spectral discriminant structure while retaining the advantage of spatial sparse priorities. First, a heterogeneous-homogeneous super-band compression (HH-SBC) organizes spectral bands into meaningful hetero-groups. It naturally reduces redundancy and retains the representative spectral information. Second, the quaternionic sparse recovery (QSR) model encodes both spatial structures and intra-group spectral correlations via hypercomplex algebra, ensuring robust anomaly preservation and enhanced detection of spectral variations. Third, we design a differences-guided quaternionic convolution detector (DG-QCD) to maintain spectral coherence via group processing and enhance discriminative capability via inter-group difference operation. Extensive experiments show that SuperB-QSNet enhances anomaly detection accuracy while ensuring computational efficiency across real-world hyperspectral datasets.

Geolocation is a crucial step in the processing of synthetic aperture radar (SAR) images. High-altitude airborne SAR systems present unique geolocation challenges due to travelling long distances through the troposphere. However, the impact of tropospheric delay on geolocation is often overlooked in existing airborne SAR studies, which can lead to inaccuracies. To address this issue, we propose a new positioning method, the tropospheric delay-compensated range-Doppler (TDC-RD) model. The TDC-RD model leverages reference atmospheric models to estimate and compensate for the tropospheric delay in SAR images. This model effectively mitigates the impact of tropospheric delay in SAR geolocation. To further optimize the TDC-RD model solution, a digital elevation model (DEM)-assisted dual iteration method is proposed. This method iteratively adjusts the target’s plane position and elevation in an alternating manner. The effectiveness of the TDC-RD model has been validated through both simulation experiments and actual flight experiments. The results show a significant improvement in geolocation accuracy compared to existing methods, with a maximum reduction of 11.39 m and 24.33% in the mean absolute error (MAE) of SAR geolocation. The TDC-RD model has a great advantage in long-range SAR geolocation. Our research enhances the accuracy and stability of high-altitude airborne SAR geolocation without requiring ground control points.

Multimodal change detection (MCD) is a topic of increasing interest in remote sensing. Due to different imaging mechanisms, the multimodal images cannot be directly compared to detect the changes. In this article, we explore the topological structure of multimodal images and construct the links between class relationships (same/different) and change labels (changed/unchanged) of pairwise superpixels, which are imaging modality-invariant. With these links, we formulate the MCD problem within a mathematical framework termed the locality-preserving energy model (LPEM), which is used to maintain the local consistency constraints embedded in the links: the structure consistency based on feature similarity and the label consistency based on spatial continuity. Because the foundation of LPEM, i.e., the links, is intuitively explainable and universal, the proposed method is very robust across different MCD situations. Noteworthy, LPEM is built directly on the label of each superpixel, so it is a paradigm that outputs the change map (CM) directly without the need to generate intermediate difference image (DI) as most previous algorithms have done. Experiments on different real datasets demonstrate the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/LPEM.

Cross-view geo-localization aims to associate geographical location with different view images shot from different platforms. One of the critical challenges is how to effectively emphasize architectural features and reducing background interference to achieve robust cross-view matching. Most of the existing methods fail to adequately address the features of buildings, treating foreground and background equally. Leveraging prior knowledge to enhance the features of crucial architectural foreground yields greater benefits in Geo-Localization. A comprehensive three stage approach (TirSA) is proposed in this paper, which consists of three components: Pre-processing, Generate Feature Embedding, and Post-processing. In the Pre-processing stage, we employ a self-supervised feature enhancement method (SFEM) to obtain the building aware mask. Without adding additional auxiliary information, the model is guided to learn from discriminative building regions. Besides, in the Generate Feature Embedding stage, we propose an adaptive feature integration module (AFIM) to enhance feature representation capability. We also train the Siamese network using a novel improved cross-domain triplet loss to reduce the impact of inter-view domain gap. Finally, in the Post-processing stage, we employ a re-ranking method to optimize the initial retrieval list, further enhancing the matching accuracy. Remarkably, extensive experiments show that our proposed TirSA exceeds state-of-the-art by a large margin and achieves optimality in both drone-view target localization and drone navigation. Especially in the drone navigation task, our method is superior to the existing methods, achieving an improvement of approximately 5%. Code will be released at https://github.com/SunJ1025/TirSA.

Change detection (CD) between heterogeneous images is an increasingly interesting topic in remote sensing. The different imaging mechanisms lead to the failure of homogeneous CD methods on heterogeneous images. To address this challenge, we propose a structure cycle consistency-based image regression method, which consists of two components: the exploration of structure representation and the structure-based regression. We first construct a similarity relationship-based graph to capture the structure information of image; here, a k -selection strategy and an adaptive-weighted distance metric are employed to connect each node with its truly similar neighbors. Then, we conduct the structure-based regression with this adaptively learned graph. More specifically, we transform one image to the domain of the other image via the structure cycle consistency, which yields three types of constraints: forward transformation term, cycle transformation term, and sparse regularization term. Noteworthy, it is not a traditional pixel value-based image regression, but an image structure regression, i.e., it requires the transformed image to have the same structure as the original image. Finally, change extraction can be achieved accurately by directly comparing the transformed and original images. Experiments conducted on different real datasets show the excellent performance of the proposed method. The source code of the proposed method will be made available at https://github.com/yulisun/AGSCC.

Heterogeneous change detection (HCD) in remote sensing has gained significant attention. Heterogeneous images come from different sensors, which cannot be compared directly to detect changes. This letter proposes a change alignment-based graph structure learning method (CAGSL) for unsupervised HCD, which detects changes by calculating forward and backward structure differences. To achieve this objective, CAGSL incorporates two pivotal improvements. Firstly, CAGSL utilizes a graph auto-encoder (GAE) to optimize the graph structure, enabling a more accurate representation of the topological relationships between the real land covers. Secondly, CAGSL introduces a change alignment constraint based on the HCD task property that the forward and backward structural differences represent the same change event in order to enhance the optimization of the graph structure. Subsequently, the optimized graph structure is used to compute the structure difference images through graph mapping. Finally, the change map (CM) is obtained through Otsu segmentation. Experimental results demonstrate the effectiveness of the proposed CAGSL when compared to some state-of-the-art (SOTA) methods.

Multimodal change detection (MCD) is an increasingly interesting but very challenging topic in remote sensing, which is due to the unavailability of detecting changes by directly comparing multimodal images from different domains. In this paper, we first analyze the structural asymmetry between multitemporal images and show their negative impact on the previous MCD methods using image structures. Specifically, when there is a structural asymmetry, previous structure based methods can only complete a structure comparison or image regression in one direction and fails in the other direction, that is, they cannot transform or convert from complex structural images (with more categories) to simple structural images (with fewer categories). To reduce the influence of structural asymmetry, we propose a structural regression fusion based method (SRF) that simultaneously transforms the pre-event and post-event images into the image domain of each other, calculating the forward and backward changed images, respectively. Noteworthy, different from previous late fusion methods that fuse the forward and backward changed images in the post-processing stage, SRF incorporates fusion into the regression process, which can fully explore the connection between changed images, and thus improve image transformation performance and obtain better changed images. Specifically, SRF yields three types of constraints to perform the fused image transformation: structure consistency based regression term, change smoothness and alignment based fusion term, and prior sparsity based penalty term. Finally, the changes can be extracted by comparing the transformed and original images. The proposed SRF is verified on six real data sets by comparing with some state-of-the-art methods. Source code of the proposed method will be made available at https://github.com/yulisun/SRF.

Hyperspectral unmixing is an important image interpre-tation technique that aims to estimate the pure constituent materials (endmembers) and their corresponding fractional abundances in each mixed pixel. Nonnegative matrix factorization (NMF) has attracted a lot of attention because of its ability to solve mixed pixel scenarios. The sparse NMF method achieves better unmixing results thanks to its full use of the sparse characteristics of the data. However, most existing sparse NMF unmixing techniques lack the consid-eration of spatial information. In fact, hyperspectral images contain intrinsic geometric information as well as rich spatial information. In this paper, a spatial graph regularized nonneg-ative matrix factorization unmixing framework (SGNMF) is established. For the proposed SGNMF, on the one hand, the graph regularization is introduced to characterize the latent manifold structure of the data, and on the other hand, the spatial weighting factor is used to mine the spatial correlation between pixels. The optimization problem of the SGNMF model can be solved by a multiplicative iterative rule. Exper-imental results on synthetic data sets indicate that the newly proposed SGNMF method is able to produce better results than other advanced spectral unmixing algorithms.

Change detection (CD) of heterogeneous remote sensing images is a challenging topic, which plays an important role in natural disaster emergency response. Due to the different imaging mechanisms of heterogeneous sensors, it is hard to directly compare the images. To address this challenge, we explore an unsupervised CD method based on adaptive local structure consistency (ALSC) between heterogeneous images in this letter, which constructs an adaptive graph representing the local structure for each patch in one image domain and then projects this graph to the other image domain to measure the change level. This local structure consistency exploits the fact that the heterogeneous images share the same structure information for the same ground object, which is imaging modality-invariant. To avoid heterogeneous data confusion, the pixelwise change image is calculated in the same image domain by graph projection. By comparing with some state-of-the-art methods, the experimental results show the effectiveness of the proposed ALSC-based CD method.

Optical and SAR modalities provide complementary information of land properties, which can lead to outstanding classification performance. Recently, factorized bilinear coding (FBC) as an extension of bilinear pooling in respect of coding-pooling perspective, which extracted compact bilinear fusion features with second-order interaction information in the form of sparse representation, brought the performance improvements on multimodal learning tasks. However, it lost locality attributes among similar samples to be encoded. In this letter, we propose a novel locality-constrained bilinear network (LC-BNet) for land cover classification with heterogeneous remote sensing (RS) images. Specifically, the locality-constrained bilinear coding (LC-BC) introduces locality information to generate compact and discriminative fusion features for land cover classification. Extensive experimental results show superior performances of our work on two broad coregistered optical and SAR datasets.

High-resolution remote sensing (HR2S) images contain complex land objects of difference sizes, and it is important for semantic segmentation of the HR2S images to extract multiscale information. In this letter, we introduce a novel multilevel adaptive-scale context aggregating network (MACANet) for semantic segmentation of the HR2S images, which mainly consists of two parts—adaptive-scale context extraction block (AS-CEB) and sequential aggregation block (SAB). In particular, the AS-CEB introduces an inflexible strategy to obtain the features with appropriate scale information based on different asymmetric convolutions and the gated mechanism. Meanwhile, the SAB progressively aggregates multilevel adaptive-scale features, which are used to relieve the semantic gap between different-level features and generate precise score maps. Experimental results on representative HR2S datasets show the advantages of our method. The code is available athttps://github.com/RSIP-NUDT/MACANet.

Remote sensing scene classification (RSSC) is a fundamental yet challenging task in the domain of remote sensing (RS). Currently, the methods based on deep features from convolutional neural networks (CNNs) have significantly improved the scene classification accuracy (ACC). However, the standard convolution operations have limited capacity to model the long-range correlations and cannot effectively obtain global contextual understanding ability. In this letter, we propose a novel scene classification framework, termed cross-layer nonlocal network (CL-NL-Net), consisting of a backbone network, a cross-layer nonlocal (CL-NL) module, and a classifier. Among them, the backbone network is used to obtain multilayer convolutional features. The CL-NL module is the core of the proposed method, which captures the long-range correlations between different layers, so as to achieve a better global scene understanding ability. To verify the effectiveness of the proposed CL-NL-Net, we conduct experiments on four benchmark datasets, and the results demonstrate that the proposed method achieves competitive classification ACC and outperforms some state-of-the-art methods.

Image matching is a primary technology to fuse the complementary information from optical and SAR images. Due to the high nonlinear radiometric and geometric relationship, the optical and SAR image matching task remains a widely unsolved challenge. In this study, we propose to use a Siamese convolutional neural network (CNN) architecture to learn pixelwise deep dense features. The proposed network is able to balance the learning of high-level semantic information and low-level fine-grained information, which is nonnegligible for feature matching task. Under the local searching framework, the loss function is defined based on the score map produced by the sum of squared differences (SSDs) between the learned pixelwise dense features of local optical and the SAR image patches, with a fast implementation in the frequency domain. The hardest negative mining strategy is adopted to increase the discrimination of the network. Extensive experiments are conducted on optical and SAR image pairs of different spatial resolution and different landcover types, verifying the superiority and robustness of the proposed method in terms of matching accuracy and matching precision.

Optical and SAR modalities can provide the complementary information on the land properties, which usually lead to more robust and better classification performance. However, due to the restriction of imaging condition, not all modalities included into the training data sets could be available in real testing samples. Therefore, it is important to explore how to learn discriminative representations using multimodal data during the training stage, while achieving fine land cover classification using missing modalities at test time. In this article, we propose a novel dynamic-hierarchical attention distillation network (DH-ADNet) with multimodal synergetic instance selection (MSIS) for land cover classification using missing data modalities. First, the MSIS realizes the selection of the most representative multimodal instances to enhance the DH-ADNet’s ability of discriminative feature extraction. Then, the DH-ADNet is training on the basis of the curriculum learning strategy and promotes the hallucination stream to learn the privileged information. In particular, a novel dynamic-hierarchical attention distillation module (DH-ADM) is introduced, which adaptively highlights different contributions of multilayer attention distillation by carefully exploring the classification losses of multilayer features over the training iterations. Comprehensive evaluations on two coregistered optical and SAR data sets and report state-of-the-art results in the privileged information scenario.

Multimodal land cover classification (MLCC) is a fundamental problem in remote sensing interpretation, which can obtain excellent performance on account of the complementary information between the optical and SAR modalities. However, it is usually impossible to obtain multimodal data at the same time, due to the restriction of imaging conditions. When one of the modalities data is completely missing during test phase, classical multimodal learning methods might not be able to handle the MLCC task with privileged modality. In this paper, we propose an efficient Dense Adaptive Grouping Distillation Network (DAGDNet), which learns privileged information from available modalities in the train sets, and improves the classification performance in the test sets when one modality data is scarce. More specifically, to relieve the heterogeneous gaps between different modalities and then transfer the privileged information, we propose an Interactive Gated-based Feature Grouping Module (IG-FGM), which decomposes multimodal features into modalities-shared and modality-specific components to realize the decoupling of multimodal features and grouping distillation. Furthermore, the IG-FGM is inserted into different layers of the “teacher" network to implement progressive blending of multi-modalities. Then, to adaptively highlight the importance of hierarchical features distillation and grouping distillation, we propose a Multi-stage Adaptive Distillation Learning (MS-ADL) strategy so that the weights of different distillation losses are required to change continuously along with the training process. Finally, we evaluate the superior performances of our model on representative co-registered optical and SAR datasets.

Multimodal Land Cover Classification (MLCC) using the optical and Synthetic Aperture Radar (SAR) modalities has resulted in outstanding performances over using only unimodal data due to their complementary information on land properties. Previous multimodal deep learning (MDL) methods have relied on handcrafted multi-branch convolutional neural networks (CNN) to extract the features of different modalities and merged them for land cover classification. However, natural images-oriented handcrafted CNN models may not the optimal strategies to handle Remote Sensing (RS) image interpretation problems, due to the huge difference in terms of imaging angles and imaging ways. Furthermore, few MDL methods have analyzed optimal combinations of hierarchical features from different modalities. In this article, we propose an efficient multimodal architecture search framework, namely Multimodal Semantic Consistency-Based Fusion Architecture Search (M2SC-FAS) in continuous search space with the gradient-based optimization method, which can not only discover optimal optical- and SAR-specific architectures according to the different characteristics of the optical and SAR images, respectively, but also realizes the search of optimal multimodal dense fusion architecture. Specifically, the semantic-consistency constraint is introduced to guarantee dense fusion between hierarchical optical and SAR features with high semantic consistency and then capture the complementary performance on land properties. Finally, the basis of curriculum learning strategy is adopted on the M2SC-FAS. Extensive experiments show superior performances of our work on three broad co-registered optical and SAR datasets.

This paper provides a new strategy for the heterogeneous change detection (HCD) problem: solving HCD from the perspective of graph signal processing (GSP). We construct a graph to represent the structure of each image, and treat each image as a graph signal defined on the graph. In this way, we convert the HCD into a GSP problem: a comparison of the responses of signals on systems defined on the graphs, which attempts to find structural differences and signal differences due to the changes between heterogeneous images. Firstly, we analyze the GSP for HCD from the vertex domain. We show that once a region has changed, the local structure of image changes,i.e. the connectivity of the vertex containing this region changes. Therefore, we can compare the output signals of the same input graph signal passing through filters defined on the two graphs to detect changes. We analyze the negative effects of changing regions on the change detection results from the viewpoint of signal propagation, and we also design different filters from the vertex domain to explore the high-order neighborhood information hidden in original graphs. Secondly, we analyze the GSP for HCD from the spectral domain. We explore the spectral properties of different images on the same graph, and show that their spectra exhibit commonalities and dissimilarities. Specifically, it is the change that leads to the dissimilarities of their spectra. With the help of graph spectral analysis, we propose a regression model for the HCD, which decomposes the source signal into the regressed signal and changed signal, and constrains the spectral property of the regressed signal. Experiments conducted on seven real data sets show the effectiveness of the vertex domain filtering based and spectral domain analysis based HCD methods. Source code will be made available at https://github.com/yulisun/HCD-GSP.

Change detection of heterogeneous multitemporal satellite images is an important and challenging topic in remote sensing. Since the imaging mechanisms of heterogeneous sensors are different, it is not possible to directly compare heterogeneous images to detect changes as in the homogeneous images. To address this challenge, we propose an unsupervised image regression-based change detection method based on the structure consistency. The proposed method first adaptively constructs a similarity graph to represent the structure of a pre-event image, then uses the graph to translate the pre-event image to the domain of the post-event image, and then computes the difference image. Finally, a superpixel-based Markovian segmentation model is designed to segment the difference image into changed and unchanged classes. The proposed adaptive structure consistency-based image regression model can not only alleviate the impact of noise and changed pixels on the regression process by using the structure-based transformation, but also easily distinguish between changed and unchanged classes in the difference image by using the prior sparse knowledge of changes. Experimental results on six different datasets demonstrate the effectiveness of the proposed method by comparing with some state-of-the-art methods.

Change detection (CD) of remote sensing (RS) images is one of the important problems in earth observation, which has been extensively studied in recent years. However, with the development of RS technology, the specific characteristics of remotely sensed images, including sensor characteristics, resolutions, noises, and distortions in imagery, make the CD more complex. In this article, we propose a structure consistency-based method for CD, which detects changes by comparing the structures of two images, rather than comparing the pixel values of images. Because the image structure is imaging modality-invariant and not sensitive to noise, illumination, and other interference factors, the proposed method can be applied to a variety of CD scenarios and has strong robustness. Structural comparison is realized by constructing and mapping an improved nonlocal patch-based graph (NLPG) to avoid the data leakage of two images. First, we demonstrate the effectiveness of the method in homogeneous and heterogeneous CD, which shows that the proposed method can be used as a unified CD framework. Second, we extend the method to the heterogeneous CD with multichannel synthetic aperture radar (SAR) image, which can provide a reference for future research as the heterogeneous CD with multichannel SAR is rarely studied. Third, through the decomposition and in-depth analysis of NLPG, we modify the graph construction process, structure difference calculation, and the difference image fusion to make it more robust and accurate. Experiments on six scenarios 12 data sets demonstrate the effectiveness of the proposed method.

To fully explore the complementary information from optical and synthetic aperture radar (SAR) imageries, they need first to be coregistered with high accuracy. Due to the vast radiometric and geometric disparity, the problem to match high-resolution optical and SAR images is quite challenging. The present deep learning-based methods have shown advantages over the traditional approaches, but the performance increment is not significant. In this article, we explore a better network framework for high-resolution optical and SAR image matching from three aspects. First, we propose an effective multilevel feature fusion method, which helps to take advantage of both the low-level fine-grained features for precious feature location and the high-level semantic features for better discriminative ability. Second, a feature channel excitation procedure is conducted using a novel multifrequency channel attention module, which is able to make image features of different types and multiple levels effectively collaborate with each other and produce image matching features with high diversity. Third, the self-adaptive weighting loss is introduced, with which, each sample is assigned with an adaptive weighting factor, and therefore, information buried in all nearby samples can be better exploited. Under a pseudo-Siamese architecture, the proposed optical and SAR image matching network (OSMNet) is trained and tested on a large and diverse high-resolution optical and SAR dataset. Extensive experiments demonstrate that each component of the proposed deep framework helps to improve the matching accuracy. Also, the OSMNet shows overwhelming superior to the state-of-the-art handcrafted approaches on imageries of different land-cover types.

Convolutional Neural Networks (CNNs) have been shown remarkable performance in the task of remote sensing image scene classification. Recent works demonstrate that aggregating multi-scale convolutional features can significantly improve the classification accuracy. However, existing methods either use some unsupervised feature encoding methods or based on feature aggregation methods to aggregate multi -scale convolutional features, ignoring the information redundancy and semantic ambiguity between of them. To address the above-mentioned limitations, an end-to-end multi-scale feature aggregation network (MSF A) based on channel-spatial attention module is proposed to learn discriminative scene representation for remote sensing scene classification. The experimental results on the aerial image data set (AID) demonstrate that the proposed method achieves competitive classification performance compared with other state-of-the-art methods.

Optical and SAR modalities can provide the complementary information on land properties for better land cover classification. Most of existing multi-modal land cover classification methods based on two-streams convolutional neural networks (CNNs), which obtained fusion features by merging optical and SAR features that come from manually selective layer of different streams. However, they ignored different semantic between manually selective optical and SAR features, which might result in suboptimal fusion features. We tackle the problem of finding good fusion architectures for multimodal land cover classification inspired by the network architecture search (NAS), and introduces the multi-modal fusion architecture search network (M2PASNet). Extensive experimental results show superior performances of our work on a broad co-registered optical and SAR dataset.

Adversarial training is an effective method to enhance adversarial robustness for deep neural networks. However, it qequires large amounts of labeled data, which are often difficult to acquire. Recent research has shown that self-supervised learning can help to improve model performance and model uncertainty using unlabeled data. In this paper, we introduce a new adversarial self-supervised learning framework to learn a robust pretrained model for remote sensing scene classification. The proposed method exploits the advantage of dual network structure, and it requires neither labeled data for adversarial example generation nor negative samples for contrastive learning. Specifically, it consists of three major steps. Firstly, we train the online model and the target model to extract deep image features. Secondly, we generate two kinds of instance-wise adversarial examples. Finally, we iteratively learn a robust model by implicit comparing the difference between clean data and their perturbed counterpart. Preliminary experimental results on remote sensing scene classification dataset shows that our method can obtain higher robust accuracy. Our method can also be combined with other adversarial defense techniques to further promote model robustness.

Existing land cover classification methods mostly rely on either the optical or synthetic aperture radar (SAR) features alone, which ignore the mutual complementary effects between optical and SAR sources. In this article, we compare the distribution histograms of deep semantic features extracted from optical and SAR modalities within land cover categories, which intuitively demonstrates that there are the large complementary potentials between the optical and SAR features. Therefore, we propose a novel collaborative attention-based heterogeneous gated fusion network (CHGFNet), which hierarchically fuses both optical and SAR features for land cover classification. More specifically, the CHGFNet consists of three main components: two-stream feature extractor, multimodal collaborative attention module (MCAM), and the gated heterogeneous fusion module (GHFM). Given optical and SAR patch pairs, two-stream feature extractor introduces multistage feature learning methodology to acquire discriminative optical and SAR features. Then, to explore the inherent complementarity between optical and SAR features, MCAM is embedded into CHGFNet, which provides an efficient stage to capture the correlation between optical and SAR features by jointly calculating the collaborative attention in joint feature space. Finally, to automatically learn the varying contributions of both optical and SAR features for classifying different land categories, GHFM is used to fuse both optical and SAR features. Extensive comparative evaluations demonstrate the advantages of CHGFNet within land cover classification over the state-of-the-art methods on three co-registered optical and SAR data sets.

Speckle noise inherent in synthetic aperture radar (SAR) images seriously affects the visual effect and brings great difficulties to the postprocessing of the SAR image. Due to the edge-preserving feature, total variation (TV) regularization-based techniques have been extensively utilized to reduce the speckle. However, the strong scatters in SAR image with radiometry several orders of magnitude larger than their surrounding regions limit the effectiveness of TV regularization. Meanwhile, the ℓ1-norm first-order TV regularization sometimes causes staircase artifacts as it favors solutions that are piecewise constant, and it usually underestimates high-amplitude components of image gradient as the ℓ1-norm uniformly penalizes the amplitude. To overcome these shortcomings, a new hybrid variation model, called Fisher-Tippett (FT) distribution-ℓp-norm first-and second-order hybrid TVs (HTpVs), is proposed to reduce the speckle after removing the strong scatters. Especially, the FT-HTpV inherits the advantages of the distribution based data fidelity term, the nonconvex regularization, and the higher order TV regularization. Therefore, it can effectively remove the speckle while preserving point scatters and edges and reducing staircase artifacts well. To efficiently solve the nonconvex minimization problem, an iterative framework with a nonmonotone-accelerated proximal gradient (nmAPG) method and a matrix-vector acceleration strategy are used. Extensive experiments on both the simulated and real SAR images demonstrate the effectiveness of the proposed method.

Change detection (CD) of remote sensing images is an important and challenging topic, which has found a wide range of applications in many fields. In particular, one of the main challenges is to detect changes between heterogeneous images, where the difference in imaging mechanism makes it difficult to carry out a direct comparison. In this article, we propose an unsupervised CD framework based on the patch similarity graph matrix (PSGM), which assumes that the patch similarity graph structure of each homogeneous or heterogeneous image is consistent if no change occurs. First, it learns the PSGM of one image based on the self-expressive property, which can be interpreted as containing the edges of the fully connected graphs with each image patch as a vertex. Then, the change level depends on how much one image still conforms to the similarity graph structure learned from the other image. Meanwhile, the change map can be further optimized by using the prior sparse knowledge that only a small part of the image changed and most areas remain unchanged. Experiments with both homogeneous and heterogeneous data sets demonstrate the effective performance of the proposed PSGM-based CD method.

This work presents a robust graph mapping approach for the unsupervised heterogeneous change detection problem in remote sensing imagery. To address the challenge that heterogeneous images cannot be directly compared due to different imaging mechanisms, we take advantage of the fact that the heterogeneous images share the same structure information for the same ground object, which is imaging modality-invariant. The proposed method first constructs a robust K -nearest neighbor graph to represent the structure of each image, and then compares the graphs within the same image domain by means of graph mapping to calculate the forward and backward difference images, which can avoid the confusion of heterogeneous data. Finally, it detects the changes through a Markovian co-segmentation model that can fuse the forward and backward difference images in the segmentation process, which can be solved by the co-graph cut. Once the changed areas are detected by the Markovian co-segmentation, they will be propagated back into the graph construction process to reduce the influence of changed neighbors. This iterative framework makes the graph more robust and thus improves the final detection performance. Experimental results on different data sets confirm the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/IRG-McS.

Fast multiclass object detection for remote sensing images plays an important role for a wide range of applications. Traditional methods based on a sliding window search lead to heavy computational costs and are unsuitable for multiclass detection. Recently, deep learning algorithms, especially faster region based convolutional neural networks (Faster R-CNN), which adopt a region proposal paradigm to avoid exhaustive search, has achieved state-of-the-art multiclass detection performance in computer vision. This paper investigates the use of Faster R-CNN in the earth observation community. We have three contributions: 1) It's the first time to successfully use Faster R-CNN for object detection in remote sensing images. It achieved faster speed (22 ×faster) and better performance (a mAP of 78% vs. 72%) than traditional methods; 2) we adopt data augmentation to train Faster R-CNN with limited samples; 3) we successfully tested our method on large-scale google earth images, which shows robustness of our method.

This paper proposes a coupled region based convolutional neural networks (R-CNN) to automatically detect vehicles in aerial images. Traditional methods are mostly based on sliding-window search, and use handcrafted or shallow-learning based features. They have limited description ability and heavy computational costs. Recently, a series of R-CNN based methods have achieved great success in general object detection. Inspired by the previous work, we propose a coupled R-CNN to detect small size vehicles in large-scale aerial images. First, a vehicle proposal network (VPN) is proposed to generate candidate vehicle-like regions, using a hyper feature map combined by feature maps of different layers. Then, a vehicle classification network (VCN) is developed to further verify the candidate regions and classify vehicles in eight directions. In this study, our method is tested on a challenge Munich vehicle dataset and the collected vehicle dataset, with improvements in accuracy and speed compared to existing methods.

In this paper, an efficient multi-focus image fusion approach is proposed based on scale vector norm of image nonsubsampled contourlet transform(NSCT). According to the analysis of multi-focus imaging mechanism, the defocused optical imaging system can be characterized as a Gaussian low pass filter. Combining with the NSCT of an image, whether a region is in focus or out of focus can be determined by its corresponding high frequency coefficients' energy. Therefore, a scale vector norm is put forward in this paper and the fusion principles for different sub-band coefficients are also presented based on the regional scale vector norm. Experimental results demonstrate the proposed method can extract more details from multi-focus source images at a large extend and achieve more satisfactory results compared with other NSCT based methods.

To reduce the impact of outliers and noises on point pattern matching, a novel point pattern matching algorithm based on local topological characteristic and probabilistic relaxation labeling (LTC-PRL) is proposed in this paper. For each point in a point set, partial adjacent points are used to describe its local topological characteristic. To avoid the defects in angle coding of the existing global topological characteristic, a binary adjacent code is adopted in the local topological characteristic. And since the assignment of angle is greater than the distance, bigger weight is given to the angle while computing the similarity of the local topological characteristic among points. Finally, a robust compatibility measurement is defined and the support function is iterated by probabilistic relaxation labeling to get the best matching result. Experiments on synthetic data and the real image data show that the LTC-PRL has great matching performance when outliers and noises exist.

A novel affine invariant region detector based on the 4th differential invariant (DI4) is proposed in this paper. The detector combines scale-space theory with an autocorrelation matrix. Since it is proved that DI4 is a scale- space selection function, feature points and their characteristic scales are first detected by the local maxima of the normalized DI4 over scale-space. Then, the auto-correlation matrices, which are used to describe the affine shapes, are estimated on the characteristic scales of the feature points. The ellipse regions given by the auto-correlation matrices are affine invariant. In order to verify the affine invariance, we build up a simulation experiment to test affine invariance using two single- parameter transforms. The experimental results show the detected regions are invariant to rotation, scale and affine transforms as well as robust to illumination changes.

With more and more manufacturers and tradesmen put logistic outsourcing as a viable option for their business, fourth party logistics (4PL) has become a hot topic currently. The concept and the evaluation of 4PL are introduced. Focusing on the e-business environment, the character, the operation mode and the support relation are analyzed. Then, based on a logistic enterprise, structure and functionality of 4PL with reference to the TY-4PL software platform are discussed, critical success factors for realization project are named.

Quality function development (QFD) is an important methodology for concurrent engineering. QFD contacts the customer requirements and production process. In distributed environment, customer requirements, multiple team members and process are geographically, culturally, and functionally diverse. It requires cooperation and conflict resolution among design, production and sales process. In this paper, we introduce a multi-agent-based distributed QFD framework, which orients concurrent engineering (MAS-CEQFD). In this framework, QFD core module, manufacture module, distributed TM module and market module are presented. Agents are used to emulate the entities, i.e. various module and their internal departments. The framework facilitates the development of the product design and the production process.

Distribution requirement planning (DRP) can be regarded as the extension of manufacturing requirement planning (MRP) in the logical view. DRP, starting from customers' requirements, is used to solve distribution problems of material resources according to time, quantity and location in the field of circulation. It is a typical problem in supply chain management. Complexity caused by distributed supply and demand relationships has a great impact on making plans, and uncertainty of resources is one of the most important factors which must be considered seriously in the problem optimization. The paper describes research on distribution requirement planning under limited supply capacity, and an optimum model is built. By mathematical deduction, the model is equivalent to a linear program. Then the punctuality distribution requirement planning approach is proposed.

As the advanced management technology of manufacturing systems, supply chain management has made great progress. At present, the centralized hierarchical control mode is adopted by many supply chain management systems. In such systems, entities are serially linked by orders. Each entity obtains the relevant order from its upper entity, and provides the input requirements information with the lower entity. Such a system has a slow response to environments since information is transmitted and fed back according to the strict multi-layer hierarchical structure. With the global market competition, it is much more necessary to put forward a new and efficient control method based on the existing supply chain management systems. In the paper, the supply chain management mode based on coordination is proposed and discussed in detail.