Multimodal change detection (MCD) has important applications in disaster assessment, but the nonlinear distortion of features and spatial misalignment caused by sensor imaging differences make it difficult to obtain changes through direct comparison. To overcome the above problems, this study aims to realize MCD by capturing the modality-independent structural commonality features between Multimodal Remote Sensing Images (MRSIs). To achieve this, we devise a basic Graph Kolmogorov-Arnold Network (GKAN) to excavate spatial structural relationships and cross-modal nonlinear mappings simultaneously. Based on this, we propose a Dual-branch GKAN (DGKAN) for unsupervised MCD, which can capture spatial-spectral structural commonality features and compare them directly to detect changes. Concretely, the GKAN is used within the DGKAN to build two autoencoders consisting of a Siamese encoder and two independent decoders to learn spatial-spectral structural commonality features through feature reconstruction. Besides, we introduce a Covariance Structural Commonality Loss (CSCL), which guides the network in extracting spatial-spectral structural commonality features between MRSIs by unsupervised constraints on the distributional consistency of cross-modal features. Experiments on several MCD datasets show that the proposed DGKAN can achieve convincing results, and ablation studies verify the effectiveness of the GKAN and CSCL.

Facial expressions are a powerful medium for conveying emotions. In facial expression recognition (FER) field, the difficulty of collecting specific expressions often leads to class imbalance in mainstream datasets, significantly reducing the classification accuracy of deep neural networks. To address these issues, we propose a stable-diffusion-based augmentation method for facial expression (SDAFE) that resolves class imbalance problems and enhances data generation quality through cross-modal label guidance. By leveraging the neutrality of neutral faces, we generate additional expressions to balance the dataset classes. We introduce a peak signal-to-noise ratio (PSNR) filter to ensure the high quality of the generated images and a cosine similarity cross-modal filter based on CLIP encoders to ensure that the content of the generated images accurately aligns with their labels. Furthermore, we introduce a novel model, FERNeXt, which demonstrates outstanding performance in FER tasks, surpassing the state-of-the-art accuracy on the FER2013 dataset and achieving strong results on the RAF-DB and NHFI datasets. Subsequently, the performance of several models across different datasets significantly improves through the use of SDAFE in our experiments.

Land-cover change detection with hyperspectral remote sensing images (HyperCD) has become attractive in the applications of remote sensing images. Many existing studies have indicated that attention mechanisms play an important role in HyperCD. However, methods based on attention enhancement for HyperCD require further improvement. In this letter, we propose a novel multiscale difference feature-fusion network (MDFN) to improve the detection performance of HyperCD. First, a submodule named multiattention feature enhancement (MAFE) module was designed and embedded on each scale in the backbone of the proposed MDFN to capture subtle changes. Second, with the motivation of exploring the feature connection of a target on different scales, the attention feature maps from each scale were fused via a proposed novel cross-scale residual fusion module (CS-RFM). Finally, a softmax function was adopted to generate a binary change detection map based on the fused features. Experimental results based on comparison with five existing related works indicated that the proposed MDFN not only has some advantages in improving change detection performance with real hyperspectral remote sensing images (HRSIs) but also exhibits superiority in the requirement of training samples that are preferred in practical applications. For instance, using only 5% of the training samples, the average accuracy (AA) on the Farmland dataset is improved by 0.63%. The code will be available athttps://github.com/ImgSciGroup/2024-MDFN.

Multimodal change detection (MCD) has garnered significant interest due to its capacity to address a variety of emergencies in a timely and effective manner. However, discrepancies in sensors and imaging techniques often hinder the direct comparison of heterogeneous remote sensing images (HRSIs), making it difficult to extract change information. To overcome this challenge, we propose a novel superpixel-based AutoEncoder Kolmogorov-Arnold Network (AEKAN) for unsupervised MCD. The primary objective of AEKAN is to excavate the latent commonality features between HRSIs. Notably, commonality features in unchanged regions are generally more pronounced than those in changed regions, which can be leveraged to assess change magnitude. To achieve this, the proposed method utilizes the Kolmogorov-Arnold Network (KAN), renowned for its capability to model data distributions, to extract these commonality features between HRSIs. Concretely, the proposed AEKAN consists of a Siamese KAN encoder and dual KAN decoders. The Siamese encoder aims to map HRSIs and extract latent commonality features, while the dual decoders reconstruct original bitemporal images from these features. In addition, we incorporate a hierarchical commonality loss function within the Siamese encoder to train AEKAN. This loss function is designed to intentionally guide the network in capturing commonality features by minimizing the discrepancies in features extracted from HRSIs at each layer of the Siamese encoder. The extracted commonality features are then adopted to quantify the change magnitude between images through mean square error (MSE). Extensive experiments on five MCD datasets demonstrate that the proposed AEKAN outperforms existing methods. The source code is available at:https://github.com/TongfeiLiu/AEKAN-for-MCD.

Achieving land cover change detection (LCCD) through remotely sensed images (RSIs) is important in the observation of the changes on the Earth’s surface. In such detection, spectral-reflectance noise and the uncertainty of the imaging external conditions for the bitemporal RSIs usually cause some salt-and-pepper noisy pixels in the results and reduce the change detection accuracy. In this article, a hierarchical feature-fusion triple network (HFTN) is proposed to improve the performance of LCCD with RSIs. Overall, the proposed HFTN aims to learn representative features to improve change detection performance via two feature learning enhancement strategies and a hierarchical feature-fusion mechanism. First, an image feature difference model is proposed to generate the input feature for the middle branch and guide the learning performance. Second, a progressive denoising module (PDM) is proposed and applied to each temporal image to reduce the noise before feeding the features into the backbone of the proposed HFTN. Finally, a hierarchical feature-fusion module (HFFM) is proposed to fuse the learned deep feature for generating a change-magnitude image. Additionally, multiscale convolution, cross-scale fusion, and a shared weight are adopted in the backbone of the proposed HFTN to further enhance the feature learning performance. Compared with eight state-of-the-art methods, experimental results verified the feasibility and superiority of the proposed HFTN for LCCD with RSIs. For example, the proposed HFTN achieved improvement rates of approximately 0.43%–11.83% for overall accuracy (OA) and 0.11%–4.81% for false alarms (FAs) across six pairs of real RSIs. The code can be available athttps://github.com/ImgSciGroup/HFTN-NET.git.

Sample augmentation is crucial for improving land cover classification performance when the samples are limited. However, the traditional sample augmentation approach concentrates on enlarging the quantity of sample via generation and synthetic technique directly, the sample quality is usually neglected. In this article, we propose a novel sample augmentation approach with threshold estimation (SATE) to improve both the quantity and quality of samples for hyperspectral remotely sensed image (HRSI) classification. Firstly, a threshold estimation algorithm (TEA) is proposed to identify high-confidence potential samples from the initial classification map by utilizing the prediction probabilities of different classes. Second, a semi-variational model is employed to detect and correct pseudo-labels in the spatial domain, further enhancing the quality of selected potential samples. Finally, a farthest point sampling (FPS) algorithm optimizes sample distribution in the spectral domain, improving representation for intra-class heterogeneity. Experimental results based on four real HRSIs and compared with eight state-of-the-art few-shot-based methods verify the feasibility and superiority of the proposed SATE approach. The improvement achieved by our proposed approach is about 0.79% ~ 4.31% in terms of the overall accuracy. Code is available at https://github.com/ImgSciGroup/SATE.

Achieving satisfactory land cover classification performance with high-resolution remote sensing images (HRSIs) usually requires sufficient samples for a supervised classifier. However, labeling sufficient samples is labor-intensive and time-consuming. In this article, a Novel Sample Augmentation Approach (NSAA) is proposed to synthesize new samples and improve classification accuracies for HRSI when initial known samples are very limited. First, a very small sample set of each class is prepared manually for the algorithm’s initialization. Second, a sample generator based on normal cloud model is proposed, and an adaptive region growing algorithm is suggested to explore some potential samples around a known sample for parameter estimation of the sample generator. Third, to further refine the generated samples around an initial known sample, a near-to-far space constraint strategy is proposed based on the K-means clustering algorithm to improve the quality of the generated samples. The proposed sample augmentation approach is incorporated with a classifier iteratively, and a sample balancing strategy is suggested in the iterative progress. Experiment results based on six real HRSIs and compared with eight state-of-the-art methods demonstrate the feasibility and superiorities of the proposed sample augmentation approach. Moreover, the reliability and robustness of the generated samples are verified by popular deep-learning networks and typical traditional classifiers. The improvement achieved by our proposed approach is about 0.12% – 0.95% in terms of the overall accuracy.

Diffusion model has achieved excellent performance in natural image processing, which can learn the noise distribution by the degradation and restoration processes. However, the model is limited to Gaussian noises. Actually, Polarimetric Synthetic Aperture Radar(PolSAR) images have complex non-Gaussian speckle noises, for which the Gaussian diffusion model is difficult to learn their intrinsic statistical characteristics. In this paper, we propose a novel scattering mechanism inspired non-Gaussian diffusion model for PolSAR image classification. To better simulate the PolSAR speckle noise, a mixed noise distribution is defined for PolSAR covariance matrices by combining Gamma multiplicative and Gaussian additive noises. A non-Gaussian forward noising process is derived to degrade a clean PolSAR image to a noisy image by steps. Then, the U-net structure is trained to remove noises for each step, effectively extracting non-Gaussian statistical features. However, statistical features can only characterize the overall distribution of the dataset, which is insufficient to describe complicated individual objects; the original PolSAR data reflect the detailed scattering mechanism for individual pixels, which can provide complementary object information for classification. Therefore, a scattering-statistical joint learning network is further developed with a dual-branch architecture to enhance discrimination ability. In particular, a multiscale pyramid module and attention mechanism are designed to improve the ability of feature learning. Experimental results on five real PolSAR datasets demonstrate that the proposed method effectively captures edge details and preserves homogeneous regions for terrain classification, especially in heterogeneous regions.

Semantic change detection (CD) not only helps pinpoint the locations where changes occur, but also identifies the specific types of changes in land cover and land use. Currently, the mainstream approach for semantic CD (SCD) decomposes the task into semantic segmentation (SS) and CD tasks. Although these methods have achieved good results, they do not consider the incentive effect of task correlation on the entire model. Given this issue, this article further elucidates the SCD task through the lens of multitask learning theory and proposes a semantic change detection network based on boundary detection and task interaction (BT-SCD). In BT-SCD, the boundary detection (BD) task is introduced to enhance the correlation between the SS task and the CD task in SCD, thereby promoting positive reinforcement between SS and CD tasks. Furthermore, to enhance the communication of information between the SS and CD tasks, the pixel-level interaction strategy and the logit-level interaction strategy are proposed. Finally, to fully capture the temporal change information of the bitemporal features and eliminate their temporal dependency, a bidirectional change feature extraction module is proposed. Extensive experimental results on three commonly used datasets and a nonagriculturalization dataset (NAFZ) show that our BT-SCD achieves state-of-the-art performance. The code is available at https://github.com/TangYJ1229/BT-SCD.

Deep learning models have demonstrated excellent performance for polarimetric SAR image classification. However, existing approaches generally use a fixed square window to sample image blocks as the network input, which may not effectively extract various terrain objects. To alleviate this issue, we proposed an adaptive region sampling network to learn different terrain types by introducing a novel sampling scheme with varying direction and scale. Initially, a complex PolSAR image is segmented into homogeneous, heterogeneous and boundary regions. Subsequently, small-scale and large-scale sampling windows are designed for homogeneous and heterogeneous regions, to capture local and global features for two types of regions respectively. Additionally, an adaptive directional sampling window is designed for boundary regions to ensure context consistency in the image block and prevent edge confusion. Experiments conducted on real PolSAR data sets demonstrate that our method achieves superior classification results, providing both regional consistency and boundary preservation.

Popular Transformer networks have been successfully applied to remote sensing (RS) image change detection (CD) identifications and achieved better results than most convolutional neural networks (CNNs), but they still suffer from two main problems. First, the computational complexity of the Transformer grows quadratically with the increase of image spatial resolution, which is unfavorable to RS images. Second, these popular Transformer networks tend to ignore the importance of fine-grained features, which results in poor edge integrity and internal tightness for largely changed objects and leads to the loss of small changed objects. To address the above issues, this letter proposes a lightweight structure-aware Transformer (LSAT) network for RS image CD. The proposed LSAT has two advantages. First, a cross-dimension interactive self-attention (CISA) module with linear complexity is designed to replace the vanilla self-attention (SA) in the visual Transformer, which effectively reduces the computational complexity while improving the feature representation ability of the proposed LSAT. Second, a structure-aware enhancement module (SAEM) is designed to enhance difference features and edge detail information, which can achieve double enhancement by difference refinement and detail aggregation to obtain fine-grained features of bi-temporal RS images. Experimental results show that the proposed LSAT achieves significant improvement in detection accuracy and offers a better tradeoff between accuracy and computational costs than most state-of-the-art (SOTA) CD methods for RS images.

Supervised classification with hyperspectral remote-sensing images (HRSIs) plays an important role in practical applications. However, labeling samples with HRSIs for supervised classification is time-consuming and labor-intensive. In this letter, we propose a new sample enhancement approach to improve the classification performance of HRSIs. First, the uncertainty and representativeness of the sample are defined to achieve sample possibility measurement for each pixel, and some pixels with high possibility can be selected as candidate samples. Then, two rules related to label correlation analysis and spectral similarity are defined to further refine the candidate samples used for generating the final sample set. Finally, the above-mentioned steps are fused into an iterative algorithm to enhance and balance the training samples for each class. The feasibility of the proposed approach was verified by applying it to classification with two real HRSIs. A comparison with some typical traditional sample enhancement methods and widely used few-shot deep-learning methods indicated the advantages of the proposed approach for improving classification accuracies. The improvement achieved by our proposed approach is about 0.79% ~ 2.31% in terms of the overall accuracy (OA). The code of the proposed approach is available athttps://github.com/ImgSciGroup/2023-GRSL-SIEA.

Multiscale strategy based on deep learning networks has been widely used in land cover change detection (LCCD) with remote sensing images (RSIs). However, ground targets with varying shapes and sizes often exhibit different feature performances at different scales. In this article, we proposed two submodules to extend the classical UNet and then improve the performance of LCCD with RSIs. First, a hierarchical multiscale attention fusion (HMAF) was proposed to cover the changing area with multiscale shapes and sizes. Second, a change weight learning module (CWLM) was proposed to describe the change probability between the learned features from each encoding layer. Finally, the adaptive weight acquired by CWLM was adopted to guide the decoding process of the proposed network. Compared to five state-of-the-art methods using three pairs of real RSIs, the proposed network is feasible to improve the change performance of LCCD with RSIs, such as it achieved an improvement of 0.75% in overall accuracy (OA) and 14.03% in precision in terms of Dataset-A.

Influenced by deep learning, extracting roads from high-resolution remote sensing images has attracted extensive attention. However, most previous works have focused on fully supervised models relying on large amounts of annotated data and have not considered the characteristics of narrow and elongated roads. In order to alleviate the model’s dependency on labeled data, reduce annotation workload, and fully exploit road characteristics, we proposed a semi-supervised road extraction network via multiple consistency and multitask constraints (MCMCNet) that utilizes only minimal labeled data, while exploiting unlabeled data through the mining of pseudo-label information for constraint. Moreover, to ensure the generation of more accurate pseudo-labels, we incorporated a guided contrastive learning module (GCLM) into the model to increase interclass discriminability and enhance consistency constraints. In addition, to ensure the continuity of road extraction and integrity of the main roads, we added a road skeleton (road centerline) prediction head (RSPH) in addition to the original road segmentation prediction head. Finally, we introduced an adaptive road augment module (ARAM) to enhance linear road features and avoid learning redundant information by the use of local and global information adapted to road features. Extensive experiments demonstrated that MCMCNet achieved a 3%–5% improvement in$F1$and IoU across three benchmark datasets, compared to other classical semi-supervised road extraction models, and the visualization results confirmed that MCMCNet partially addressed challenges including road occlusion, foreground-background high-similarity regions at extremely low label rates. The code is available athttps://github.com/zhouyiqingzz/MCMCNet.

The objective of change detection (CD) is to identify the altered region between dual-temporal images. In pursuit of more precise change maps, numerous state-of-the-art (SOTA) methods design neural networks with robust discriminative capabilities. The convolutional neural network (CNN)-transformer model is specifically designed to integrate the strengths of the CNN and transformer, facilitating effective coupling of feature information. However, previous CNN-transformer studies have not effectively mitigated the interference of feature distribution differences as well as pseudovariations between two images due to cloud occlusion, imaging conditions, and other factors. In this article, we propose a domain adaptive and interactive differential attention network (DA-IDANet). This model incorporates domain adaptive constraints (DACs) to mitigate the interference of pseudovariations by mapping the two images to the same deep feature space for feature alignment. Furthermore, we designed the interactive differential attention module (IDAM), which effectively improves the feature representation and promotes the coupling of interactive differential discriminant information, thereby minimizing the impact of irrelevant information. Experiments on four datasets demonstrate the superior validity and robustness of our proposed model compared to other SOTA methods, as evident from both quantitative analysis and qualitative comparisons. The code will be available online (https://github.com/Jyl199904/DA-IDANet).

Change detection with remote sensing images (RSIs) plays an important role in the community of remote sensing applications. However, when change detection is conducted with hyperspectral remote sensing images (HRSIs), how to measure the change magnitude between bitemporal HRSIs becomes challenging due to the high dimension of HRSIs. In this article, a novel Distribution Distance based on Inconsistent Adaptive Region (D2IAR) change detection approach is proposed to measure the change magnitude between bitemporal HRSIs for improving the performance of change detection with HRSIs. First, a band selection algorithm called optimal neighborhood reconstruction is employed to reduce the dimensions of HRSIs. Then, an adaptive region around each pixel is generated to explore the contextual feature around each pixel, and kernel density estimation is suggested to estimate the spectral distribution of the pixels within an adaptive region. A distribution distance is defined based on the adaptive region to measure the change magnitude between bitemporal HRSIs. Finally, the change magnitude between pairwise adaptive regions is measured by the proposed distance between the pairwise distributions. Experimental results based on four datasets and comparisons with eight methods indicated the feasibility and superiorities of the proposed D2IAR-based change detection approach with HRSIs. The improvement rates are approximately 0.13%-24.04% for overall accuracy. The code and datasets can be available at: https://github.com/ImgSciGroup/2024-HSICD.

Landslide is one of the most serious geological disasters around the world, and acquiring landslide inventory mapping (LIM) with remote sensed images (RSIs) plays an important role in disaster relief. However, various external imaging conditions of bitemporal RSIs usually cause pseudo-changes and challenges for achieving satisfied LIMs. In this article, a pioneering change magnitude measured distance named Spectral-Spatial Similarity based on Adaptive Region (S3AR) is proposed for achieving LIMs with bitemporal RSIs. First, an adaptive region is proposed to utilize the spatial-contextual information around each pixel, because the shapes and size of a landslide site are usually irregular and unpredictable. Then, a shape description algorithm is proposed for constructing a shape description vector, which aims at measuring the spatial difference of adaptive regions. Finally, to improve the separability between the landslide area and the background, brightness is suggested to couple with the shape description vector of an adaptive region to generate spatial-spectral similarity to measure the change magnitude between pairwise adaptive regions from the bitemporal RSIs. When the entire bitemporal RSIs are scanned and calculated via these steps, a change magnitude image between bitemporal RSIs can be generated, and then binary LIMs are obtained by a binary threshold. Experiments based on comparing eight state-of-the-art approaches demonstrated the feasibility and superiorities of the proposed S3AR for achieving LIMs with bitemporal RSIs. For example, the improvements on the four datasets are 5.81%, 14.06%, 6.03%, and 20.51% in terms of total error.

Sample augmentation is effective for improving the supervised performance of land-cover classification with hyperspectral remote sensed image (HRSI) when the training samples are limited. However, numerous existing methods have neglected, considering the interclass-imbalance problem in the process of sample augmentation. In this work, new sample generation and sample balance strategies were promoted and simultaneously combined into an iteration for balancing and improving classification performance with HRSI. First, a sample augmentation with superpixel’s constraint (SASC) is designed to augment the initial training samples set to avoid the overfitting of a sample generation neural network. Second, sample generation based on generative adversarial network (SGGAN) was proposed to generate samples for each class. Then, the proposed SASC, SGGAN, and a pattern recognition neural network named 3 dimensions-convolutional neural network (3-D-CNN) are combined into an iterative classification process called iterative sample generation and balance (ISGB) for balancing the user’s accuracy for each class and optimizing the classification performance. Experiments on four widely used HRSIs are performed. The results when compared with eight state-of-the-art methods based on few-shot learning and generative adversarial network (GAN) efficiently demonstrate the feasibility and superiorities of the proposed approach for improving land-cover classification performance when the initial samples are limited. Moreover, the comparisons of the standard deviation of the user’s accuracies (SDUA) demonstrated the balancing ability of the proposed approach. The code of the proposed approach is available athttps://github.com/ImgSciGroup/ISGBA.

Labeled samples are important in achieving land cover change detection (LCCD) tasks via deep learning techniques with remote sensing images. However, labeling samples for change detection with bitemporal remote sensing images is labor-intensive and time-consuming. Moreover, manually labeling samples between bitemporal images requires professional knowledge for practitioners. To address this problem in this article, an iterative training sample augmentation (ITSA) strategy to couple with a deep learning neural network for improving LCCD performance is proposed here. In the proposed ITSA, we start by measuring the similarity between an initial sample and its four-quarter-overlapped neighboring blocks. If the similarity satisfies a predefined constraint, then a neighboring block will be selected as the potential sample. Next, a neural network is trained with renewed samples and used to predict an intermediate result. Finally, these operations are fused into an iterative algorithm to achieve the training and prediction of a neural network. The performance of the proposed ITSA strategy is verified with some widely used change detection deep learning networks using seven pairs of real remote sensing images. The excellent visual performance and quantitative comparisons from the experiments clearly indicate that detection accuracies of LCCD can be effectively improved when a deep learning network is coupled with the proposed ITSA. For example, compared with some state-of-the-art methods, the quantitative improvement is 0.38%-7.53% in terms of overall accuracy. Moreover, the improvement is robust, generic to both homogeneous and heterogeneous images, and universally adaptive to various neural networks of LCCD. The code will be available at https://github.com/ImgSciGroup/ITSA.

Land cover change detection (LCCD) with bitemporal remote sensing images has been widely used in practical applications. However, when the bitemporal images are multimodal remote sensing images (MRSIs) which are acquired with different sensors, the change detection performance may be unsatisfactory, because MRSIs cannot be compared directly to generate a change magnitude and obtain a change detection map. Here a novel approach is proposed to overcome this problem, i.e., the Enhanced UNet (E-UNet) which learns deep shared features from MRSIs to achieve change detection with MRSIs. First, apre-event image to post-eventimage (P2P) transformation module based on classical Cycle-consistent Generative Adversarial Network (CGAN) is suggested to embed at the head of the proposed E-UNet to translate the pre-event image to a post-event image one. Then, multi-scale convolutions are added at each encoding layer to capture the various shapes and sizes of ground targets. Finally, a Polarized Self-Attention (PSA) module is employed before beginning the decoding progress of E-UNet with an aim to pay extra attention to changed areas. Compared with five typical state-of-the-art methods, experimental results based on two pairs of MRSIs well demonstrated the feasibility and advantages of the proposed E-UNet for LCCD with MRSIs in terms of visual observations and quantitative evaluations. For example, the improvement is 4.19% and 4.75% in terms of the overall accuracy for the Sardinia dataset and California dataset, respectively. The code of the proposed approach can be found at https://github.com/ImgSciGroup/E-UNet.

Learning performance is unsatisfactory when training deep-learning networks without prior-knowledge guidance. In this paper, a multi-scale change detection neural network guided by a change gradient image (CGI) was proposed. First, a multi-scale information attentional module was embedded in the backbone of UNet to achieve a multi-scale information fusion task of bi-temporal images. Second, the position channel attention module was promoted to make the neural network pay more attention to the spectral and spatial information in the multi-scale fused feature map. Finally, a change gradient guide module was proposed to optimize backpropagation and overcome the negative effects of pseudo-change. Compared with seven state-of-the-art methods using three pairs of real remote sensing images, the proposed approach could smoothen the salt-and-pepper noise from the detection maps and improve the detection accuracy. The quantitative improvements are about 1.67% and 3.00% in terms of overall accuracy and Kappa coefficient, respectively, thus confirming the feasibility and superiority of the proposed approach for detecting land cover change with remotely sensed images. Code: https://github.com/ImgSciGroup/MACGGNet.git.

Deep convolutional neural networks have achieved much success in remote sensing image change detection (CD) but still suffer from two main problems. First, existing multi-scale feature fusion methods often employ redundant feature extraction and fusion strategies, which often leads to high computational costs and memory usage. Second, the regular attention mechanism in CD is difficult to model spatial-spectral features and generate 3D attention weights at the same time, ignoring the cooperation between spatial features and spectral features. To address the above issues, an efficient ultra-lightweight spatial-spectral feature cooperation network (USSFC-Net) is proposed for CD in this paper. The proposed USSFC-Net has two main advantages. First, a multi-scale decoupled convolution (MSDConv) is designed, which is clearly different from the popular atrous spatial pyramid pooling (ASPP) module and its variants since it can flexibly capture the multi-scale features of changed objects by using cyclic multi-scale convolution. Meanwhile, the design of MSDConv can greatly reduce the number of parameters and computational redundancy. Second, an efficient spatial-spectral feature cooperation strategy (SSFC) is introduced to obtain richer features. The SSFC differs from existing 2D attention mechanisms since it learns 3D spatial-spectral attention weights without adding any parameters. The experiments on three datasets for remote sensing image CD demonstrate that the proposed USSFC-Net achieves better CD accuracy than most convolutional neural networks-based methods and requires lower computational costs and fewer parameters, even it is superior to some Transformer-based methods. The code is available at https://github.com/SUST-reynole/USSFC-Net.

Deep learning techniques have become popular in land cover change detection (LCCD) with remote sensing images (RSIs). However, many existing networks mostly concentrate on learning deep features but without considering the effect of different features’ attention and fusion strategy on detection performance. In this paper, a novel hierarchical attention feature fusion (HAFF)-based network for LCCD with RSIs is proposed. In the proposed HAFF-based network, novel multi-scale convolution fusion filters (MCFFs) explore the global semantic feature of the interested targets from multi-perspectives ways. To achieve that objective, the proposed MCFFs are composed by a well-known position attention module (PAM) and a novel multi-perspectives feature filter block with different kernel sizes. In addition, a compound loss function was proposed for balancing the impact from the features at different levels in terms of backpropagation error. Experiments conducted on six pairs of real RSIs, including three pairs of homogeneous images and three pairs of heterogeneous images, confirmed the superiority of the proposed HAFF network over other cognate methods. Moreover, the ablation experiments further confirmed the feasibility and superiority of the proposed MCFFs, whereas quantitative observations indicated that competitive improvements are achieved by the proposed MCFFs in terms of all the evaluation indicators. The code for the proposed approach will be available at https://github.com/ImgSciGroup/HAFF.

Samples play a crucial role in the supervised classification of remote sensing images. However, labeling large samples for training a classifier or deep learning network is not only time-consuming but also labor-intensive. In this paper, a novel land cover classification with nonparametric sample augmentation is proposed to improve the performance of hyperspectral remote sensing images (HRSIs) classification. First, initial samples with limited quantity are selected randomly from the ground truth map. Second, based on the gray image, a nonparametric adaptive region generation (NARG) algorithm is developed for utilizing the contextual information around each sample. Then, an nonparametric sample augmentation algorithm is developed with NARG to explore reliable samples iteratively around each initial sample. Finally, the above steps are fused into an iterative progress to obtain the final classification map. Compared with some typical traditional methods and some widely used deep learning methods based on four real HRSIs, our proposed approach exhibits some advantages in improving the visual performance and quantitative accuracies of HRSIs classification, such as the improvement is about 2.0% ~ 10.34% for four real HRSIs in term of the overall accuracy.

Land cover change detection (LCCD) using bitemporal remote sensing images is a crucial task for identifying the change areas on the Earth’s surface. However, the utilization of hyperspectral remote sensing images (HRSIs) introduces challenges as the detection performance is affected by the spectral noise and deducing change detection accuracies. In this work, we concentrated on utilizing spatial-contextual information to improve the change detection performance while using HRSIs. First, a band selection approach is used to minimize the spectral redundancy of HRSIs. Second, an iterative spatial-adaptive filter is proposed to smooth the noise of HRSIs. Thereafter, the change magnitude between bitemporal HRSIs is measured by coupling change vector analysis and the adaptive region around each pixel, resulting in a change magnitude image (CMI). Subsequently, the CMI is divided into a binary change detection map by using an Ostu threshold method. The experimental results on three pairs of real HRSIs efficiently demonstrated the feasibility and superiorities of the proposed approach compared with six state-of-art methods. For example, the improvement rates are approximately 0.43%-11.83% and 1.05%-15.41% for overall accuracy and average accuracy, respectively. The code of our proposed approach will be available at: https://github.com/ImgSciGroup/2023-HSICD.

Spectral-spatial features are important for ground target identification and classification with High Spatial Resolution Remotely Sensed (HSRRS) Imagery. In this paper, two novel features, named the Gaussian-Weighting Spectral (GWS) feature and the Area Shape Index (ASI) feature, are proposed to complement the deficiency of the basic image feature for land cover classification with HSRRS imagery. The proposed GWS feature is an adaptive region-based feature that aims to improve the spectral homogeneity of a local area surrounding a pixel. Additionally, it is well known that the spectral feature is inadequate for classifying HSRRS imagery. Therefore, one spatial feature called the ASI feature is proposed here to describe the relationship between the area and shape for an adaptive region around each pixel. The proposed GWS and ASI features coupled with the basic red-green-blue feature are fed into a supervised classifier to obtain the final classification map. Experiments based on four real HSRRS images demonstrate that the proposed GWS and ASI features are capable of improving classification accuracies compared with some cognate state of the art methods. Moreover, the experiments also reveal that the proposed spectral-spatial features can complement each other for enhancing the classification performance with HSRRS images.

Land cover change detection (LCCD) with very high-resolution remote-sensing images (VHR_RSIs) is important in observing surface change on Earth. However, pseudo changes usually reduces the accuracy of the detection map. In this paper, novel piecewise distance based on adaptive region key-points extraction called sparse key-point distance (SKPD) is developed to measure the change magnitude between the bitemporal VHR_RSIs for LCCD. The proposed approach consists of three steps. First, an adaptive region generation algorithm is promoted for exploring spatial-contextual information. Then, the adaptive region around each pixel is sparsely represented with the box-whisker plot theory and the adaptive region is converted into a sparse key point vector. Finally, a piecewise distance is defined to measure the change magnitude between the bi-temporal images. While the entire VHR_RSIs are scanned and the proposed SKPD method proceeds on a pixel by pixel basis, a change magnitude image (CMI) can be generated and a binary threshold method can be applied on the CMI to obtain a change detection map. Experimental results based on four pairs of real VHR_RSIs and four state-of-the-art methods effectively demonstrated the superiority of the proposed approach for achieving LCCD with VHR_RSIs, such as the improvements for the four datasets are 5.25%, 14.76%, 18.13%, and 22.24%, respectively in terms of overall accuracy.

The vulnerability of deep neural networks has garnered significant attention. Various advanced adversarial attack methods have been proposed. However, these methods exhibit higher attack performance on three-band natural images while struggling to handle high-dimensional attacks in terms of attack transferability and robustness. Hyperspectral images, unlike natural images, possess high-dimensional and redundant spectral information. On one hand, different classification models focus on distinct discriminative spectral bands, leading to poor transferability. On the other hand, most existing attack methods are implemented at the pixel-level, making them less resilient to image processing-based defenses. In this paper, we address the improvement of transferability and robustness in high-dimensional attacks and introduce a universal object-level adversarial attack method in hyperspectral image classification. We found that perturbations with higher similarity in a local region can decrease the sensitivity of adversarial attacks to various discriminative spectral patterns and enhance resistance to image processing-based defenses. Consequently, we construct spatial and spectral oversegmented templates by utilizing the local smooth properties of hyperspectral images, aiming to promote similarity among perturbations within a local region. Extensive experiments conducted on two real hyperspectral image datasets validate that our method enhances the attack transferability and robustness of several existing attack methods. By incorporating the object-level adversarial attack with baseline fast gradient sign method (FGSM), momentum iterative FGSM (MI-FGSM), and variance tuning MI-FGSM (VMI-FGSM), the average transferability success rate of the proposed method has increased by 7.38% on the PaviaU dataset and 9.30% on the HoustonU 2018 dataset than the baselines, respectively. Meanwhile, the proposed method outperforms the baselines by an average of 6.19% on the PaviaU dataset and 10.05% on the HoustonU 2018 dataset in attacking image processing-based defense models. The code is available at https://github.com/AAAA-CS/SS_FGSM_HyperspectralAdversarialAttack.

Although deep learning-based change detection (CD) methods achieve great success in remote sensing images, they still suffer from two main challenges. First, popular Convolutional Neural Networks (CNNs) are weak in extracting discriminated features focusing on changed regions, since most methods ignore the multi-frequency components of bi-temporal images. Second, although existing CD methods employ the Transformer structure to capture long-range dependency for global feature representation, it is difficult for them to simultaneously take into account the long-range dependency of changed objects at various scales. To address the above issues, we propose a triple change detection network (TCD-Net) via joint multi-frequency and full-scale Swin-Transformer. The proposed TCD-Net has two main advantages. First, we propose a multi-frequency channel attention (MFCA) module to boost the ability of modeling the channel correlation, which can compensate for the problem of insufficient feature representation caused by only performing global average pooling (GAP). Furthermore, a joint multi-frequency difference feature enhancement (JM-DFE) guiding block is proposed to improve the boundary quality and the position awareness of truly changed objects, which can effectively extract channel features of multi-frequency information and thus improve the discriminative ability of features. Second, unlike Siamese-based structures, we propose a full-scale Swin-Transformer (FST) module as the third branch to model and aggregate the long-range dependency of multi-scale changed objects, which can alleviate the missed detections of small objects and achieve more compact changed regions effectively. Experiments on three public CD datasets exhibit that the proposed TCD-Net achieves better CD accuracy with smaller model complexity than state-of-the-art methods. The code is publicly available at https://github.com/RSCD-mz/TCD-Net.

Abstract Restoring damaged multichannel visual data with high loss ratio is quite a challenging task. To address this problem, an effective LRTC (low‐rank tensor completion) model integrated with total α‐order variation (TV α ) in the fractional bounded variation space BV α is proposed to perform superior fractional‐in‐space regularization. Based on using LR constraint to restore global patterns, TV α regularization is integrated to exploit nonlocally‐correlated information on each channel to infer the lost data and simultaneously effectively deal with complex details due to the powerful fractional calculus. Then, a nonlocal fractional regularization strategy for multi‐dimensional data and an effective numerical optimization method are creatively designed to solve this problem. Two novel fractional derivative matrix approximations are derived and applied to the first two unfolding modes of the tensor respectively to conveniently solve the fractional regularization subproblem by using an element‐wise shrinkage‐thresholding operation. In addition, boundary extension and adjustment strategy are designed for the unfolded matrices to alleviate the influence of inaccurate boundary conditions in fractional derivative computations. Experiments are conducted to illustrate its performance and efficiency for YUV video, RGB and HSI restoration, especially its ability to effectively recover complex structures and the details of multi‐component visual data with relatively high missing rate.

Images from different sensors can be different in intensity or data structure to present the same object on the ground. The distinct appearances make the change detection task more difficult to obtain accurate change regions. This letter presents a novel bipartite adversarial autoencoders with structural self-similarity (BASNet) for detecting land cover changes in heterogeneous remote sensing images. The main novelty lies in the following two aspects. First, a structural consistency loss is defined by the crossmodal distance in a new affinity space. It enforces the network to transform the heterogeneous images into a common domain for style alignment of the original image and the transformed image. Second, an adversarial loss term is designed to force the network to make image translation with more consistent style by distinguishing the artificial output from the real input pixels. Experiments on four heterogeneous remote sensing image datasets are provided to demonstrate the performances of the proposed method.

Change detection with heterogeneous remote sensing images (HRSIs) is attractive for observing the Earth’s surface when homogeneous images are unavailable. However, HRSIs cannot be compared directly because the imaging mechanisms for bitemporal HRSIs are different, and detecting change with HRSIs is challenging. In this letter, a simple yet effective deep learning approach based on the classical UNet is proposed. First, a pair of image patches are concatenated together to learn a shared abstract feature in both image patch domains. Then, a multiscale convolution module is embedded in a UNet backbone to cover the various sizes and shapes of ground targets in an image scene. Finally, a combined loss function, which incorporates the focal and dice losses with an adjustable parameter, was incorporated to alleviate the effect of the imbalanced quantity of positive and negative samples in the training progress. By comparisons with five state-of-the-art methods in three pairs of real HRSIs, the experimental results achieved by our proposed approach have the best overall accuracy (OA), average accuracy (AA), recall (RC), and F-Score that are more than 95%, 79%, 60%, and 61%, respectively. The quantitative results and visual performance indicated the feasibility and superiority of the proposed approach for detecting land cover change with HRSIs.

Landslide inventory mapping (LIM) plays an important role in landslide susceptibility analysis. Many LIM approaches based on change detection techniques have been proposed, but with various drawbacks. For example, existing approaches have limited capability to capture the objects of varying shapes/sizes present in an area impacted by landslide. Many existing approaches are supervised and require parameter tuning. Moreover, some methods are prone to salt-and-pepper noise. To overcome these limitations, in this letter, an algorithm based on automatic adaptive region extension using very-high-resolution remote sensing images is developed. First, a simple yet effective k-means clustering method is used to generate training samples for landslide and nonlandslide classes, which refer to changed and unchanged areas, respectively. Second, an automatic adaptive region extension algorithm is developed and applied to each pixel of the postevent image, and the label of an extended region around a pixel is determined by the nearest distance between the central pixel and the changed or unchanged samples. Finally, the labels of a pixel are recorded because a pixel in different adaptive regions may be reassigned dissimilar labels, and the final label of the pixel is consistent with its maximum assigned label. To verify the performance of the proposed approach, we conducted experiments on two different landslide sites with VHR remote sensing images in Lantau Island, Hong Kong, China. Experimental results clearly demonstrate that the proposed approach has several advantages in improving the performance of LIM with VHR remote sensing images.

Training samples are usually required to train a classifier for supervised classification of very high spatial resolution (VHR) remote sensing images. However, labeling samples is often a labor-intensive and time-consuming task. To solve this problem, this study integrates histogram distribution analysis, double-window flexible pace search (DFPS), and box–whisker plot (BP) techniques into an iterative algorithm to enrich training samples. The major steps of the proposed algorithm are given as follows. First, to acquire the feature distribution of a class, a histogram of each class (HOC) based on the raw classification map is generated. Second, to cover the spectral heterogeneity of an intraclass, some pixel points in each bin of HOC are selected as the coarse training sample set (CTS). Third, to further purify the CTS, DFPS, and BP techniques are adopted to exclude outlier samples and select the representative samples to signify the corresponding class. Finally, the refined training samples are used to retrain the classifier, and the preceding steps are constructed as an iterative algorithm. Experiments were performed on three real VHR remote sensing images to demonstrate the superiorities of the proposed approach in improving classification performance with respect to the maps obtained directly by the initial training set. In addition, compared with cognate state-of-the-art methods, the proposed approach achieved an approximately 2%–13% improvement in classification accuracy. Code available here:https://github.com/ImgSciGroup/IEEE-GRSL-GSEA-Code.

Many land cover change detection (LCCD) approaches applied on very high resolution (VHR) remote sensing images utilize spatial information by using a regular window or strict mathematical model. However, regular shape or strict models cannot fit the various shapes and sizes of the ground targets. In this article, a novel LCCD approach without the parameter is proposed to detect land cover change with VHR remote sensing images. First, an adaptive spatial-context extraction algorithm is applied to explore contextual information around a pixel. Second, the change magnitude between pairwise pixels is quantitatively measured by computing the band-to-band distance which is defined by the pairwise adaptive regions around the corresponding pixels. Finally, after the generation of a change magnitude image (CMI), a binary threshold method called double-window flexible pace search (DFPS) is adopted to divide CMI into a binary change detection map. The performance of the proposed approach is verified by comparing it with five state-of-the-art methods with three pairs of VHR images. The comparisons demonstrated that the proposed approach achieved the improved detected results comparing with state-of-the-art LCCD methods. The code of the proposed approach is available athttps://github.com/TongfeiLiu/ASEA-CD.

Land cover change detection (LCCD) with very high-resolution (VHR) remote sensing images has been widely used in various applications. However, pseudo-changes and noise usually affect the performance of detection map. In this letter, an object-oriented sorted-histogram similarity measurement (OSSM) is proposed for measuring the change magnitude between bi-temporal remote sensing images. First, multi-scale objects are acquired for the post-event image using a multi-scale segmentation algorithm, and then the pixels within each object are considered to construct the pairwise histograms and the bin of each histogram is sorted in descending order. Second, a bin-to-bin (B2B) distance is defined to measure the change magnitude between the pairwise object-based histograms, and the change magnitude image (CMI) is generated after all the bi-temporal images are scanned object by object. Finally, a simple yet effective method called Otsu is used to divide the CMI into binary change detection maps. The experiments on three pairs of VHR images produced promising results compared with five popular LCCD approaches, for example, the improvement is about 2.5% for F-score.

The performance of a neural network is highly dependent on the labeled samples. However, the labeled samples are primarily clean, which prevents the network from capturing the features of the samples near the decision boundary. For hyperspectral images (HSIs), high spectral dimensions and same-spectra foreign matter lead to more boundary samples in the data. In this letter, we investigate an adversarial attack algorithm against these problems for HSIs. A modified DeepFool algorithm is implemented to generate boundary adversarial samples with minimal disturbance, and the generated boundary adversarial samples are simply added to the training set to improve the accuracy of the boundary samples in the data. Furthermore, we iteratively complete network training and boundary adversarial sample generation so that the decision boundary can be adjusted according to the real-time classification situation. Extensive experiments are carried out on the two HSI datasets, and the results demonstrate that the modified DeepFool algorithm can improve the accuracy of the decision boundary. Our findings also show that adversarial attacks are sensitive to high-dimensional and multiple-category data and are worthy of further study.

With increasing spatial resolution of remote sensing images, accurate classification of land classes depends more on the number of labeled samples. However, the acquisition of labeled samples is difficult and time-consuming. Hence, generative adversarial networks (GANs) have become a new method for collecting training samples for very-high-resolution (VHR) remote sensing image classification. A traditional GAN generates new samples with the same distribution as the labeled samples. However, the generated samples have features close to their class center, and the network cannot obtain effective discriminative ability for the samples close to the decision boundary. This letter presents an improved GAN (IGAN) for VHR remote sensing image classification. In the proposed framework, the generator aims to generate synthetic samples close to the classification boundary, and the discriminator aims to constrain the labels of the synthetic samples. The obtained synthetic samples can effectively improve the classification accuracy of the classification boundary. Experiments are conducted on two VHR remote sensing images, and the results show that the proposed method performs better than several state-of-the-art methods.

With the fast development of remote sensing platforms and sensors technology, change detection with heterogeneous remote sensing images (Hete-CD) has become an attractive topic in recent years and plays a vital role in land cover change detection for responding to natural disaster emergencies when homogeneous images are unavailable. Although Hete-CD has been developed for about three decades, and various related methods have been developed and applied successfully in practice, a systematic and comprehensive review of the current achievements regarding Hete-CD remains lacking. Therefore, in this article, we first present an overview of Hete-CD in terms of the related literature. Second, the major techniques of Hete-CD are reviewed in terms of publicly available datasets, the taxonomy of major techniques, results, performance, and quantitative evaluation. Then, some classical methods are selected for comparison and discussion. Finally, based on the discussion and literature review, challenges, opportunities, and future directions for Hete-CD are concluded. The review aims to provide a “one-stop-shop” understanding of the problems with the categories of existing approaches, open opportunities and challenges, and potential future directions for Hete-CD.

Land cover change detection (LCCD) using remote sensing images (RSIs) plays an important role in natural disaster evaluation, forest deformation monitoring, and wildfire destruction detection. However, bitemporal images are usually acquired at different atmospheric conditions, such as sun height and soil moisture, which usually cause pseudo and noise change into the change detection map. Changed areas on the ground also generally have various shapes and sizes, consequently making the utilization of spatial contextual information a challenging task. In this paper, we design a novel neural network with spatial-spectral attention mechanism and multi-scale dilation convolution modules. This work is based on the previously demonstrated promising performance of convolutional neural network for LCCD with RSIs and attempts to capture more positive changes and further enhance the detection accuracies. The learning of the proposed neural network is guided with a change magnitude image. The performance and feasibility of the proposed network are validated with four pairs of RSIs that depict real land cover change events on the Earth’s surface. Comparison of the performance of the proposed approach with that of five state-of-art methods indicates the superiority of the proposed network in terms of 10 quantitative evaluation metrics and visual performance. Such as, the proposed network achieved an improvement about 0.08%~14.87% in terms of OA for Dataset-A.

Landslide inventory mapping (LIM) is an important application in remote sensing for assisting in the relief of landslide geohazards. However, while conducting LIM tasks performing change detection analysis using bi-temporal very high-resolution (VHR) remote sensing images, due to landslide usually occurred in a mountain area, the phenological difference and outcrop rock may bring pseudo-changes to LIM results. In this paper, a novel change detection approach based on Adaptive Region Shape Similarity (ARSS) is proposed for LIM with VHR remote sensing images to improve detection performance. First, an adaptive region around each pixel is extended to explore the contextual information. Then, direction lines within an adaptive region are defined to describe the shape of the adaptive region. Finally, the pixels located on each direction line are taken into account to build the corresponding histogram. The shape similarity between the pairwise histogram curves is measured by using the Discrete Frchet Distance (DFD). Once the bi-temporal images are processed by using the abovementioned steps, a change magnitude image (CMI) is generated, while a threshold is then used to obtain a final binary change map. The proposed approach is applied to three pairs of landslide sites images acquired with aerial plane and one land use change dataset acquired by Quick Bird Satellite. Compared with ten state-of-the-art methods, the proposed approach achieved LIMs and detection results with higher accuracies and better performance.

Deep neural networks have strong feature learning ability, but their vulnerability cannot be ignored. Current research shows that deep learning models are threatened by adversarial examples in remote sensing (RS) classification tasks, and their robustness drops sharply in the face of adversarial attacks. Therefore, many adversarial attack methods have been studied to predict the risks faced by a network. However, the existing adversarial attack methods mainly focus on single-modal image classification networks, and the rapid growth of RS data makes multimodal RS image classification a research hotspot. Generating multimodal adversarial examples needs to consider a high attack success rate, subtle perturbation, and collaborative attack ability between different modalities. In this article, we investigate the vulnerability of multimodal RS classification networks and propose a multifeature collaborative adversarial network (MFCANet) for generating multimodal adversarial examples. Two modality-specific generators are designed to generate the multimodal collaborative perturbations with strong attack ability, and two modality-specific discriminators make the generated multimodal adversarial examples closer to the real instances. In addition, a modality-specific generative loss and a modality-specific discriminative loss are proposed, and an alternating optimization strategy is designed for training the proposed MFCANet. Extensive experiments are carried out on the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen 2D dataset and ISPRS Potsdam 2D dataset. The results show that the attack performance of the proposed method is stronger than that of the fast gradient sign method (FGSM), project gradient descent (PGD), and Carlini and Wagner (C&W) attack methods.

The majority of the change detection (CD) methods consider spatial information by using a regular window or strict mathematical model. Moreover, these methods use the spectra directly to measure the change magnitude between bitemporal images. To solve this problem, local histogram-based analysis (LHBA) is proposed for detecting a land cover change in this letter. This new approach aims to inhibit the pseudo change by defining the local histogram trend (LHT) in an adaptive manner instead of using spectral values to measure change magnitude directly. In the proposed approach, the spatial information around each pixel is first exploited by defining an adaptive local histogram. The LHT distance between the pairwise local histograms is then developed to measure the change magnitude between the pairwise pixels of bitemporal images. Finally, the change magnitude image is generated, and a binary CD is achieved by a threshold method. Experiments based on two pairs of very high-resolution remote sensing images, which refer to land use change and landslides events, demonstrate the advantages and performance of the proposed approach.

Imbalanced training sets are known to produce suboptimal maps for supervised classification. Therefore, one challenge in mapping land cover is acquiring training data that will allow classification with high overall accuracy (OA) in which each class is also mapped onto similar user's accuracy. To solve this problem, we integrated local adaptive region and box-and-whisker plot (BP) techniques into an iterative algorithm to expand the size of the training sample for selected classes in this article. The major steps of the proposed algorithm are as follows. First, a very small initial training sample (ITS) for each class set is labeled manually. Second, potential new training samples are found within an adaptive region by conducting local spectral variation analysis. Lastly, three new training samples are acquired to capture information regarding intraclass variation; these samples lie in the lower, median, and upper quartiles of BP. After adding these new training samples to the ITS, classification is retrained and the process is continued iteratively until termination. The proposed approach was applied to three very high-resolution (VHR) remote-sensing images and compared with a set of cognate methods. The comparison demonstrated that the proposed approach produced the best result in terms of OA and exhibited superiority in balancing user's accuracy. For example, the proposed approach was typically 2%-10% more accurate than the compared methods in terms of OA and it generally yielded the most balanced classification.

Very high-resolution (VHR) remote sensing images can geometrically depict ground targets in detail but are usually insufficient in the spectral domain. This characteristic leads to a considerable amount of noise and pseudo change in the produced binary change detection maps (BCDMs) when VHR remote sensing images are used for change detection. Here, to solve the aforementioned problem, an object-oriented key point vector distance (KPVD) is proposed to measure the change magnitude between bitemporal VHR images when land cover changes are detected. The proposed KPVD-based change detection approach comprises the following major steps. First, multiscale objects based on a postevent image are extracted by the fractional net evaluation segmentation approach, and then, the segments are taken as the unit for measuring the change magnitude between bitemporal images. Second, key points and the corresponding vector are defined to describe the object feature instead of using the total pixels within the object. Finally, KPVD is proposed to measure the change magnitude between the local areas referenced to the object in the bitemporal images. The change magnitude image (CMI) between the bitemporal images is generated while the entire images are scanned and processed object by object. A well-known automatic binary method, the Otsu approach, is employed in this article to divide CMI into a BCDM. Experimental results conducted on four real data sets demonstrate the feasibility and outperformance of the proposed KPVD-based change detection approach compared with five state-of-the-art methods in terms of visual performance and quantitative measurements.

Most of the approaches used for Landslide inventory mapping (LIM) rely on traditional feature extraction and unsupervised classification algorithms. However, it is difficult to use these approaches to detect landslide areas because of the complexity and spatial uncertainty of landslides. In this letter, we propose a novel approach based on a fully convolutional network within pyramid pooling (FCN-PP) for LIM. The proposed approach has three advantages. First, this approach is automatic and insensitive to noise because multivariate morphological reconstruction is used for image preprocessing. Second, it is able to take into account features from multiple convolutional layers and explore efficiently the context of images, which leads to a good tradeoff between wider receptive field and the use of context. Finally, the selected PP module addresses the drawback of global pooling employed by convolutional neural network, FCN, and U-Net, and, thus, provides better feature maps for landslide areas. Experimental results show that the proposed FCN-PP is effective for LIM, and it outperforms the state-of-the-art approaches in terms of five metrics, $Precision$ , $Recall$ , $Overall~Error$ , $F$ -$score$ , and $Accuracy$ .

Detecting land cover change through very-high-resolution (VHR) remote sensing images is helpful in supporting urban sustainable development, natural disaster evaluation, and environmental assessment. However, the intraclass spectral variance in VHR remote sensing images is usually larger than that of median-low remote sensing images. Furthermore, the bitemporal images are usually acquired under different atmospheric conditions, sun height, soil moisture, and other factors. Consequently, in practical applications, many pseudo changes are presented in the detected map. In this paper, an adaptive histogram trend (AHT) similarity approach is promoted to quantitatively measure the magnitude between the corresponding pixels in bitemporal images in terms of change semantic. In the proposed approach, to reduce the phenological effect on the bitemporal images of land cover change detection (LCCD), we first define the quantitative description of AHT. Second, the change magnitudes between pairwise pixels are quantitatively measured by an improved bin-to-bin (B2B) distance between the corresponding AHTs. Then, the change magnitudes between two entire bitemporal images are measured AHT-by-AHT. Finally, binary threshold methods, such as the Otsu method or the double-window flexible pace search (DFPS) method, are used to divide the change magnitude image into binary change detection maps and obtain the final change detection map. The performance of the AHT-based LCCD approach is verified by four pairs of VHR remote-sensing images that correspond to two types of real land cover change cases. The detected results based on the four pairs of bitemporal VHR images outperformed the compared state-of-the-art LCCD methods.

Marginal Fisher analysis (MFA) exploits the margin criterion to compact the intraclass data and separate the interclass data, and it is very useful to analyze the high-dimensional data. However, MFA just considers the structure relationship of neighbor points, and it cannot effectively represent the intrinsic structure of hyperspectral image (HSI) that possesses many homogenous areas. In this paper, we proposed a new dimensionality reduce (DR) model, termed local geometric structure Fisher analysis (LGSFA), for HSI classification. At first, this method computes the reconstruction point of each point with its intraclass neighbor points. Then, an intrinsic graph and a penalty graph are constructed to reveal the intraclass and interclass relationship, respectively. Finally, the neighbor points and corresponding reconstruction points are used to enhance the intraclass compactness and interclass separability in low-dimensional space. LGSFA can effectively reveal the intrinsic manifold structure and obtains the discriminating feature of HSI data. Experiments on Salinas HSI data set show that the proposed LGSFA algorithm performs the best classification results than other state-of-the-art methods.

Spectra-based change detection (CD) methods, such as image difference method and change vector analysis, have been widely used for land-cover CD using remote sensing data. However, the spectra-based approach suffers from a strict requirement of radiometric consistency in the multitemporal images. This letter proposes a new image feature named spectrum trend, which is explored from the spectral values of the image in a local geographic area (e.g., a 3 × 3 sliding window) through raster encoding and curve fitting techniques. The piecewise similarity between the paired local areas in the multitemporal images is calculated by using a sliding window centered at the pixel to generate the change magnitude image. Finally, CD is achieved by a threshold decision or a classified method. This proposed approach, called “local spectrum-trend similarity,” is applied and validated by a case study of land-cover CD in Wuqin District, Tianjin City, China, by using SPOT-5 satellite images. Accuracies of “change” versus “no-change” detection are assessed. Experimental results confirm the feasibility and adaptability of the proposed approach in land-cover CD.

This letter presents a novel spatial feature called object correlative index (OCI) to enhance the classification of very high resolution images. This novel method considers the property of an image object based on spectral similarity to construct a useful OCI to describe the spatial information objectively. Compared with the generic features widely used in image classification, the classification approach based on the OCI spatial feature results in higher classification accuracy than those approaches that only consider spectral features or pixelwise spatial features, such as the pixel shape index and mathematical morphology profiles. Experiments are conducted on QuickBird satellite image and aerial photo data, and results confirm that the proposed method is feasible and effective.