Transformer, convolutional neural network (CNN), and their hybrid methods have been adopted in semantic segmentation. However, many existing methods have deficiencies in balancing performance and the number of parameters, which limits the usability of the model. Meanwhile, the comprehensive modeling ability of image information requires further exploration. Therefore, based on hybrid architecture, we propose a lightweight semantic segmentation method (FSENet), which designs a frequency sparse enhancement way and feature integration strategy. Especially, Swin Transformer (ST) is used as a coupling carrier to model the frequency and spatial domain information, while focusing on the sparse characteristics of the image to enhance the modeling ability while ensuring lightweight. In addition, a lightweight feature integration model (GLFI) is constructed to combine the local features derived from residual CNN with global features obtained by frequency enhancement sparse ST. Experimental validation on the public Vaihingen and Postdam dataset, which obtain the mean intersection over union (mIOU) of 73.97% and 71.24% with only 3.54M parameters, confirms the performance of the proposed method.

Semantic change detection (SCD) extends the binary change detection (BCD) task, as it not only locates change areas but also identifies change transition types. Recent research has verified that a multitask learning network performs well in tackling the SCD task, which jointly addresses the two subtasks of change localization and semantic identification. However, it remains a challenging to specifically optimize the distinct features of these two subtasks and establish their interaction to further enhance the overall performance of the multitask network efficiently. In this paper, we propose a novel SCD method that emphasizes the refinement and collaboration of difference and semantic features (RCDSF). Specifically, a difference feature refinement branch (DFRB) is designed to integrate temporal information and highlight the difference features. Simultaneously, a semantic context refinement branch (SCRB) is developed to extract multi-scale and cross-scale semantic details. Moreover, a simple yet effective feature interaction-fusion module (FIFM) is incorporated to coordinate the two subtasks, ensuring consistency while providing additional auxiliary information for each other. Comprehensive experiments on two public remote sensing image SCD datasets demonstrate that the proposed method outperforms the state-of-the-art algorithms. The code will be available at https://github.com/wanglinlin0219/RCDSF.

With the application of convolutional neural network (CNN) and transformer in semantic segmentation, more and more effective methods have been proposed. However, the above methods still need to be improved at the level of feature analysis and integration. Therefore, we propose a semantic segmentation method based on global and local features integration (GLFI), which firstly uses CNN to obtain local features and swin transformer to extract global features, and then combines the adaptive strategy and dilation convolution to design the feature integration way, and finally the semantic segmentation results can be obtained. Experimental results on the public dataset show that the proposed GLFI achieves better performance.

Deep learning based methods, such as the representative vision transformer and convolutional neural network structures, can characterize spatial-spectral features of hyperspectral images (HSI) well and achieve outstanding classification performance. Nevertheless, when land cover is complex, the intra-class spectral consistency may be weak and difficult to express effectively in the original data space, leading to potential bias regarding the validity of spatial-spectral information utilization. We propose a new method GSPFormer that first constructs a global spectral projection space to generate land cover more robust representations and enhance the spectral consistency in local neighborhoods. After that, a space aggregation idea is introduced to obtain the central pixel’s more abundant spectral feature expression for better classification by fusing all spectral features in the local neighborhood. Extensive experiments are conducted on various HSI datasets for evaluating the classification performance of GSPFormer and other state-of-the-art networks. Comparison results indicate the superiority of the proposed method not only in classification accuracy but also in the number of parameters and convergence. The code of GSPFormer will be found at https://github.com/Preston-Dong/GSPFormer.

Convolutional neural networks (CNNs) have been widely used with its powerful discriminative ability in change detection (CD), but most CNNs-based methods are still exploring ways to capture relatively long-range context in spatial-temporal domain. The recent vision transformer (ViT) models long-range dependencies based on self-attention mechanism, which has been applied in CD. However, such transformer-based architectures do not fully consider the potential of interdependencies among the high-level semantic feature maps and easily overlook local detail features, resulting in a non-compact interior of the large-scale change area and missing small changes. Therefore, we propose a new transformer based hybrid network called interact-feature transformer network with spatial detail enhancement module (IFTSDNet), which takes advantage of transformers to capture long-range context, and of CNNs to extract local information. We design an interact-feature transformer (IFT), which can not only obtain the global contextual information, but also achieve the interactions of high-level semantic feature maps. The spatial detail enhancement module (SDEM) with a group of various receptive fields is built to refine spatial features, which incorporates more discriminative feature representations. Comparative experiments prove the effectiveness of the proposed method, which shows better performance than four recent transformer-based methods. The code will be available at https://github.com/wanglinlin0219/IFTSDNet.

Object detection for high resolution images has been an important cornerstone in remote sensing interpretation. Though considerable success has been made, there still exists issues for hierarchical semantic representation and integration, which limit the performance of existing methods. Therefore, we first analyze different integration patterns of deformable convolution on RetinaNet comprehensively to enhance the adaptability for the geometrical variations elegantly, which may provide the instructions of the future backbone designs for remote sensing images. Second, a feature pyramid network with filter mechanism (F- FPN) is proposed to strengthen the interaction of hierarchical semantics in feature reflow. Experiments on a challenging public dataset DIOR indicate the great performance.

With the increase in the amount of remote sensing images (RSIs), deep learning (DL) has been used to the change detection (CD) task in remote sensing field and achieved good results. However, most existing methods do not take full advantage of the temporal dependence of the multi-temporal images. In this paper, we propose a novel method for CD, namely SNN-LSTM (a deep Siamese neural network (SNN) with convolutional Long Short-Term Memory (ConvLSTM) and channel attention module (CAM)), especially for capturing and representing spatial-temporal information effectively. It mainly contains three parts. First, a network based on Siamese convolutional architecture is designed to extract multi-level features. Then, a ConvLSTM block is introduced to further obtain time dependency of multi-temporal RSIs, and spatial information is also extracted simultaneously. Finally, CAM blocks are used to refine the extracted multi-level features, enhance the feature of changes, and eventually generate change map. The experiments are conducted on LEVIR-CD dataset, both visual results and quantitative assessment prove that the proposed method outperforms several state-of-the-art methods.

Due to the difficulty and time-consuming of labeling ground truth map in practical situations, unsupervised multiple change detection (MCD) for multispectral images (MSIs) have attracted much attention in recent years. One possible strategy to obtain multiple changes is to assign labels to the binary change result. However, some methods are difficult to obtain the accurate binary result because of the complexity of backgrounds; moreover, assigning labels is also a challenge owing to the limitation of the number of spectral channels in MSIs. Therefore, we propose a novel unsupervised MCD framework based on auto-updating multitemporal matrix factorization (AMMF) and spatiospectral channel augmentation (SSCA). In AMMF, the accurate binary change result can be detected based on joint matrix factorization, during which the distribution and subspace information of each temporal image are regularized to encode the spatiotemporal correlation. In SSCA, some novel augmentation strategies are introduced to increase the number of channels in MSIs to form the normalized high-dimensional maps for each temporal image based on nonlinear operations and convolutional sparse analysis, respectively. MCD can be achieved by integrating the binary change result and directional information that can be calculated by high-dimensional maps of different temporal images. Experiments are conducted on two real MSIs, indicating that the proposed framework performs well in detecting multiple changes.

This letter presents a novel method for hyperspectral image (HSI) restoration, which aims to improve the removal effectiveness of the sparse noise. In contrast to the existing approaches that employ the$L_{1}$-norm for tractable optimization, we apply the non-convex non-smooth$L_{0}$-norm to measure the sparsity of the impulse noise, stripes, deadlines, and other outliers accurately. By combining the low-rank and total variation (TV) priors to exploit the intrinsic properties of the clean HSI and using the patch scheme to preserve local features, the$L_{0}$-PLRTV restoration model is established. In order to deal with the optimization problem, we introduce an equivalent primal-dual formulation to reformulate the$L_{0}$-norm term, and develop a minimization approach for the objective function based on the alternating iterative method. The simulated and real data experiments confirm that the proposed algorithm can effectively reduce the sparse noise in HSI.

With the development of convolutional neural network (CNN), many CNN-based object detection methods have made a remarkable success in very high-resolution (VHR) remote sensing images (RSIs). However, the standard convolution has a fixed receptive field, which makes it deficient in dynamic feature capture; complex backgrounds may also lead to the degradation of detection performance. Accordingly, this letter proposes a novel multiscale semantic guidance network (MSGN) to tackle these problems, wherein, based on the deformable convolution, an improved feature extraction backbone is proposed to capture features dynamically. Moreover, features from different layers are used to ensure the ability for detecting multiscale objects. Furthermore, a multilevel semantic guidance filtering subnetwork is proposed based on the designed backward semantic guidance filtering (BSGF) module, to suppress the complex backgrounds. Experimental results show that the proposed MSGN has stronger robustness and a better accuracy for multiscale object detection, compared with other reference methods.

With the wide application of deep learning (DL), change detection (CD) for remote-sensing images (RSIs) has realized the leap from the traditional to the intelligent methods. However, many existing methods still need further improvement in practical applications, especially in increasing the effectiveness of feature extraction and reducing the model computational cost. In this article, we propose a novel deep multiscale Siamese network with parallel convolutional structure (PCS) and self-attention (SA) (MSPSNet), which has excellent capabilities of feature extraction and feature integration under an acceptable consumption. It mainly contains three subnetworks: deep multiscale feature extraction, feature integration by the PCS, and feature refinement based on the SA. In the first subnetwork, a deep multiscale Siamese network based on convolutional block is designed to depict the image features at different scales for different temporal images. In the subsequent subnetworks, a PCS model is proposed to integrate multiscale features of different temporal images, and then, an SA model is constructed to further enhance the representation of image information. Experiments are conducted on two public RSI datasets, indicating that the proposed framework performs well in detecting changes.

Change detection (CD) for multitemporal remote sensing images can find change trends and reveal the development patterns. However, the typical methods based on algebraic operation, image transformation, or segmentation may not yield satisfactory results due to the spectral variability and noise complexity. In a sense, it can be considered that multitemporal images are composed of unchanged and changed regions, as well as noise. In the light of this, a stepwise subtraction method for CD is proposed, based on auto-updating multitemporal matrix factorization (AMMF) and spectral constraint, to remove the unchanged regions and noise from the original images step by step. The unchanged regions are first identified by AMMF, during which the distribution and subspace information of each temporal image are regularized to encode the spatio-temporal correlation. Then, mean shift smoothness is adopted as a spectral constraint so as to remove the noise. In this way, the changed regions have been highlighted so that the change map can be obtained by a postsegmentation method. Experiments have been conducted on three multitemporal data sets, including images from Quick Bird, aerial, and GF-1, indicating that the proposed method is effective and robust, which is superior to some state-of-the-art methods.

The richness of spectral information in multitemporal hyperspectral images (MHSIs) offers the possibility to effectively detect subtle changes and properties of grounds. However, severe spectral variabilities and inadequate spatial co-exploitation capabilities limit the performance of existing methods due to differences in acquisition times and conditions. Therefore, this article proposes a strategy of joint unmixing and multitemporal spatial information coguidance (JUC) to fully exploit the spatio-temporal-spectral features. First, a multitemporal joint unmixing method is used to achieve endmembers’ extraction and abundance estimation. Wherein the method adds spectral perturbed regularization when compared to the traditional unmixing strategy, making it robust to spectral variability. Second, we propose a multitemporal coguidance method that highlights the contrast between changed and unchanged regions and removes the noise by transferring the common structure information between the multitemporal first principal component map and the abundance difference maps. It will obtain an enhanced abundance difference maps and achieve effective combination of multitemporal spatial information. The final change result can be obtained by combining and thresholding these enhanced abundance difference maps. Experiments on some data sets demonstrate that the proposed algorithm is sufficiently valid and robust for multitemporal images, especially for data containing spectral variabilities and obvious structures.

With to the development of sensors, the fusion of features from multisource data becomes an interesting but challenging problem. In this paper, the fusion of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data is investigated with a novel and simplified deep learning architecture, named the dual-branch convolutional neural network (DB-CNN). More specifically, a 3D CNN framework as one of the two branches is used to extract spectral-spatial features simultaneously from HSI, which can keep three-dimensional structural characteristics of HSI. Another one is 2D CNN with cascade blocks, which is developed to extract elevation feature from LiDAR data, and it can exploit the multiscale features. Finally, the features of two branches will be flattened and stacked, and then sent to the fully connected layers. The experiments show that the proposed DB-CNN method can effectively fuse the HSI and LiDAR data, and yield higher classification performance than some existing methods.

Change detection techniques for remote sensing images are increasingly applied to many fields, such as disaster monitoring, vegetation coverage analysis and so on. How to improve the accuracy of detection has been a critical topic that confuse the researchers for a long time. In this paper, a method combining multiscale segmentation and fusion for high-resolution images is presented. The strategy of multiscale segmentation is to segment the same image several times under different scales, and then extract the features of objects. After, the features are used as inputs of change detection. The final results are achieved by decision-level fusion. The experiments show that, comparing with other typical methods, the method proposed in this paper has a superior performance in change detection for high-resolution images.