With the rapid advancements in computer vision, high-resolution remote sensing imagery has become a crucial data source for object detection. Nevertheless, effectively utilizing limited computational resources and reducing the burden on satellite edge devices remains a significant challenge. To effectively reduce model complexity while maintaining its representational capacity, this article proposes a scale-aware pruning framework (SAPF) to enhance remote sensing object detection ability. First, this article classifies the convolutional layers in object detection models into two categories: layers with a single-scale feature representation and layers with a multiscale feature representation. For convolutional layers with single-scale features, we utilize singular value decomposition (SVD) to quantify feature importance and assess filter redundancy to enhance model efficiency. By removing less critical filters, this pruning criteria aims to reduce the model size and computational load without compromising performance. However, convolutional layers with multiscale features are crucial for optimizing feature extraction and balancing information capture across various scales. To address this, this article evaluates the similarity between convolutional layers with different scales to determine the contribution of various scale features in multiscale fusion. Surprisingly, the SAPF can reduce the FLOPs and parameters, as well as ensure the representational ability obviously when the YOLO v5s and Faster-RCNN are adopted to classify the NWPU VHR-10, RSOD, and SIMD datasets. This means we can save the training computation resources for the model. Additionally, SAPF can significantly improve the efficiency of the model in object detection to ensure its real-time performance.

Multimodal change detection (MCD), as an emerging task, aims at recognizing change regions from bi-temporal remote sensing images (RSI) of different modalities. Inspired by the success of the self-attention mechanism in transformer, attempts have been made to solve MCD through the transformer variants. However, transformer-based network optimization requires high-quality training samples. In addition, due to the significant differences in the data distribution, semantic information, and feature representation of multimodal data, transformer-based methods have obvious deficiencies in local feature representation and spatial consistency, especially when dealing with heterogeneous images. To address the above challenges, we propose a collaborative frequency-aware transformer for MCD (CFAT-MCD). As an unsupervised framework, CFAT-MCD is capable of learning more fine-grained patterns of land cover change through a few pseudo-labels. The CFAT is designed to enhance spatial consistency and align the features on a multi-scale basis, which can effectively mitigate the effects of modal differences. In addition, we propose a window-based spatial-frequency collaborative representation (SFCR) module to introduce frequency information into the spatial domain and improve the discriminability of spatial features. Extensive experiments on public datasets and quantitative analyses have validated the superior detection performance of our approach and the effectiveness of each module.

The vulnerability to poor local optimum and the memorization of noise data limit the generalizability and reliability of massively parameterized convolutional neural networks (CNNs) on complex real-world data. Self-paced curriculum learning (SPCL), which models the easy-to-hard learning progression from human beings, is considered as a potential savior. In spite of the fact that numerous SPCL solutions have been explored, it still confronts two main challenges exactly in solving deep networks. By virtue of various designed regularizers, existing weighting schemes independent of the learning objective heavily rely on the prior knowledge. In addition, alternative optimization strategy (AOS) enables the tedious iterative training procedure, thus there is still not an efficient framework that integrates the SPCL paradigm well with networks. This article delivers a novel insight that attention mechanism allows for adaptive enhancement in the contribution of diverse instance information to the gradient propagation. Accordingly, we propose a general-purpose deep SPCL paradigm that incorporates the preferences of implicit regularizer for different samples into the network structure with inductive bias, which in turn is formalized as the self-paced curriculum network (SPCNet). Our proposal allows simultaneous online difficulty estimation, adaptive sample selection, and model updating in an end-to-end manner, which significantly facilitates the collaboration of SPCL to deep networks. Experiments on image classification and scene classification tasks demonstrate that our approach surpasses the state-of-the-art schemes and obtains superior performance.

Pruning techniques have been utilized widely for convolutional neural networks (CNNs) to reduce the computation resources in remote sensing scene image classification. However, conventional pruning techniques are weight-based, which can not balance the pruning ratio and representation ability appropriately. In this paper, we propose a Data Customization-based Multiobjective Optimization Pruning (DCMOP) framework for the pruning in remote sensing scene image classification, which can not only trade-off between pruning ratio and capability for CNNs, but also speed up the evolutionary process for the pruning. We adopt the multiobjective evolutionary algorithms (MOEAs) to search for a trade-off between the pruning ratio and capability for CNNs. However, a big concern of pruning for networks via MOEAs is that the evaluation of sub-networks is time-costing. This originates that the slimmed sub-networks require a lot of retraining operation, which will burden the hardware. In order to alleviate this limitation, we design a Data Customization-based Proxy Mechanism (DCPM) to reduce the size of the input dataset in terms of the structure of the slimmed sub-network to accelerate significantly the evolutionary process for the pruning. According to this, our proposed DCMOP achieves the pruning with higher efficiency and performance by cooperating with MOEAs and DCPM. Experimental results based on four datasets of AID, NWPURESISC45, PatternNet, and WHU-RS19 show that the proposed DCMOP can achieve a balance between model performance and pruning rate, while obviously reducing the time cost of the pruning.

The existing few-shot scene classification (FSSC) algorithms have achieved satisfactory results, but they are limited by the paradigm of centralized machine learning, i.e., private remote sensing data need to be centralized on a certain server for training. However, remote sensing images generally contain sensitive information such as national security and company privacy, so it is realistically difficult to collect remote sensing data from all the parties. Therefore, there is a pressing requirement in FSSC for a novel paradigm to achieve multi-party collaborative learning without compromising remote sensing data privacy. In this paper, we formulate a novel personalized multi-party few-shot learning (PMPFSL) paradigm for remote sensing scene classification. In PMPFSL, different participants can achieve multi-party collaborative learning without sacrificing the privacy of their local data, and their respective local models are able to recognize the unseen remote sensing scene categories with a small number of labeled samples. Importantly, the proposed PMPFSL is applicable to various multi-party learning algorithms and few-shot scene classification networks. Moreover, to address the problems of local model overfitting and poor discriminability of few-shot metrics, we propose the personalized adaptive distillation (PAD) scheme and multi-scale feature matching network (MSFMNet) on PMPFSL, respectively. Specifically, each participant obtains the MSFMNet with initialization parameters, and implements a certain number of local training on their respective private machines. Global aggregation is subsequently achieved by uploading only the local models to the central server. In a new round of local training, the participants realize personalized data adaptation to the global model based on the PAD. Overall, the proposed PMPFSL customizes a personalized few-shot model for each participant that is more tailored to their respective remote sensing scenarios. The experimental results demonstrate that our PMPFSL is superior in three benchmark FSSC datasets. We also extensively studied and analyzed the contributions of PAD and MSFMNet in the proposed PMPFSL framework.

Remote sensing scene classification (RSSC) aims to understand and analyze the semantic information at the scene level with complex geographical properties. Despite the profound success of advanced deep models in automatically capturing hierarchical embedding representations and the gradual dominant trend in RSSC, it still remains a great challenge to precisely focus on targets at variable scales that are considered highly relevant to the corresponding scene and separated from the background. Motivated by this recognition, in this article, we present the gradient-guided multiscale focal attention network (GMFANet) for RSSC to adaptively localize the representative multiscale semantic representation for complex scenes. In particular, a lightweight parameterized hierarchical multiscale attention (HMA) mechanism is proposed, which constitutes the main aim of adaptively enhancing physical detail and high-level semantic information at different layers, rather than regarding each scale set with equivalent insight, while eliminating redundant information inherent in conventional attention mechanisms. Subsequently, a gradient-guided spatial focused attention (GSFA) module is specifically designed to accurately localize critical regions at multiple scales, with the dynamic combination of gradient-activated reference attention map and prediction attention map from supervised information-based learning. In addition, a curriculum-driven dynamic attention fusion (CDAF) strategy is tailored to fuse the spatial attention above from easy to hard for avoiding from poor local optimum and decreasing the early learning ambiguity. Our extensive comparative experiments and ablation analyses implemented on real-world public RSSC datasets indicate that our approach achieves the state-of-the-art performance exactly. The code is available athttps://github.com/bling2beyond/GMFANet.

End-to-end deep clustering method utilizes deep neural networks to jointly learn representation features and clustering assignments. Although many k-means-friendly deep clustering models have been explored, the existing division-based methods tend to directly implement clustering with a specific number of clusters, which suffers from poor performance resulted from indistinguishable clusters, and contributes to bad local optimum. At the same time, the representation learning of fuzzy$c$-means clustering in the feature space still needs more research. In this article, a deep fuzzy curriculum clustering method with the learning strategy of clustering from easy to complex automatically is proposed to tackle the above issues. First, considering the soft flexible allocation of fuzzy$c$-means and the preservation of local structure of original data, the fuzzy clustering loss and the autoencoder's reconstruction loss are constructed to learn the embedded features and clustering centers simultaneously. Second, curriculum loss is introduced into the constraint to make clusters successively merge in line with implementing clustering from easy to complex, and realize the bottom-up deep aggregative clustering automatically. In addition, novel curriculum information is proposed as constraint to guide the merging of clusters belonging to the same class. Experimental results on four real-world datasets show the superiority of the proposal.