Adversarial examples pose many security threats to convolutional neural networks (CNNs). Most defense algorithms prevent these threats by finding differences between the original images and adversarial examples. However, the found differences do not contain features about the classes, so these defense algorithms can only detect adversarial examples without recovering the correct labels. In this regard, we propose the Adversarial Feature Genome (AFG), a novel type of data that contain both the differences and features about classes. This method is inspired by an observed phenomenon, namely, the Adversarial Feature Separability, where the difference between the feature maps of the original images and adversarial examples becomes larger with deeper layers. On top of that, we further develop an adversarial example recognition framework that detects adversarial examples and can recover the correct labels. In the experiments, the detection and classification of adversarial examples by AFGs has an accuracy of more than 90.01% in various attack scenarios. To the best of our knowledge, our method is the first method that focuses on both attack detecting and recovering. AFG gives a new data-driven perspective to improve the robustness of CNNs.

Convolutional neural networks (CNNs) have achieved remarkable results in semantic segmentation of high-resolution remote sensing images (HRRSIs). However, the scales and textures of HRRSIs are diverse, which makes it difficult for a fixed-layer CNN to obtain rich features. In this regard, we propose an end-to-end ensemble fully convolutional network (EFCNet), which mainly includes two modules: the adaptive fusion module (AFM) and the separable convolutional module (SCM). The AFM can fuse features of different scales based on ensemble learning, whereas the SCM can reduce the complexity of the model under multifeature fusion. In the experiment, we use UNet and PSPNet to verify the framework on the ISPRS Vaihingen and Potsdam datasets. The experimental results show that the EFCNet can effectively improve the final segmentation performance and reduce the complexity of the ensemble model.

Adversarial examples fool the models into predicting wrong results through generated perturbations, demonstrating the vulnerability of convolutional neural networks (CNNs). Recent studies also show that many CNNs applied to remote sensing image (RSI) scene classification are still subject to adversarial example attacks. Through further analysis of adversarial examples of RSIs, it is found that the misclassified classes are not random, and these adversarial examples have demonstrated attack selectivity. Based on this finding, we propose a soft threshold defense method. First, we take the images with the correct prediction of each class as positive samples and adversarial examples as negative samples. Then, we take their output confidence as input and get the decision boundaries by the logistic regression algorithm. Finally, the confidence threshold of each class can be further obtained based on the decision boundary. It is the soft threshold used for defense, which can determine whether the image is an adversarial example or not. When the model predicts the new RSI, the input is an original image if the output confidence is higher than the soft threshold of the corresponding class, and the opposite is an adversarial example. Our proposed algorithm does not require modification of the model structure and is computationally uncomplicated, and it is simple and effective. For the FGSM, BIM, Deepfool, and C&W attack algorithms, their fooling rates are reduced by an average of 97.76%, 99.77%, 68.18%, and 97.95% in several scenarios. The soft threshold defense method can effectively defend against adversarial examples.

Convolutional neural networks (CNNs) have recently been widely used in remote-sensing scene classification. Additionally, it is becoming very popular to automatically learn specific CNN architectures for specific data sets. The rich contextual information in high-resolution remote-sensing images (RSIs) is critical to remote-sensing intelligent understanding tasks. However, architecture learning approaches tend to simplify the original data (i.e., resizing images to smaller resolution) for efficiency, yet result in contextual information loss of RSIs. In this article, we proposed a contextual information-preserved architecture learning (CIPAL) framework for remote-sensing scene classification to utilize the contextual information in RSIs as much as possible during the architecture learning process. We introduce channel compression into CIPAL, which can reduce the memory and time consumption of architecture learning and make it possible to construct a larger architecture space. We add potential operators that are rarely used for scene classification tasks (i.e., atrous convolution) into the architecture space to explore unknown architectures that are more suitable for remote-sensing scenes. The experimental results on four remote-sensing scene classification benchmarks indicate that CIPAL learns architectures with less time consumption than similar works, and the newly found architectures outperform popular hand-designed architectures for better use of contextual information in RSIs. Different architectures are good at learning different representations, and our proposed architecture learning method potentially helps us understand which types of representations are crucial for RSI intelligent understanding.

Few-shot remote sensing scene classification tries to make a model quickly adapt to new scenes with only a few samples that do not appear in the closed training set. Since limited samples can hardly describe the distribution of data, it is a challenge for a model to learn good generalized features. Since limited samples are rarely representative, it is another challenge for a model to learn classification boundaries that depend on sample bias. Therefore, we propose a method called metakernel networks (MKNs) to solve the challenges via integrating a parametric linear classifier (PLC) into the metalearning framework to address the former problem assembling a metakernel strategy (MKS) and a stretching loss (Stloss) to address the later problem. The PLC learns prior knowledge from different tasks sampled from the same task family to learn rich features. The MKS is designed to remap low-dimensional indistinguishable features to a high-dimensional space to solve the low-dimensional feature entanglement caused by large intraclass differences between remote sensing images. The Stloss reduces the dependence of the hyperplane formed by a few points in each class on sample selection by reducing the intraclass and interclass variance ratios, thus solving the boundary fragility problem caused by interclass similarity. Experiments on three public datasets, UC_Merced, NWPU-RESISC45, and AID, show the state-of-the-art performance by the accuracy improvement of 5.04%, 4.81%, and 4.69% respectively. Our findings provide a new perspective by suggesting that the previously neglected issue of classification boundaries may be a key factor for few-shot remote sensing scene classification.

High-resolution remote sensing images (HRRSIs) contain substantial ground object information, such as texture, shape, and spatial location. Semantic segmentation, which is an important task for element extraction, has been widely used in processing mass HRRSIs. However, HRRSIs often exhibit large intraclass variance and small interclass variance due to the diversity and complexity of ground objects, thereby bringing great challenges to a semantic segmentation task. In this letter, we propose a new end-to-end semantic segmentation network, which integrates lightweight spatial and channel attention modules that can refine features adaptively. We compare our method with several classic methods on the ISPRS Vaihingen and Potsdam data sets. Experimental results show that our method can achieve better semantic segmentation results. The source codes are available at https://github.com/lehaifeng/SCAttNet.

Deep neural networks (DNNs), which learn a hierarchical representation of features, have shown remarkable performance in big data analytics of remote sensing. However, previous research indicates that DNNs are easily spoofed by adversarial examples, which are crafted images with artificial perturbations that fool DNN models toward wrong predictions. To comprehensively evaluate the impact of adversarial examples on the remote sensing image (RSI) scene classification, this study tests eight state-of-the-art classification DNNs on six RSI benchmarks. These data sets include both optical and synthetic-aperture radar (SAR) images of different spectral and spatial resolutions. In the experiment, we create 48 classification scenarios and use four cutting-edge attack algorithms to investigate the influence of the adversarial example on the classification of RSIs. The experimental result shows that the fooling rates of the attacks are all over 98% across the 48 scenarios. We also find that, for the optical data, the seriousness of the adversarial problem has a negative relationship with the richness of the feature information. Besides, adversarial examples generated from SAR images are used easily for fooling the models with an average fooling rate of 76.01%. By analyzing the class distribution of these adversarial examples, we find that the distribution of the misclassifications is not affected by the types of models and attack algorithms-adversarial examples of RSIs of the same class cluster on fixed several classes. The analysis of classes of adversarial examples not only helps us explore the relationships between data set classes but also provides insights for further designing defensive algorithms.

Training a modern deep neural network on massive labeled samples is the main paradigm in solving the scene classification problem for remote sensing, but learning from only a few data points remains a challenge. Existing methods for a few-shot remote sensing scene classification are performed in a sample-level manner, resulting in easy overfitting of learned features to individual samples and inadequate generalization of learned category segmentation surfaces. To solve this problem, learning should be organized at the task level rather than the sample level. Learning on tasks sampled from a task family can help tune learning algorithms to perform well on new tasks sampled in that family. Therefore, we propose a simple but effective method, called RS-MetaNet, to resolve the issues related to few-shot remote sensing scene classification in the real world. On the one hand, RS-MetaNet raises the level of learning from the sample to the task by organizing training in a metaway, and it learns to learn a metric space that can well classify remote sensing scenes from a series of tasks. We also propose a new loss function, called balance loss, which maximizes the generalization ability of the model to new samples by maximizing the distance between different categories, providing the scenes in different categories with better linear segmentation planes while ensuring model fit. The experimental results on three open and challenging remote sensing data sets, UCMerced_LandUse, NWPU-RESISC45, and Aerial Image Data, demonstrate that our proposed RS-MetaNet method achieves state-of-the-art results in cases where there are only 1 ~ 20 labeled samples.