Scene classification of very high resolution (VHR) images is an active research subject in remote sensing community, and it has provided data or decision supports for many practical applications. Although existing CNN-based methods have achieved good classification results, they have not fully exploited rich potential information contained in pre-trained models. In this paper, a novel framework termed hierarchical features fusion of convolutional neural network (HFFCNN) is developed for scene classification of VHR images. On the whole, the HFFCNN covers two parallel modules to severally process convolutional features and fully connected (FC) features. At the first module, an adaptive spatial-wise attention based multi-scale nonlinear bag-of-visual-words (ASA-MNBoVW) model is designed to encoding convolutional feature maps, and the responses of discriminative regions are highlighted without introducing any additional parameters. For the second module, a weighted image pyramid structure is adopted to reveal geometric information and spatial layouts by aggregating local image patch-based FC features. Finally, these hierarchical features are combined for mutually complementing, and a linear classifier is adopted to predict semantic labels. Experimental results organized on two challenging data sets prove that the developed HFFCNN approach obtains more dramatic performance of scene classification than some state-of-the-art methods in terms of OAs.

Scene classification is an important research topic in the field of remote sensing (RS), and deep features from convolutional neural networks (CNNs) have shown good classification performance. However, a key issue is how to effectively combine context features for further improving classification accuracy. In this letter, an end-to-end framework termed deep neural network combined with context features (CFDNN) is proposed for scene classification. At first, the pretrained VGG-16 is transferred as feature extractor to obtain convolutional features. Then, two parallel modules, global average pooling (GAP) and long short-term memory (LSTM), are employed to extract global features and context features, respectively. Finally, a weighted concatenation method is introduced to combine the global and context features. As a result, the CFDNN method can adapt high spatial resolution (HSR) images with arbitrary size and obtain satisfactory classification accuracy. The experimental results on the aerial image data set (AID) demonstrate that the proposed CFDNN method has competitive classification performance compared with some state-of-the-art methods.

Scene classification is an indispensable part of remote sensing image interpretation, and various convolutional neural network (CNN)-based methods have been explored to improve classification accuracy. Although they have shown good classification performance on high-resolution remote sensing (HRRS) images, discriminative ability of extracted features is still limited. In this letter, a high-performance joint framework combined CNNs and vision transformer (ViT) (CTNet) is proposed to further boost the discriminative ability of features for HRRS scene classification. The CTNet method contains two modules, including the stream of ViT (T-stream) and the stream of CNNs (C-stream). For the T-stream, flattened image patches are sent into pretrained ViT model to mine semantic features in HRRS images. To complement with T-stream, pretrained CNN is transferred to extract local structural features in the C-stream. Then, semantic features and structural features are concatenated to predict labels of unknown samples. Finally, a joint loss function is developed to optimize the joint model and increase the intraclass aggregation. The highest accuracies on the aerial image dataset (AID) and Northwestern Polytechnical University (NWPU)-RESISC45 datasets obtained by the CTNet method are 97.70% and 95.49%, respectively. The classification results reveal that the proposed method achieves high classification performance compared with other state-of-the-art (SOTA) methods.

Scene classification of high-resolution images is an active research topic in the remote sensing community. Although convolutional neural network (CNN)-based methods have obtained good performance, large-scale changes of ground objects in complex scenes restrict the further improvement of classification accuracy. In this letter, a global–local dual-branch structure (GLDBS) is designed to explore discriminative features of the original images and the crucial areas, and the strategy of decision-level fusion is applied for performance improvement. To discover the crucial area of the original image, the energy map generated by CNNs is transformed to the binary image, and the coordinates of the maximally connected region can be obtained. Among them, two shallow CNNs, ResNet18 and ResNet34, are selected as the backbone to construct a dual-branch network, and a joint loss is designed to optimize the whole model. In the GLDBS, the two streams employ the same structure (ResNet18-ResNet34) as the backbone, while the parameters are not shared. Experimental results on the aerial image data set (AID) and NWPU-RESISC45 datasets prove that the proposed GLDBS method achieves remarkable classification performance compared with some state-of-the-art (SOTA) methods. The highest overall accuracies (OAs) on the AID and NWPU-RESISC45 datasets are 97.01% and 94.46%, respectively.

Hyperspectral images (HSIs) not only possess abundant spectral features but also present a detailed spatial distribution of land cover, and they have significant advantages in the fine classification of ground materials. Recently, using convolutional neural networks (CNNs) to extract spectral–spatial features has become an effective way for HSI classification. However, conventional convolution kernels learn features from fixed regular square regions, and rich spatial information has not been effectively explored. In this letter, an end-to-end model named spectral–spatial residual graph attention network (S2RGANet) is developed for HSI classification, and it has two crucial elements, including spectral residual and graph attention convolution modules. At first, two spectral residual modules are employed to capture discriminant spectral features. Then, graphs are constructed to reveal the relationship between points in local neighborhoods. By graph attention mechanism, local spatial information is adaptively aggregated from neighboring nodes. Experiments on two public HSI datasets demonstrate that the S2RGANet is significantly superior to some state-of-the-art (SOTA) methods with limited training samples.

Scene classification is an active research topic in the remote sensing community, and complex spatial layouts with various types of objects bring huge challenges to classification. Convolutional neural network (CNN)-based methods attempt to explore the global features by gradually expanding the receptive field, while long-range contextual information is ignored. Vision transformer (ViT) can extract contextual features, but the learning ability of local information is limited, and it has a large computational complexity simultaneously. In this article, an end-to-end method is exploited by employing ViT as an excellent teacher for guiding small networks (ET-GSNet) in the remote sensing image scene classification. In the ET-GSNet, ResNet18 is selected as the student model, which integrates the superiorities of the two models via knowledge distillation (KD), and the computational complexity does not increase. In the KD process, the ViT and ResNet18 are optimized together without independent pretraining, and the learning rate of teacher model gradually decreases until zero, while the weight coefficient of the KD loss module is doubled. Based on the above procedures, dark knowledge from the teacher model can be transferred to the student model more smoothly. Experimental results on the four public remote sensing datasets demonstrate that the proposed ET-GSNet method possesses the superior classification performance compared to some state-of-the-art (SOTA) methods. In addition, we evaluate the ET-GSNet on a fine-grained ship recognition dataset, and the results show that our method has good generalization for different tasks in terms of some metrics.

Scene classification of high spatial resolution (HSR) images can provide data support for many practical applications, such as land planning and utilization, and it has been a crucial research topic in the remote sensing (RS) community. Recently, deep learning methods driven by massive data show the impressive ability of feature learning in the field of HSR scene classification, especially convolutional neural networks (CNNs). Although traditional CNNs achieve good classification results, it is difficult for them to effectively capture potential context relationships. The graphs have powerful capacity to represent the relevance of data, and graph-based deep learning methods can spontaneously learn intrinsic attributes contained in RS images. Inspired by the abovementioned facts, we develop a deep feature aggregation framework driven by graph convolutional network (DFAGCN) for the HSR scene classification. First, the off-the-shelf CNN pretrained on ImageNet is employed to obtain multilayer features. Second, a graph convolutional network-based model is introduced to effectively reveal patch-to-patch correlations of convolutional feature maps, and more refined features can be harvested. Finally, a weighted concatenation method is adopted to integrate multiple features (i.e., multilayer convolutional features and fully connected features) by introducing three weighting coefficients, and then a linear classifier is employed to predict semantic classes of query images. Experimental results performed on the UCM, AID, RSSCN7, and NWPU-RESISC45 data sets demonstrate that the proposed DFAGCN framework obtains more competitive performance than some state-of-the-art methods of scene classification in terms of OAs.

In this paper, we develop a CNN-GCN joint network (CGJNet) to learn global scene features and context information of high resolution remote sensing (HRRS) images. The proposed CGJNet method is composed of two streams, including CNN-stream (C-stream) and GCN-stream (G-stream). In the C-stream, a variation of the DenseNet-121 is developed to describe global visual information of HRRS images. In the G-stream, a GCN model is designed to reveal spatial structure by constructing adjacency graphs. As a result, accuracy of scene classification is effectively improved via integrating the two parts of crucial information. Experimental results on the AID data set demonstrate that the proposed CGJNet framework achieves remarkable classification results compared with many state-of-the-art (SOTA) methods, and the highest OA reaches 97.14%.

The scene classification of high spatial resolution (HSR) images is a challenging task in the remote sensing community. How to construct a discriminative representation of the HSR scene is a key step to improve classification performance. In this letter, we propose a novel feature extraction method termed multilayer feature fusion network (MF2Net) for scene classification. At first, the transferred VGGNet-16 model is employed as a feature extractor to acquire multilayer convolutional features. Then, several layers including pooling, transformation, and fusion layers are designed to process hierarchical features in four branches, and the prediction probability can be obtained for classification. Finally, the proposed model is optimized by fine-tuning techniques, where a novel data augmentation approach is explored to improve generalization ability. As a result, MF2Net effectively applies useful information from multilayers to improve the accuracy of scene classification. The experimental results on AID and NWPU-RESISC45 data sets exhibit that the MF2Net method obtains quite competitive classification results compared with many state-of-the-art methods.