Recently, with the rapid development of deep learning, the collaborative classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) image has become a research hotspot in remote sensing (RS) technology. However, existing methods either only consider complementary learning of spatial-domain information, or do not take into account the intrinsic dependencies between pixels and overlook the importance difference of pixels. In this letter, we propose a Mamba-Wavelet Cross-Modal Fusion Network with Graph Pooling (MW-CMFNet) for HSI and LiDAR joint classification. First, a Two-Branch Feature Extraction (TBFE) is used to extract spatial and spectral features. Then, in order to dig deeper into the complementary information of different modalities and fully fuse them under the guidance of frequency-domain information, a Mamba-Wavelet Cross-Modal Feature Fusion (MW-CMFF) Module is devised, it aims to utilize Mamba’s outstanding long-range modeling ability to learn complementary information in the spatial and frequency domains, Finally, the Graph Pooling module is designed to sense the intrinsic dependencies of neighbouring pixels and explore the importance difference of pixels, rather than assigning the same weight to different pixels. Experiments on the Houston2013 and Trento datasets show that the MW-CMFNet achieves higher classification accuracy compared to other state-of-the-art methods.

Recent studies have found that compared to single-modal data, the joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) multimodal data can utilize their complementary information to further improve the accuracy of land-cover classification. However, due to the significant differences between multimodal data, the complementarity among them is difficult to be fully exploited and utilized, and the features after fusion are not refined and optimized, which limits the further improvement of land-cover classification accuracy. To alleviate these issues, a novel Mamba Cross-Modal Information Fusion Self-Distillation (Mb-CMIFSD) model is designed. Specifically, Mb-CMIFSD first uses conventional convolutional neural networks (CNN) to transform each patch into a token sequence. Second, a Mamba Cross Modal Information Fusion (MCMIF) module is developed to combine cross-modal attention with bidirectional Mamba mechanism, which can better explore the complementarity of multimodal remote sensing (RS) data and obtain more discriminative multimodal fusion features. Finally, a Prototype Constrained Self-Distillation (PCSD) module is designed to utilize the constructed prototype orthogonal regularization knowledge distillation function to further refine cross-modal fusion features, thereby enhancing the robustness and adaptability of feature extraction. The experimental results on three benchmark HSI and LiDAR datasets show that the designed Mb-CMIFSD model has higher classification accuracy compared to other state-of-the-art methods, and the ablation experiments also confirm the positive effect of the designed two key modules.

Image recognition is an important task in computer vision with broad applications. In recent years, with the advent of deep learning, lightweight convolutional neural network (CNN) has brought new opportunities for image recognition, which allows high-performance recognition algorithms to run on resource-constrained devices with strong representation and generalization capabilities. This paper first presents an overview of several classical lightweight CNN models. Then, a comprehensive review is provided on recent image recognition techniques using lightweight CNN. According to the strategies applied to optimize image recognition performance, existing methods are classified into three categories: (1) model compression, (2) optimization of lightweight network, and (3) combining Transformer with lightweight network. In addition, some representative methods are tested on three commonly used datasets for performance comparison. Finally, technical challenges and future research trends in this field are discussed.

Transformer-based object detection models usually adopt an encoding-decoding architecture that mainly combines self-attention (SA) and multilayer perceptron (MLP). Although this architecture does not require nonmaximum suppression (NMS) and can really achieve end-to-end object detection, it also suffers from the disadvantage of insufficient multiscale object perception in the image, which leads to low accuracy in detecting small objects. Focusing on these issues, a new full-scale bidirectional interactive attention (FSBDIA) mechanism is constructed, thereby a novel hierarchical full-scale interactive transformer (HFSI-TF) model is designed for object detection in remote sensing image (RSI). First, in order to enhance the multiscale perception ability of the model, the FSBDIA mechanism is designed under the guidance of full-scale information. Then, based on FSBDIA, a hierarchical HFSI-TF encoder is constructed to interactively fuse multilayer feature maps layer by layer, thereby obtaining multiscale encoded features of RSI. Finally, a mixed cross attention (MCA) mechanism is also constructed, and an iterative decoding architecture is designed based on it to improve the accuracy of small object detection. Comparative experiments based on two benchmark datasets (i.e., DIOR and HRSC2016) show that the designed HFSI-TF model can effectively improve the accuracy of object detection in RSI, and the model we designed has superior performance compared to other state-of-the-art methods.

In the scene classification task of remote sensing image (RSI), in order to fully perceive multi-scale local objects in the image and explore their interdependencies to mine the scene semantics of RSI, this letter designs a novel Position-Sensitive Cross-Layer Interactive Transformer (PSCLI-TF) model to improve the accuracy of RSI scene classification. Firstly, ResNet50 is utilized as the backbone to extract the multi-layer feature maps of RSI. Then, in order to enhance the model’s position sensitivity to local objects in RSI, a new Position-Sensitive Cross-Layer Interactive Attention (PSCLIA) mechanism is designed, and based on it a novel PSCLI-TF encoder is constructed to perform layer-by-layer interactive fusion on the multi-layer feature maps to obtain the multi-granularity Cross-Layer Fusion (CLF) feature of RSI. Finally, a prototype-based self-supervised loss function is constructed to alleviate the semantic gap problem of "large intra-class variance and small inter-class variance" in RSI scene classification. Comparative experimental results based on three datasets (i.e., AID, NWPU and UCM) indicate that the classification performance of the designed PSCLI-TF model is highly competitive compared to other state-of-the-art methods.

In order to implement rapid retrieval of large-scale crime scene investigation shoeprint image (SPI) in the intelligent mobile terminal of Public Security Internet of Things (PSIoT), a novel dual knowledge distillation (DKD) network is designed by fusing spatial attention (SA) distillation and feature distillation to solve its problems of limited computing and storage capacity. First, ResNet50 was modified by adding a new SA module and hash layer as the teacher model, and a light convolutional neural network (CNN) with the same SA and hash layer is designed as the student model. Then, the attention distillation loss function is constructed to distill the SA knowledge in the teacher module to the student module to improve the ability of the convolutional layer at the front end of the student network to capture the underlying visual features of the SPI. Finally, the feature distillation loss function is constructed to distill the semantic knowledge in the teacher module to the student module to improve the ability of the student module to express high-level semantics of the SPI. we compare our method on two data sets of SPID and FID-300 with other state-of-the-art methods in the SPI retrieval domain. The experimental results show that our method can improve the SPI retrieval baseline by a large margin and better than other methods.

In the application of remote sensing image (RSI) scene classification, in order to solve the contradiction between the accuracy of Convolutional Neural Network (CNN) and the large amount of model parameters, a novel dual knowledge distillation (DKD) model combining dual attention (DA) and spatial structure (SS) is designed. First, new DA and SS modules are constructed and introduced into ResNet101 and light-weight CNN designed as teacher and student networks respectively. Then, in order to improve its local feature extraction and high-level semantic representation abilities for RSI by transmission the DA and SS knowledge in the teacher network to the student network, we design the corresponding DA and SS distillation losses. The comparative experimental results based on AID and NWPU-45 datasets show that when the training ratio is 20%, the accuracy of the student network after DKD is improved by 7.57% and 7.28% respectively, and in the case of fewer parameters, DKD has higher accuracy than most other methods.

In this paper, a novel multi-instance learning (MIL) algorithm based on pyramid match kernel (PMK) and classifier ensemble is proposed for recognizing pornographic scene from image database. First, an improved JSEG image segmentation technique is deployed for dividing every image into several regions, and regards the whole image as a "bag", the low-level visual features (i.e. color and texture) of each segmented region as "instance". As a result, the pornographic images filtering problem can be transferred into a typical MIL problem. Second, similarity between the multi-instance bags is measured by PMK method, which allows MIL problem to be solved directly by the support vector machine (SVM). Finally, many base classifiers based on PMK with different levels are constructed, and the performance weighting rule is used to dynamically determine the weights of them, so the strategy of classifier ensemble is used to improve the filtering accuracy. In a real condition image set that the ratio of normal image to pornographic image is 9:1, experimental results show that the proposed algorithm, named PMKCE-MIL, is robust, and its performance is superior to other algorithms.