Remote sensing (RS) image scene classification has wide applications in the field of RS. Although existing methods have achieved remarkable performance, there are still limitations in feature extraction and lightweight design. Current multi-branch models, although performing well, have large parameter counts and high computational costs, making them difficult to deploy on resource-constrained edge devices such as unmanned aerial vehicles (UAVs). On the other hand, lightweight models like StarNet, having less parameter, but rely on element-wise multiplication to generate features and lack the capture of explicit long-range spatial feature, resulting in insufficient classification accuracy. To address these issues, this letter proposes a lightweight mamba-based hybrid network, namely LMHMamba, whose core is an innovative lightweight multi-feature hybrid mamba (LMHM) module. This module combines the advantage of StarNet in implicitly generating high-dimensional nonlinear features, introduces a lightweight state space module to enhance spatial feature learning capabilities, and then uses local and global attention modules to emphasize local and global features. This enables effective multi-dimensional feature fusion while maintaining low parameter. We validate the performance of LMHMamba model on three remote sensing scene classification datasets and compare it with mainstream lightweight models and the latest methods. Experimental results show that LMHMamba achieves advanced levels in both classification accuracy and computational efficiency, significantly outperforming existing lightweight models, providing an efficient solution for edge deployment. Code is available at https://github.com/yizhilanmaodhh/LMHMamba.

Multi-view clustering (MVC) based on anchor learning has been proven to be effective in improving clustering accuracy and efficiency. Existing MVC methods are mainly based on single-granularity anchor learning, that is, the number of anchors corresponding to different views is constant and consistent, which will lead to information redundancy or insufficient mining. In addition, aggregating anchors of varying scales from all views to obtain multi-view shared clustering results remains a problem to be further explored. To address the above problems, a novel MVC method named View-adaptive Multi-granularity Anchor Learning (VMAL) is proposed in this paper, where view-adaptive anchor pruning and view-shared sample clustering are jointly optimized. On the one hand, VMAL can dynamically adjust the optimal number of anchors for each view during optimization by exploiting the reconstruction error of samples. On the other hand, an intuitive and effective mapping-aggregation message passing strategy is cleverly designed, which first maps the anchor representations of different views to the cluster space and then transfers the obtained cluster information of anchors to the sample space through an aggregation matrix. As a byproduct, VMAL can directly obtain the discrete cluster distribution of samples without additional partitioning. Finally, an iterative optimization algorithm is developed to solve the proposed VMAL method. Experimental results on multiple datasets have demonstrated the superiority of VMAL in terms of clustering results when compared with other state-of-the-art methods.

Extracting rotation-invariant local descriptors is vital for many downstream tasks, such as image matching and 3D reconstruction. However, learning-based methods designed specifically for arbitrary rotations have rarely been studied. Therefore, we propose a novel local descriptors learning method consisting of circular polar transform (CPT) and lightweight cylindrical convolution network (L2C-Net) in this paper. On the one hand, in order to better cope with rotation, CPT is first used to obtain representations of input patches in polar coordinate space, which converts a rotation problem into a translation problem suitable for convolutional networks. Compared with polar transform, CPT can achieve relatively uniform sampling from the center to the circumference along the radius direction of polar coordinates, thereby effectively reducing information loss. On the other hand, L2C-Net based on cylindrical convolution is designed to deal with the property of warp-around connections of the obtained polar representations. As a result, local descriptors that are invariant to arbitrary rotation angles can be obtained. Extensive experimental results have demonstrated that our proposed method can surpass the current state-of-the-art rotation-invariant descriptors under various rotations and also shows competitive performance on four different tasks.

Cross-view image geo-localization aims to estimate the geographic position of a query image from the ground platform (such as mobile phone, vehicle camera) by matching it with geo-tagged reference images from the aerial platform (such as drone, satellite). Although existing studies have achieved promising results, they usually rely only on depth features and fail to effectively handle the serious changes in geometric shape and appearance caused by view differences. In this paper, a novel Semantic Concept Perception Network (SCPNet) with interactive prompting is proposed, whose core is to extract and integrate semantic concept information reflecting spatial position relationship between objects. Specifically, for a given of pair input images, a CNN stem with positional embedding is first adopted to extract depth features. Meanwhile, a semantic concept mining module is designed to distinguish different objects and capture the associations between them, thereby achieving the purpose of extracting semantic concept information. Furthermore, to obtain global descriptions of different views, a feature bidirectional injection fusion module based on attention mechanism is proposed to exploit the long-range dependencies of semantic concept and depth features. Finally, a triplet loss with a flexible hard sample mining strategy is used to guide the optimization of the network. Experimental results have shown that our proposed method can achieve better performance compared with state-of-the-art methods on mainstream cross-view datasets.

Camouflaged object detection (COD) aims to discover objects that are seamlessly embedded in the environment. Existing COD methods have made significant progress by typically representing features in a discrete way with arrays of pixels. However, limited by discrete representation, these methods need to align features of different scales during decoding, which causes some subtle discriminative clues to become blurred. This is a huge blow to the task of identifying camouflaged objects from clear subtle clues. To address this issue, we propose a novel continuous feature representation network (CFRN), which aims to represent features of different scales as a continuous function for COD. Specifically, a Swin transformer encoder is first exploited to explore the global context between camouflaged objects and the background. Then, an object-focusing module (OFM) deployed layer by layer is designed to deeply mine subtle discriminative clues, thereby highlighting the body of camouflaged objects and suppressing other distracting objects at different scales. Finally, a novel frequency-based implicit feature decoder (FIFD) is proposed, which directly decodes the predictions at arbitrary coordinates in the continuous function with implicit neural representations, thus propagating clearer discriminative clues. Extensive experiments on four challenging COD benchmarks demonstrate that our method significantly outperforms state-of-the-art methods. The source code will be available at https://github.com/SongZeHNU/CFRN.

Open set recognition (OSR) aims to identify whether a test sample belongs to a semantic class in the classifier training set. Existing OSR methods exhibit prominent performance on various image datasets. However, they are primarily designed for general object recognition rather than more complex camouflaged object recognition. When an object is camouflaged, i.e., it exhibits a similar pattern to the background, it is difficult to finely identify it and differentiate between known and unknown categories. To address this problem, we propose a novel multi-granularity context perception network (MCPNet) for OSR of camouflaged objects, which can accurately identify camouflaged objects by fusing coarse-grained and fine-grained context features. In MCPNet, the vision transformer is first utilized to extract coarse-grained context features to locate the approximate location of camouflaged objects. Then, an adaptive local focus module (ALFM) is proposed to pick out the most discriminative regions and learn the fine-grained context of these regions. Finally, multi-granular context features are fused to obtain recognition results. During the training, a contrastive clustering module (CCM) is introduced to guide the network to effectively utilize multi-granularity context to generate high-confidence decision boundaries. We also built two camouflaged object classification datasets named ACOC and NCOC which mainly consist of artificial camouflage and natural camouflage respectively to facilitate research in OSR of camouflaged objects. Experimental results on two datasets show that MCPNet outperforms state-of-the art methods.

Real-time semantic segmentation plays an important role in auto vehicles. However, most real-time small object segmentation methods fail to obtain satisfactory performance on small objects, such as cars and sign symbols, since the large objects usually tend to devote more to the segmentation result. To solve this issue, we propose an efficient and effective architecture, termed small objects segmentation network (SOSNet), to improve the segmentation performance of small objects. The SOSNet works from two perspectives: methodology and data. Specifically, with the former, we propose a dual-branch hierarchical decoder (DBHD) which is viewed as a small-object sensitive segmentation head. The DBHD consists of a top segmentation head that predicts whether the pixels belong to a small object class and a bottom one that estimates the pixel class. In this situation, the latent correlation among small objects can be fully explored. With the latter, we propose a small object example mining (SOEM) algorithm for balancing examples between small objects and large objects automatically. The core idea of the proposed SOEM is that most of the hard examples on small-object classes are reserved for training while most of the easy examples on large-object classes are banned. Experiments on three commonly used datasets show that the proposed SOSNet architecture greatly improves the accuracy compared to the existing real-time semantic segmentation methods while keeping efficiency. The code will be available at https://github.com/StuLiu/SOSNet.

Bipartite graph (BiG) has been proven to be efficient in handling massive multiview data for clustering. However, how to regulate the structural information of view-specific anchors and view-shared BiG is still open and needs to be further studied. Hence, a novel dual-structural BiG learning (DsBiGL) method is proposed in the article. It transforms BiG learning into a joint optimization problem of IntrA-view and InteR-view subspace learning (IASL and IRSL) with the structural constraints, such as k-nearest neighbor (KNN) and low-rank. On one hand, IASL uses the KNN and view-specific low-rank constraints to enhance the discriminativeness of view-specific anchors. On the other hand, IRSL uses an adaptive weighting strategy to obtain view-shared BiG directly from multiview samples, where the KNN and view-shared low-rank constraints are adopted to encode local connectivity and cluster information between samples. Note that IASL and IRSL are integrated into a unified optimization model, which ensures the interactive enhancement of view-specific anchor representation and view-shared BiG learning. Finally, an algorithm based on iterative optimization is designed to solve the proposed DsBiGL model. Experimental results on various multiview datasets have demonstrated the superiority of DsBiGL in terms of clustering results when compared with other comparative methods.

Interactive image segmentation (IIS) has been widely used in various fields, such as medicine, industry, etc. However, some core issues, such as pixel imbalance, remain unresolved so far. Different from existing methods based on pre-processing or post-processing, we analyze the cause of pixel imbalance in depth from the two perspectives of pixel number and pixel difficulty. Based on this, a novel and unified Click-pixel Cognition Fusion network with Balanced Cut (CCF-BC) is proposed in this paper. On the one hand, the Click-pixel Cognition Fusion (CCF) module, inspired by the human cognition mechanism, is designed to increase the number of click-related pixels (namely, positive pixels) being correctly segmented, where the click and visual information are fully fused by using a progressive three-tier interaction strategy. On the other hand, a general loss, Balanced Normalized Focal Loss (BNFL), is proposed. Its core is to use a group of control coefficients related to sample gradients and forces the network to pay more attention to positive and hard-to-segment pixels during training. As a result, BNFL always tends to obtain a balanced cut of positive and negative samples in the decision space. Theoretical analysis shows that the commonly used Focal and BCE losses can be regarded as special cases of BNFL. Experiment results of five well-recognized datasets have shown the superiority of the proposed CCF-BC method compared to other state-of-the-art methods. The source code is publicly available at https://github.com/lab206/CCF-BC.

Image geo-localization aims to locate a query image from source platform (e.g., drones, street vehicle) by matching it with Geo-tagged reference images from the target platforms (e.g., different satellites). Achieving cross-modal or cross-view real-time (>30fps) image localization with the guaranteed accuracy in a unified framework remains a challenge due to the huge differences in modalities and views between the two platforms. In order to solve this problem, a novel fine-grained overlap estimation based image geo-localization method is proposed in this paper, the core of which is to estimate the salient and subtle overlapping regions in image pairs to ensure correct matching. Specifically, the high-level semantic features of input images are extracted by a deep convolutional neural network. Then, a novel overlap scanning module (OSM) is presented to mine the long-range spatial and channel dependencies of semantic features in various subspaces, thereby identifying fine-grained overlapping regions. Finally, we adopt the triplet ranking loss to guide the proposed network optimization so that the matching regions are as close as possible and the most mismatched regions are as far away as possible. To demonstrate the effectiveness of our FOENet, comprehensive experiments are conducted on three cross-view benchmarks and one cross-modal benchmark. Our FOENet yields better performance in various metrics and the recall accuracy at top 1 (R@1) is significantly improved, with a maximum improvement of 70.6%. In addition, the proposed model runs fast on a single RTX 6000, reaching real-time inference speed on all datasets, with the fastest being 82.3 FPS.

In recent years, deep-learning-based pixel-level unified image fusion methods have received more and more attention due to their practicality and robustness. However, they usually require a complex network to achieve more effective fusion, leading to high computational cost. To achieve more efficient and accurate image fusion, a lightweight pixel-level unified image fusion (L-PUIF) network is proposed. Specifically, the information refinement and measurement process are used to extract the gradient and intensity information and enhance the feature extraction capability of the network. In addition, these information are converted into weights to guide the loss function adaptively. Thus, more effective image fusion can be achieved while ensuring the lightweight of the network. Extensive experiments have been conducted on four public image fusion datasets across multimodal fusion, multifocus fusion, and multiexposure fusion. Experimental results show that L-PUIF can achieve better fusion efficiency and has a greater visual effect compared with state-of-the-art methods. In addition, the practicability of L-PUIF in high-level computer vision tasks, i.e., object detection and image segmentation, has been verified.

Camouflaged object detection (COD) aims to identify object pixels visually embedded in the background environment. Existing deep learning methods fail to utilize the context information around different pixels adequately and efficiently. In order to solve this problem, a novel pixel-centric context perception network (PCPNet) is proposed, the core of which is to customize the personalized context of each pixel based on the automatic estimation of its surroundings. Specifically, PCPNet first employs an elegant encoder equipped with the designed vital component generation (VCG) module to obtain a set of compact features rich in low-level spatial and high-level semantic information across multiple subspaces. Then, we present a parameter-free pixel importance estimation (PIE) function based on multiwindow information fusion. Object pixels with complex backgrounds will be assigned with higher PIE values. Subsequently, PIE is utilized to regularize the optimization loss. In this way, the network can pay more attention to those pixels with higher PIE values in the decoding stage. Finally, a local continuity refinement module (LCRM) is used to refine the detection results. Extensive experiments on four COD benchmarks, five salient object detection (SOD) benchmarks, and five polyp segmentation benchmarks demonstrate the superiority of PCPNet with respect to other state-of-the-art methods.

Hyperspectral image (HSI) classification methods have made great progress in recent years. However, most of these methods are rooted in the closed-set assumption that the class distribution in the training and testing stages is consistent, which cannot handle the unknown class in open-world scenes. In this work, we propose a feature consistency-based prototype network (FCPN) for open-set HSI classification, which is composed of three steps. First, a three-layer convolutional network is designed to extract the discriminative features, where a contrastive clustering module is introduced to enhance the discrimination. Then, the extracted features are used to construct a scalable prototype set. Finally, a prototype-guided open-set module (POSM) is proposed to identify the known samples and unknown samples. Extensive experiments reveal that our method achieves remarkable classification performance over other state-of-the-art classification techniques.

Hyperspectral oil spill mapping aims to distinguish the type of oil spill. Recently, most hyperspectral oil spill detection methods are based on supervised methods that work well with rich training samples. However, in the marine oil spill detection scenario, pixel annotations are difficult and costly. Moreover, the labels obtained by domain experts within a hyperspectral image (HSI) are often scarce. To address these issues, a self-supervised spectral-spatial transformer network is proposed for hyperspectral oil spill mapping. First, we propose a transformer-based contrastive learning network to extract the deep discriminative features. Then, the learned features are transferred to the downstream classification network that is fine-tuned with very few labeled samples. Experiments on hyperspectral oil spill database (HOSD) constructed by ourselves indicate that the proposed method can obtain more promising performance than several state-of-the-art oil spill classification techniques in discriminating different types of oil spills, i.e., thick oil, thin oil, sheen, and seawater.

Change detection (CD) aims to find areas of specific changes in multi-temporal remote sensing images. The existing methods fail to adequately explore the cross-temporal global context, making the establishment of spatial-temporal deep global associations insufficient and inefficient. As a result, their performance is vulnerable to complex and various objects in changing scenes. Hence, we propose a cross-temporal context learning network, termed as CCLNet, where the intra- and inter-temporal long-range dependency are mined and interactively fused, to fully exploit the cross-temporal context information. Specifically, a lightweight convolutional neural network is first used to extract deep semantic features. Then, a well-designed cross-temporal fusion transformer (CFT) is proposed to locate the changing objects in the scene by establishing the long-range dependency across bitemporal images. Thanks to this, the temporal-specific information extraction and cross-temporal information integration are seamlessly integrated into the same network, thereby significantly improving the discriminative features of changing objects. Furthermore, this allows us using naive backbones with low computational cost to achieve reliable CD performance. Experiments on mainstream benchmarks show that our proposed method can handle CD task faster than state-of-the-art methods while maintaining better or comparable matching accuracy on a single RTX3090.

Camouflaged object detection (COD) aims to discover objects that blend in with the background due to similar colors or textures, etc. Existing deep learning methods do not systematically illustrate the key tasks in COD, which seriously hinders the improvement of its performance. In this paper, we introduce the concept of focus areas that represent some regions containing discernable colors or textures, and develop a two-stage focus scanning network for camouflaged object detection. Specifically, a novel encoder-decoder module is first designed to determine a region where the focus areas may appear. In this process, a multi-layer Swin transformer is deployed to encode global context information between the object and the background, and a novel cross-connection decoder is proposed to fuse cross-layer textures or semantics. Then, we utilize the multi-scale dilated convolution to obtain discriminative features with different scales in focus areas. Meanwhile, the dynamic difficulty aware loss is designed to guide the network paying more attention to structural details. Extensive experimental results on the benchmarks, including CAMO, CHAMELEON, COD10K, and NC4K, illustrate that the proposed method performs favorably against other state-of-the-art methods.

Oil spill mapping is a very challenging problem in marine environmental monitoring. In this paper, the potential of hyperspectral image classification for mapping oil spills is comprehensively investigated. First, several representative hyperspectral image classification methods are reviewed in a general framework. Second, three oil spill mapping cases are designed to analyze the performance of different classification methods in detecting the spatial distribution, classifying the type, and estimating the thickness of oil spills. Finally, the experimental results are analyzed in detail, and some conclusions are given, which bring a comprehensive understanding to scholars who are interested in the fields of hyperspectral remote sensing and oil spill mapping.

In this paper, a novel unmanned aerial vehicle (UAV) hyperspectral image stitching framework based on radiation correction and seam-cutting blending is proposed. Firstly, spectral correlation constraints are introduced to eliminate mismatched pairs in the transform matrix estimation step. Then, a spectral correction method based on intrinsic images is proposed to ensure spectral consistency of stitching results. In order to obtain more natural stitching results without edge effect, a seam-cutting and multi-scale blending strategy is adopted in the final blending stage. Experimental results on real unmanned aerial vehicle hyperspectral strip images show that the proposed method is superior to a representative image stitching approach.

It is well known that classification accuracy highly relies on the number of labeled samples. However, it is difficult to obtain sufficient labeled samples in real-world applications. To solve this issue, a novel hyperspectral image (HSI) classification method based on polygonal partition is proposed for crop mapping. This method only needs single sample per class as an initial training set. Specifically, multiscale polygonal partition is applied on the first three components of the HSI. Then, a spectral similarity-based sample expansion method is proposed to obtain more labeled samples. Next, a pixel-wise classifier, the support vector machine (SVM), is used to acquire an initial classification result. Finally, classification result is further optimized according to the partition maps. Experimental results show that classification performance of the proposed method is satisfactory even when the number of labeled sample is single for each class.

For hyperspectral images (HSIs), it is a challenging task to select discriminative bands due to the lack of labeled samples and complex noise. In this article, we present a novel local-view-assisted discriminative band selection method with hypergraph autolearning (LvaHAl) to solve these problems from both local and global perspectives. Specifically, the whole band space is first randomly divided into several subspaces (LVs) of different dimensions, where each LV denotes a set of low-dimensional representations of training samples consisting of bands associated with it. Then, for different LVs, a robust hinge loss function for isolated pixels regularized by the row-sparsity is adopted to measure the importance of the corresponding bands. In order to simultaneously reduce the bias of LVs and encode the complementary information between them, samples from all LVs are further projected into the label space. Subsequently, a hypergraph model that automatically learns the hyperedge weights is presented. In this way, the local manifold structure of these projections can be preserved, ensuring that samples of the same class have a small distance. Finally, a consensus matrix is used to integrate the importance of bands corresponding to different LVs, resulting in the optimal selection of expected bands from a global perspective. The classification experiments on three HSI data sets show that our method is competitive with other comparison methods.

For applications based on hyperspectral imagery (HSI), selecting informative and representative bands without the degradation of performance is a challenging task in the context of big data. In this paper, an unsupervised band selection method, scalable one-pass self-representation learning (SOP-SRL), is proposed to address this problem by processing data in a streaming fashion without storing the entire data. SOP-SRL embeds band selection into a scalable self-representation learning, which is formulated as an adaptive linear combination of regression-based loss functions, with the row-sparsity constraint. To further enhance the representativeness of bands, the local similarity between samples constructed by the selected bands is dynamically measured by means of graph-based regularization term in the embedded space. Moreover, a cache with memory function that reflects the quality of bands in the historical data is designed to keep the consistency between data coming at different times and guide subsequent band selection. An efficient algorithm is developed to optimize the SOP-SRL model. The HSI classification is conducted on three public data sets, and the experimental results validate the superiority of SOP-SRL in terms of performance and time when compared with other state-of-the-art band selection methods.

For hyperspectral image classification, high-dimensional spectral features not only increase the computational and storage burden but also degrade the classification accuracy due to the Hughes phenomenon. Band selection is an important technique to solve these issues without destroying the interpretation of data. This paper presents a matrix-based margin-maximization method of band selection with data-driven diversity. In particular, the matrices composed of adjacent pixels in space are fed to the hinge loss function with a row-sparse constraint. This constraint is used to select the expected bands and preserve the spatial structure information simultaneously while maximizing the margin between the classes. In consideration of the continuity of bands in the spectral dimension, a novel regularization term that is continually updated according to the current context is added to promote the differential expression of dissimilar bands. Finally, the one-against-all parallel mechanism is used to learn a coefficient matrix for each class, and class-related bands are then carefully selected by a partitioning strategy (e.g., k-means clustering) based on the learned coefficient matrix. Experiments are conducted on three hyperspectral data sets and four widely used classifiers. The experimental results have shown that our proposed method is superior to several state-of-the-art methods, especially when the number of selected bands is relatively small.