Recent open-vocabulary human-object interaction (OV-HOI) detection methods primarily rely on large language model (LLM) for generating auxiliary descriptions and leverage knowledge distilled from CLIP to detect unseen interaction categories. Despite their effectiveness, these methods face two challenges: (1) feature granularity deficiency, due to reliance on last layer visual features for text alignment, leading to the neglect of crucial object-level details from intermediate layers; (2) semantic similarity confusion, resulting from CLIP’s inherent biases toward certain classes, while LLM-generated descriptions based solely on labels fail to adequately capture inter-class similarities. To address these challenges, we propose a stratified granular comparison network. First, we introduce a granularity sensing alignment module that aggregates global semantic features with local details, refining interaction representations and ensuring robust alignment between intermediate visual features and text embeddings. Second, we develop a hierarchical group comparison module that recursively compares and groups classes using LLMs, generating fine-grained and discriminative descriptions for each interaction category. Experimental results on two widely-used benchmark datasets, SWIG-HOI and HICO-DET, demonstrate that our method achieves state-of-the-art results in OV-HOI detection. Codes is available at GitHub.

Deep metric learning (DML) has shown promising results in few-shot hyperspectral image (HSI) classification. The core idea of DML is to learn a generalized metric space, in which pixels from unseen classes can be effectively classified with only a few labeled samples. However, the existing DML methods mainly adopt traditional Euclidean distance to achieve the feature metric, which ignores the category uncertainty of spatial-spectral features in mixed and edge pixels. To address this issue, we fully exploit fuzzy logic theory and propose a deep fuzzy metric learning (DFML) method for few-shot HSI classification. First, we design a novel hybrid CNN-transformer spatial-spectral feature extraction network to fully capture the spatial-spectral features of HSI pixels. Then, a fuzzy set representation method based on Gaussian membership function for spatial-spectral features is proposed, which describes the inherent fuzziness of the spatial-spectral features. Finally, to perform the fuzzy similarity measure between the fuzzy sets of query samples and prototypes, we construct a spatial-spectral fuzzy metric space, in which HSI pixels with category uncertainty in their features can be better classified under the condition of small-scale labeled samples. Extensive experimental results on three public HSI datasets demonstrate that the proposed DFML method outperforms the state-of-the-art few-shot HSI classification methods.

Due to the complex environment of hyperspectral image (HSI) gathering area, it is difficult to obtain a large number of labeled samples for HSI. Therefore, how to effectively achieve the HSI few-shot classification is a hot spot of current research. Prototypical network (PN) is one of the most classical few-shot learning algorithms, which has been widely employed for few-shot image classification and few-shot object detection. However, existing PN-based algorithms for HSI only utilize the single-scale spatial-spectral feature extracted from the last layer, ignoring the semantic information with different scales contained in the other layers. To solve this problem, a novel multi-scale spatial-spectral prototypical network (MSSPN) is proposed in this letter. The contribution of this letter is threefold. Firstly, a multi-scale spatial-spectral feature extraction algorithm based on ladder structure is proposed to effectively achieve the integration of spatial-spectral features with different scales. Secondly, with the theory of ladder-structure-based extraction algorithm, we design a multi-scale spatial-spectral prototype representation, which is suggested to be more robust and effective in the multi-scale spatial-spectral metric space. Finally, our proposed MSSPN has the advantage of expandability, and can be easily applied for the other PN-based few-shot learning methods. The experimental results on HSI few-shot classification indicate that our proposed MSSPN algorithm can achieve higher accuracy than the representative HSI classifiers and the existing PN-based algorithms.

For the large amount of spatial and spectral information contained in hyperspectral image (HSI), feature description of HSI has attracted widespread concern in recent years. Existing deep learning-based HSI feature description algorithms require a large number of training samples and have poor interpretability. Therefore, it is necessary to develop an efficient HSI features description algorithm with interpretability based on machine learning. Local binary pattern (LBP) is a classical descriptor used to extract the local spatial texture features of images, which has been widely applied to image feature description and matching. However, the existing LBP algorithms for HSI are based on the single-dimensional description, which leads to the limitations on the expression of spatial–spectral information. Therefore, a multidimensional LBP (MDLBP) based on Clifford algebra for HSI is proposed in this article, which is able to extract spatial–spectral feature from multiple dimensions. First, with the theory of the Clifford algebra, a new representation of HSI including spatial and spectral information is built. Second, the geometric relationship between the local geometry of HSI in Clifford algebra space is calculated to realize the local multidimensional description of the local spatial–spectral information. Finally, a novel LBP coding algorithm for HSI is implemented based on the local multidimensional description to calculate the feature descriptor of HSI. The experimental results on HSI classification show that our proposed MDLBP algorithm can achieve higher accuracy than the representative spatial–spectral features and the existing LBP algorithms, especially in the scenery of small-scale training samples.

Hyperspectral remote sensing image (HRSI) can provide additional spectral information of objects and have been widely used in many fields. However, due to the complex environment of the HRSI gathering area, collecting the labeled samples of HRSI is time-consuming and labor-intensive. The scarcity of labeled samples is one of the major difficulties for HRSI analysis and processing. In this letter, a spatial-spectral prototypical network (SSPN) for HRSI is proposed for solving the problem of lack of labeled samples. The contribution of this letter is threefold. First, we design a novel local pattern coding algorithm to combine the spatial and spectral information of HRSI pixels based on spatial neighborhood correlation. Then, a spatial-spectral feature extraction algorithm based on 1-D convolutional neural network (1-D-CNN) is suggested to learn the spatial-spectral metric space where HRSI pixels can be correctly classified with only a few labeled samples. Finally, a novel prototype representation for HRSI in spatial-spectral metric space is proposed to better classify the mixed pixels existing in HRSI. The experimental results on three popular HRSI data sets demonstrate that the proposed SSPN is significantly better than the traditional algorithms.