Existing cross-domain image retrieval (CDIR) methods exhibit a strong dependency on prior knowledge of training categories, which leads to problems of class confusion and domain shift when encountering unseen categories in open-set environments. In this paper, we explore the CDIR task towards open-set environments and introduce the Hypergraph-Based Remaining Prototype Alignment (RePro) framework for this task. Specifically, to address the problem of unseen class confusion caused by the category differences, we utilize the Remaining Prototype Embedding (RPE) module to generate the remaining embeddings of images and treat these embeddings as domain noise, rather than directly mapping them to the explicit domain-unified prototypes. To overcome the problem of domain shift, our method leverages the high-order correlations among both domains and categories through the Heterogeneous Structure Alignment (HSA) module, by constructing a heterogeneous hypergraph based on intra-domain and inter-category correlations. Besides, we build two multi-domain datasets for open-set cross-domain image retrieval,i.e., OCD-PACS and OCD-VLCS. Each dataset is divided into seen and unseen categories for training and testing, and each class has four different domains of images. Extensive experiments and ablation studies on these two datasets demonstrate the superiority of our method over current state-of-the-art methods.

Existing 3D cross-modal retrieval (3CMR) methods heavily rely on prior knowledge of training categories, which leads to the problem of modality shift and unseen center deviation when encountering unseen categories under the open-set environment. Aiming at the open-set 3CMR, this paper introduces the Hypergraph-Based Residual Fuzzy Alignment (ReFA) framework, which revisits the open-set retrieval task and navigates uncertainty of it through the lens ofFuzzy Theory. Facing the challenges of boundaryless space caused by uncertain unseen categories, we explore the representation and measurement in the fuzzy membership space as an alternative to fixed close-set category space. Specifically, to address the problem of modality shift caused by unseen categories, we utilize the Residual Sampling Generation (RSG) module to generate modality sampling embeddings that are independent of seen categories under the guidance of fuzzy representation, which residually decouples the entangled interactions of seen categories and modalities. To overcome the problem of unseen center deviation, we propose the Center Fuzzy Alignment (CFA) module to leverage the high-order fuzzy correlations for generalized metric, by constructing a fuzzy hypergraph based on the inherent and fuzzy correlations among both modalities and categories. The comprehensive evaluations of comparison and ablation studies on the four benchmarks demonstrate the superiority of our proposed framework compared to state-of-the-art methods.

Existing methods of 3D cross-modal retrieval heavily lean on category distribution priors within the training set, which diminishes their efficacy when tasked with unseen categories under open-set environments. To tackle this problem, we propose the Structure-Aware Residual-Center Representation (SRCR) framework for self-supervised open-set 3D cross-modal retrieval. To address the center deviation due to category distribution differences, we utilize the Residual-Center Embedding (RCE) for each object by nested auto-encoders, rather than directly mapping them to the modality or category centers. Besides, we perform the Hierarchical Structure Learning (HSL) approach to leverage the high-order correlations among objects for generalization, by constructing a heterogeneous hypergraph structure based on hierarchical inter-modality, intra-object, and implicit-category correlations. Extensive experiments and ablation studies on four benchmarks demonstrate the superiority of our proposed framework compared to state-of-the-art methods.

In open-set environments, the introduction of unseen classes adds a third stage of data for incremental learning, alongside seen classes of old and new. Existing methods by binary optimization between old and new classes struggle to model the complex triadic relationships among old, new, and unseen classes. In this paper, we introduce the Triadic Elastic Structure Learning (TESR) framework for open-set incremental 3D object retrieval. Specifically, to overcome the Bi-Directional Catastrophic Forgetting (BDCF) problem in open-set incremental learning, we employ the Triadic Regularized Embedding (TRE) module. This module helps to globally preserve the triadic relationships among the three stages of classes, while simultaneously facilitating semantic-specific learning for new classes from a local perspective. To address the challenge of Bi-Directional Knowledge Transfer (BDKT), our method leverages high-order correlations among objects of different classes through the Elastic Structure Distillation (ESD) module, by constructing an elastic hypergraph based on incremental and global correlations. We construct four multi-modal datasets for this open-set incremental 3D object retrieval: OIES, OINT, OIMN, and OIAB. Extensive experiments and ablation studies on these four benchmarks demonstrate the superiority of our method over current state-of-the-art approaches.

Existing open-set learning methods consider only the single-layer labels of objects and strictly assume no overlap between the training and testing sets, leading to contradictory optimization for superposed categories. In this paper, we introduce a more practical Semi-Open Environment setting for open-set 3D object retrieval with hierarchical labels, in which the training and testing set share a partial label space for coarse categories but are completely disjoint from fine categories. We propose the Hypergraph-Based Hierarchical Equilibrium Representation (HERT) framework for this task. Specifically, we propose the Hierarchical Retrace Embedding (HRE) module to overcome the global disequilibrium of unseen categories by fully leveraging the multi-level category information. Besides, tackling the feature overlap and class confusion problem, we perform the Structured Equilibrium Tuning (SET) module to utilize more equilibrial correlations among objects and generalize to unseen categories, by constructing a superposed hypergraph based on the local coherent and global entangled correlations. Furthermore, we generate four semi-open 3DOR datasets with multi-level labels for benchmarking. Results demonstrate that the proposed method can effectively generate the hierarchical embeddings of 3D objects and generalize them towards semi-open environments.

The lack of object-level labels presents a significant challenge for 3D object retrieval in the open-set environment. However, part-level shapes of objects often share commonalities across categories but remain underexploited in existing retrieval methods. In this paper, we introduce the Hypergraph-Based Assembly Fuzzy Representation (HARF) framework, which navigates the intricacies of open-set 3D object retrieval through a bottom-up lens of Part Assembly. To tackle the challenge of assembly isomorphism and unification, we propose the Hypergraph Isomorphism Convolution (HIConv) for smoothing and adopt the Isomorphic Assembly Embedding (IAE) module to generate assembly embeddings with geometric-semantic consistency. To address the challenge of open-set category generalization, our method employs high-order correlations and fuzzy representation to mitigate distribution skew through the Structure Fuzzy Reconstruction (SFR) module, by constructing a leveraged hypergraph based on local certainty and global uncertainty correlations. We construct three open-set retrieval datasets for 3D objects with part-level annotations: OP-SHNP, OP-INTRA, and OP-COSEG. Extensive experiments and ablation studies on these three benchmarks show our method outperforms current state-of-the-art methods.