Metaverse environments offer immersive, multimedia-rich experiences with growing relevance in education, entertainment, and cultural applications. The ability to grasp the contents of these environments, consisting of many rooms or subspaces, is a key building block for implementing Metaverse retrieval systems. However, current methods remain limited, as they are not designed to separate local (e.g., individual multimedia elements, and room-level details) from global, Metaverse-level semantics. Moreover, public datasets on similar topics do not capture the complexity of multi-room environments filled with multimedia contents. Our contributions are twofold. First, we introduce HiCALM, a hierarchical Metaverse retrieval framework built around the structural hierarchy of Metaverse environments. HiCALM models Metaverses in a bottom-up way, progressively grasping local and global semantics. To reduce the gap with textual data and achieve high-performance text-to-Metaverse retrieval, a novel cross-modal hierarchical loss supervises the process by teaching the model to associate the hierarchical visual features with textual information, also extracted hierarchically. Second, to overcome the absence of suitable datasets for the task, we present Museums3k, a large-scale dataset of 3,000 virtual museums annotated with detailed descriptions, each composed of multiple rooms populated with diverse multimedia elements; and GamingMV, a smaller dataset with data coming from 239 gaming-related real-world Metaverses. Through extensive quantitative and qualitative experiments, we show that HiCALM achieves considerable improvements in text-to-Metaverse retrieval, obtaining up to 95.0% R@1 and 60.0% \(\text{nDCG}_{3}@10\) on Museums3k (more than +40% R@1 and +11% nDCG, compared to existing solutions), and up to 62.0% R@1 and 62.6% \(\text{nDCG}_{3}@10\) on GamingMV (more than +23% R@1 and +5% nDCG).

This reproducibility Companion paper supports the approach presented in our paper titled ''AdOCTeRA: Adaptive Optimization Constraints for Improved Text-Guided Retrieval of Apartments.'' In that work, we addressed the problem of apartment retrieval using textual descriptions. Specifically, we proposed a novel adaptive approach that leverages the similarity between apartment descriptions in the dataset to enforce different levels of distance-a small distance between similar elements, a slightly larger distance between less similar elements, and an even larger distance between dissimilar elements.

The Metaverse is growing rapidly, resulting in thousands of rich virtual universes. This results in a difficult search process for the user, making advanced search tools a necessity. Existing methods leverage contrastive learning to obtain a function mapping a 3D scene and its textual descriptions into similar representations. However, Metaverse scenarios are complex, multimedia-rich 3D scenes containing many elements, making cross-modal alignment difficult. For instance, a museum dedicated to Van Gogh is unrelated to Warhol, yet it shares similarities with Matisse or Monet. To make the mapping functions aware of these nuances, we propose a novel learning strategy to integrate Adaptive Optimization Constraints, computing data-dependent distances using a language-based method we design and enforcing them between the representations at training time. This novelty sets our approach apart from standard procedures enforcing the same distance. We validate the effectiveness of two datasets, one including 6000 apartments, and a novel dataset of 3000 museums that we collect. We observe consistent improvements compared to existing methods. Moreover, we obtain better generalization when with very complex scenarios, e.g. on the museums dataset it obtains an average R@1 of 5.2% compared to 1.2% obtained by existing methods. Finally, the source code is available athttps://github.com/aliabdari/ALCER3D.

Nowadays, it is common for workers to relocate to new countries while seeking better job opportunities, or to live as digital nomads. While doing so, they face the problem of finding a new place to call home, requiring them to trust online advertisements or to physically visit the apartment. Recently, the research community investigated the possibility of performing the search on the Metaverse, hence reducing time and costs related to traveling and limiting carbon emissions. The methods available are based on state-of-the-art cross-modal retrieval techniques, which learn a joint embedding space by mapping apartment-descriptions pairs close. However, these methodologies push all the other pairs far away in the embedding space. In this paper, we identify this decision as a limitation, since different apartments are likely to share many aspects. To overcome it, we propose AdOCTeRA, which automatically separates the apartments into three classes -- very similar, slightly similar, and dissimilar -- and proposes adaptive optimization constraints for each of them. We validate our methodology on a large dataset of more than 6000 apartments, obtaining considerable relative improvements over the previous state-of-the-art (+3.8% R@5 and +7.3% R@10), and consistent improvements over the baseline across all the experiments. The source code is available at \hrefhttps://github.com/aliabdari/AdOCTeRA https://github.com/aliabdari/AdOCTeRA