In this paper, we introduce a brand-new localization pipeline designed to comprehensively leverage both hand-crafted and learned features, operating at two distinct levels (point-level and object-level) while simultaneously recovering scene scale from the RGB input. The pipeline integrates a learned globally consistent descriptor matching process for initial camera pose estimation, followed by a pose optimization phase that synergistically combines various features. To generate learned descriptors, we propose a siamese Globally Consistent Feature Descriptor Network (GCFDNet), which accepts a pair of images, Inertial Measurement Unit (IMU) data, and pose sequences as inputs, producing both the image descriptors and the relative camera pose as outputs. The strengths of GCFDNet manifest in two key aspects. First, by incorporating a spatial-to-temporal feature fusion module, GCFDNet enhances relative pose regression, meanwhile enabling accurate scene scale estimation. Second, we devise a loss function that balances descriptor similarity and distance, thereby improving the quality of descriptor learning. Using the initial camera poses derived from GCFDNet, we establish data associations across multiple frames and subsequently propose a combined Bundle Adjustment (BA) optimization framework that integrates hand-crafted features, learned descriptors, and semantic objects. To evaluate the localization performance, we conduct experiments across diverse datasets, including EuRoC, ScanNet, 7 Scenes, TUM RGB-D, and Bonn. The results demonstrate state-of-the-art performance in both static and dynamic scenes, outperforming existing methods. Additionally, we present ablation studies on GCFDNet and the combined BA process to further substantiate the efficacy of our approach.

With the rapid development of Virtual Reality (VR) and haptic technology, their applications in education have demonstrated tremendous potential. However, traditional VR learning that incorporates haptic feedback usually has a limited body coverage, and there is a weak alignment between the haptic feedback and the educational content. To address the above-mentioned issues, this study established an environment that combines VR with dynamic somatosensory feedback. Through the simulation of the human blood circulation, it explored the impacts of VR and somatosensory feedback on learning effects. Sixty participants were randomly divided into three groups: the computer-based group, the VR-only group, and the VR with somatosensory feedback group. Self-efficacy, perceived enjoyment, and knowledge retention were evaluated through targeted questionnaires. The results showed that VR with somatosensory feedback significantly outperformed the computer-based and VR-only learning modes. This study provides new insights and practical support for the effective integration of VR and somatosensory feedback technology in educational environments.

Recent advancements in event-based zero-shot object recognition have demonstrated promising results. However, these methods heavily depend on extensive training and are inherently constrained by the characteristics of CLIP. To the best of our knowledge, this research is the first study to explore the understanding capabilities of large language models (LLMs) for event-based visual content. We demonstrate that LLMs can achieve event-based object recognition without additional training or fine-tuning in conjunction with CLIP, effectively enabling pure zero-shot event-based recognition. Particularly, we evaluate the ability of GPT-4o / 4turbo and two other open-source LLMs to directly recognize event-based visual content. Extensive experiments are conducted across three benchmark datasets, systematically assessing the recognition accuracy of these models. The results show that LLMs, especially when enhanced with well-designed prompts, significantly improve event-based zero-shot recognition performance. Notably, GPT-4o outperforms the compared models and exceeds the recognition accuracy of state-of-the-art event-based zero-shot methods on N-ImageNet by five orders of magnitude.

Abstract This paper introduces a novel continual learning framework for synthesising novel views of multiple scenes, learning multiple 3D scenes incrementally, and updating the network parameters only with the training data of the upcoming new scene. We build on Neural Radiance Fields (NeRF), which uses multi‐layer perceptron to model the density and radiance field of a scene as the implicit function. While NeRF and its extensions have shown a powerful capability of rendering photo‐realistic novel views in a single 3D scene, managing these growing 3D NeRF assets efficiently is a new scientific problem. Very few works focus on the efficient representation or continuous learning capability of multiple scenes, which is crucial for the practical applications of NeRF. To achieve these goals, our key idea is to represent multiple scenes as the linear combination of a cross‐scene weight matrix and a set of scene‐specific weight matrices generated from a global parameter generator. Furthermore, we propose an uncertain surface knowledge distillation strategy to transfer the radiance field knowledge of previous scenes to the new model. Representing multiple 3D scenes with such weight matrices significantly reduces memory requirements. At the same time, the uncertain surface distillation strategy greatly overcomes the catastrophic forgetting problem and maintains the photo‐realistic rendering quality of previous scenes. Experiments show that the proposed approach achieves state‐of‐the‐art rendering quality of continual learning NeRF on NeRF‐Synthetic, LLFF, and TanksAndTemples datasets while preserving extra low storage cost.

Due to the constraints posed by the complexity and vast amounts of geospatial data, achieving fine-grained drought geospatial interpolation using sparse observation points has always been a challenging problem. Therefore, to solve the problem, this letter proposes a new model called geographical attention conditional neural processes (GACNPs) that extends spatial multiattention conditional neural processes (SMACNPs). GACNPs as a type of probabilistic model inherit the flexibility of neural networks in parameterizing stochastic processes. We design a geographic information embedding module by measuring the geographic information correlation between known observation points and target points, thereby better capturing the complexities in spatial data. Considering that real-world data distributions often do not conform to Gaussian distributions, we propose a nonlinear mapping (NLM) module to handle those more complex distributions that cannot be expressed by Gaussian assumptions, such as multimodal distributions. We utilize MODIS satellite to obtain remote sensing indices with 1-km spatial resolution, and integrating these with meteorological drought indices derived from ground weather stations for multisource data fusion, we conducted an experimental analysis of drought disaster situation in Yunnan Province. Our study demonstrates that the use of GACNPs outperforms traditional methods and other neural network approaches in terms of spatial interpolation accuracy. This improvement facilitates more precise meteorological drought monitoring tasks.