Open-world 3D scene understanding is critical for applications like autonomous driving and robotics, but it faces challenges due to the scarcity of dense 3D-text pairings. While 2D vision-language models offer rich pretraining knowledge, bridging the 2D-3D gap remains challenging. To address this issue, we propose OpenVL and utilize the framework of the Vision-Language model for Open-vocabulary understanding. First, we construct a channel that connects 2D and 3D features through the method of 3D Patches, which largely avoids the problems arising from the scarcity of 3D data. Secondly, we propose a Multi-level 3D Patches Alignment. By achieving dense alignment with 3D Patches at the point, instance, and scene levels, our method not only preserves the spatial perception ability within 3D scenes but also enables a fine-grained understanding of these scenes. Experimental results across multiple bench-marks demonstrate that our approach has achieved state-of-the-art performance. OpenVL achieves significant performance improvements on ScanNet (+6.7% mAP) and nuScenes (+1.9% mAP). More importantly, it demonstrates excellent zero-shot generalization capabilities in downstream tasks including 3DVQA and 3D Embodied Planning.Embodied Planning.

Decoding visual information from brain activity is important and challenging. Existing studies have successfully reconstructed static images from fMRI signals, but fMRI-based dynamic visual reconstruction still has limitations: the low temporal resolution of fMRI limits the coherence of the video, the lack of fine-grained alignment of brain with visual features at different scales, and the deviation of diffusion modeling from the logic of visual encoding. Research has shown that the visual cortex cognitive system streams perceptual and semantic information to different brain regions for processing. Inspired by this, we propose Mind-VAD, a novel dynamic visual reconstruction paradigm that redefines the brain decoding process as cross-modal generation guided by visual stream. Specifically, we design a time-sensitive encoder to accurately achieve temporal localization, and learn different dynamic visual representations of fMRI through an autoencoder with different scales. Then proceed with full-sequence, faster video autoregressive diffusion generation through a coarse-to-fine process. We include the latest AIGC Benchmark in evaluation. Extensive experiments show that Mind-VAD realized accurate and smooth reconstruction, achieves SOTA performance (87.3% classification, 27.5% semantic improvement) on several downstream tasks and more demanding metrics. Overall, Mind-VAD provides a neural interpretable and powerful framework for visual decoding and its potential applications.

To efficiently assist humans in various tasks, it is crucial to accurately decode and understand the rich information embedded in brain's visual cognition. Existing brain-driven research often fails to overcome the challenge of small target data domains, and the lack of explicit semantic, spatial, and other information constraints on feature extractors prevents brain decoding models from learning uniform cross-domain representations, leading to degradation of their performance in unseen domains. To overcome these limitations, we propose DAMind, a multimodal EEG-based model for robust visual cross-domain alignment and decoding. Our approach integrates VLM with brain-inspired cognitive mechanisms, leveraging the strong image-text representation abilities to learn both fine-grained primary visual features and high-level semantic concepts from neural signals, provide effective visual fine-tuning using the visual guidance mechanism. DAMind introduces a stepwise EEG encoding process aligned with visual processing, and employs an instruction-based learning strategy for effective cross-domain zero-shot transfer. Its robust architecture efficiently achieves good generalization performance, enabling the mapping of EEG signals from multiple domains to a unified learning domain. We construct a comprehensive EEG decoding benchmark EBench, DAMind achieves state-of-the-art results on several visual tasks, and outperforms the baseline in zero-shot setting.

Auditory attention detection (AAD) from electroencephalography (EEG) signals has garnered significant interest for its potential in brain-computer interfaces and hearing aids. Nevertheless, accurate decoding remains challenging due to the high-dimensional, non-stationary, and inherently noisy characteristics of EEG signals. We introduce SSAAD, a novel self-supervised approach for binary AAD with three key innovations: (1) A multi-scale temporal-frequency graph convolutional network (MST-GCN) is developed to effectively capture both spatial and temporal EEG dynamics; (2) A hierarchical self-supervised contrastive learning method is designed to cultivate robust EEG representations without extensive labeled data; (3) A bespoke triple-stage mixup data augmentation strategy is proposed to enhance model generalization. Additionally, MST-GCN initialization is achieved via EEG autoencoder pre-training, facilitating superior feature extraction. The method's effectiveness is evaluated on the KUL and DTU datasets, demonstrating state-of-the-art performance with accuracy improvements of 1.4% and 1.7% for 0.1-second windows, respectively.

In monocular depth estimation, effective acquisition of global and local information is the key to improving accuracy. We introduce a novel dual branch network called Swin Vision Transformer Net (SVTNet), where the Swin Transformer and Vision Transformer are combined to learn features with global and local information. In Swin Transformer branch, we remove the Feature Pyramid Networks which reduces the noise introduced by interpolation and maintains the original characteristics of the hierarchical features extracted from each layer of Swin Transformer. In Vision Transformer Branch, we remove the multi-scale convolutional structure and keep the same-scale feature maps extracted by Vision Transformer for reducing the consumption of memory and extracting more global features. The experimental results on the NYU-Depth v2, KITTI, and CityScapes datasets show that compared with existing advanced methods, the SVTNet has achieved significant performance improvements in multiple evaluation metrics.

Despite recent progress in decoding static images from brain activity, reconstructing dynamic visual experiences from EEG signals remains challenging due to the complex temporal dynamics involved. Current approaches primarily rely on pre-trained video generation models while failing to fully leverage the rich temporal-spatial information embedded in EEG signals for video synthesis. This paper proposes MINDEV Multi-modal Integrated Neural DEcoding and Visualization), a framework that places EEG signal processing at the core of video reconstruction. We introduce three key technical contributions: (1) a dual-branch feature extractor that captures both temporal dynamics and spatial relationships in EEG signals, (2) an EEG-driven semantic bridge that uses neural patterns to guide language model interpretation, and (3) a multi-modal video synthesis pipeline where EEG features lead the generation process while semantic guidance provides refinement. Our framework prioritizes the millisecond-level temporal resolution of EEG signals, using them to drive both visual content generation and semantic understanding. Evaluated on the SEED-DV dataset, MINDEV demonstrates superior performance with a semantic classification accuracy of 93.2% and a structural similarity index (SSIM) of 0.4777, establishing a new state-of-the-art for EEG-based video reconstruction. Our code is publicly available at https://github.com/HHarr1son/MINDEV.

Translating brain activity into meaningful visual content has long been recognized as a fundamental challenge in neuroscience and brain-computer interface research. Recent advances in EEG-based neural decoding have shown promise, yet two critical limitations remain in this area: poor generalization across subjects and constraints to specific visual tasks. We introduce NEED, the first unified framework achieving zero-shot cross-subject and cross-task generalization for EEG-based visual reconstruction. Our approach addresses three fundamental challenges: (1) cross-subject variability through an Individual Adaptation Module pretrained on multiple EEG datasets to normalize subject-specific patterns, (2) limited spatial resolution and complex temporal dynamics via a dual-pathway architecture capturing both low-level visual dynamics and high-level semantics, and (3) task specificity constraints through a unified inference mechanism adaptable to different visual domains. For video reconstruction, NEED achieves better performance than existing methods. Importantly, Our model maintains 93.7% of within-subject classification performance and 92.4% of visual reconstruction quality when generalizing to unseen subjects, while achieving an SSIM of 0.352 when transferring directly to static image reconstruction without fine-tuning, demonstrating how neural decoding can move beyond subject and task boundaries toward truly generalizable brain-computer interfaces.