Visual neural decoding is an important research topic at the intersection of cognitive neuroscience and machine learning. While recent progress has been made in EEG-based neural decoding, reconstructing dynamic visual content remains challenging. In the field of EEG decoding, current models either utilize pre-trained encoders for feature extraction or employ graph neural networks to represent the spatio-temporal information embedding, resulting in poor model representation and high complexity. We propose EVOKE -- an innovative framework for zero-shot decoding of high-fidelity videos from EEG signals. EVOKE employs Implicit Neural Representations to perform complete spatial modeling of EEG and continuously decouples information in the EEG-INR perceptual space. Additionally, we construct a Hierarchical-aware Attention Module (HAM) to decode EEG from three feature anchors: visual, semantic, motion, and progressively control task inference. The Motion Attention Flow (MAF) we developed overcomes the limitations of capturing motion features in dynamic stimuli, creating a more robust representation that enhances reconstruction consistency. Comprehensive experiments prove that SOTA performance of EVOKE (0.353 SSIM, 0.715 CLIP-pcc). We provide an effective method for converting brain activity into rich visual experiences and set a new benchmark for brain multimodal generation.

Decoding visual information from brain activity is important and challenging. Existing studies have successfully reconstructed static images from fMRI signals, but fMRI-based dynamic visual reconstruction still has limitations: the low temporal resolution of fMRI limits the coherence of the video, the lack of fine-grained alignment of brain with visual features at different scales, and the deviation of diffusion modeling from the logic of visual encoding. Research has shown that the visual cortex cognitive system streams perceptual and semantic information to different brain regions for processing. Inspired by this, we propose Mind-VAD, a novel dynamic visual reconstruction paradigm that redefines the brain decoding process as cross-modal generation guided by visual stream. Specifically, we design a time-sensitive encoder to accurately achieve temporal localization, and learn different dynamic visual representations of fMRI through an autoencoder with different scales. Then proceed with full-sequence, faster video autoregressive diffusion generation through a coarse-to-fine process. We include the latest AIGC Benchmark in evaluation. Extensive experiments show that Mind-VAD realized accurate and smooth reconstruction, achieves SOTA performance (87.3% classification, 27.5% semantic improvement) on several downstream tasks and more demanding metrics. Overall, Mind-VAD provides a neural interpretable and powerful framework for visual decoding and its potential applications.

Human reasoning naturally combines concepts to identify unseen compositions, a capability that Compositional Zero-Shot Learning (CZSL) aims to replicate in machine learning models. However, we observe that focusing solely on typical image classification tasks in CZSL may limit models’ compositional generalization potential. To address this, we introduce C-EgoExo, a video-based benchmark, along with a compositional action recognition task to enable more comprehensive evaluations. Inspired by human reasoning processes, we propose a Dual-branch Hybrid Discrimination (DHD) framework, featuring two branches that decode visual inputs in distinct observation sequences. Through a cross-attention mechanism and a contextual dependency encoder, DHD effectively mitigates challenges posed by conditional variance. We further design a Copula-based orthogonal decoding loss to counteract contextual interference in primitive decoding. Our approach demonstrates outstanding performance across diverse CZSL tasks, excelling in both image-based and video-based modalities and in attribute-object and action-object compositions, setting a new benchmark for CZSL evaluation.