Multimodal neuroimaging provides complementary structural and functional insights into both human brain organization and disease-related dynamics. Recent studies demonstrate enhanced diagnostic sensitivity for Alzheimer’s disease (AD) through synergistic integration of neuroimaging data (e.g., sMRI, fMRI) with tabular data (e.g., behavioral and cognitive tests). However, the intrinsic heterogeneity across modalities (e.g., 4D spatiotemporal fMRI dynamics vs. 3D anatomical sMRI structure) presents critical challenges for discriminative feature fusion, often leading to information loss or biased fusion. To bridge this gap, we propose M2M-AlignNet: a multimodal co-attention network with latent alignment for early AD diagnosis using sMRI and fMRI. At the core of our approach is a multi-patch-to-multi-patch (M2M) contrastive loss function that quantifies and reduces representational discrepancies via weighted patch correspondence, explicitly aligning fMRI components across brain regions with their sMRI structural substrates without one-to-one constraints. Additionally, we propose a latent-as-query co-attention module to autonomously discover fusion patterns, circumventing modality prioritization biases while minimizing feature redundancy. We conduct extensive experiments to confirm the effectiveness of our method and highlight the correspondence between fMRI and sMRI as AD biomarkers.

In recent years, assisted diagnosis of Alzheimer’s disease (AD) based on brain imaging has become a forefront research area that integrated artificial intelligence. Structural magnetic resonance imaging (sMRI) and positron emission tomography (PET) techniques respectively reveal changes in brain structure and function. Utilizing deep learning technology, common AD features can be learned from a large amount of imaging data to provide technical support for doctors’ diagnostic evaluations. To achieve more precise AD diagnosis, this study proposed an end-to-end network framework. Firstly, the framework divided brain images into 3D brain regions in the form of patches and used an attention CNN for feature extraction. Subsequently, each brain region feature was associated and optimized with a transformer encoder block equipped with multi-head attention mechanism to better represent the brain region interdependencies for AD and normal controls (NC). Finally, through feature fusion under two sub-networks, the method achieved further improvement in diagnostic performance for AD detection. The effectiveness of our method was validated on the ADNI dataset, and patch-based visualization was used to highlight key brain regions of interest in both imaging modalities.

Brain sMRI-based cognitive estimation of dementia states holds significant clinical value, aiding in assessing AD pathological staging and predicting disease progression. Some existing deep learning methods do not fully integrate the characteristics of brain image, which limits the effective representation of brain image and thus weakens its performance in regression and classification tasks. In this work, we propose a neural network that combining with Transformer attention to achieve cognitive estimation. In the proposed network, we design a Transformer block based on the long-range dependency and position encoding capabilities of Transformer to adapt to brain image and effectively combine it in the regression or classification task. We adapt Transformer attention for MRI feature by incorporating 3D positional encoding and 3D convolution. Our method can automatically identify sensitive regions from whole brain sMRI for end-to-end feature learning. Furthermore, it improves attention to AD sensitive biomarkers and improves the performance of regression tasks to jointly predict multiple cognitive scores. Evaluation on ADNI data sets shows promising performance of our method in cognitive scores estimation and AD classification.

MRI(Magnetic Resonance Imaging), as a non-invasive imaging technology, provides rich information at both the structural and functional levels of brain, offering significant support for the screening of Alzheimer's disease (AD). However, due to the large heterogeneity in data format and spatial characteristics between sMRI and fMRI, achieving effective fusion of these two modalities remains a major challenge. To address this issue, we first designed a Transformer attention module incorporating 3D positional encoding to effectively encode 3D sMRI features. Next, we constructed a cascaded transformer module to address the feature encoding of fMRI and the multimodal feature fusion of MRI images from different spatial domains, thereby enhancing the feature representation of both modalities. Additionally, we adopted a multi-layer fused feature integration strategy to enhance the robustness of multimodal features. Visualization analysis is conducted to demonstrate the effectiveness of our method. And our method significantly outperforms single-modality methods using either sMRI or fMRI, exhibiting superior performance in the AD detection task.

Positron Emission Tomography (PET) plays a vital role in the diagnosis of Alzheimer's Disease (AD) by revealing the brain's metabolic activity and detecting metabolic abnormalities in the early stages of AD. However, due to significant inter-subject variability in AD imaging presentations, metabolic patterns often differ across individuals, making it challenging for traditional deep learning models to effectively capture such complex and heterogeneous features. Considering that the brain exhibits structural symmetry between hemispheres but often presents pathological asymmetry and regional differences in disease progression, we propose an end-to-end hybrid framework to better model this nonuniform distribution. Our framework integrates 3D patch CNN for local feature encoding and incorporates Transformer Encoders to enhance global semantic representation. Subsequently, a Hemisphere-aware Cross Transformer is employed to fuse inter-hemispheric information, further improving feature discriminability. To enhance robustness under complex feature distributions, we employed a hybrid cross-entropy loss function to optimize the classification task. Experimental results based on the ADNI dataset demonstrate that our method achieves significant improvement in AD diagnosis and the mild cognitive impairment (MCI) conversion task. Importantly, our visualization of key brain regions further confirms the model's attention to ADrelated biomarkers, highlighting its potential to improve early AD diagnosis and clinical application.

Decoding visual experiences from fMRI offers a powerful avenue to understand human perception and develop advanced brain-computer interfaces. However, current progress often prioritizes maximizing reconstruction fidelity while overlooking interpretability, an essential aspect for deriving neuroscientific insight. To address this gap, we propose MoRE-Brain, a neuro-inspired framework designed for high-fidelity, adaptable, and interpretable visual reconstruction. MoRE-Brain uniquely employs a hierarchical Mixture-of-Experts architecture where distinct experts process fMRI signals from functionally related voxel groups, mimicking specialized brain networks. The experts are first trained to encode fMRI into the frozen CLIP space. A finetuned diffusion model then synthesizes images, guided by expert outputs through a novel dual-stage routing mechanism that dynamically weighs expert contributions across the diffusion process. MoRE-Brain offers three main advancements: First, it introduces a novel Mixture-of-Experts architecture grounded in brain network principles for neuro-decoding. Second, it achieves efficient cross-subject generalization by sharing core expert networks while adapting only subject-specific routers. Third, it provides enhanced mechanistic insight, as the explicit routing reveals precisely how different modeled brain regions shape the semantic and spatial attributes of the reconstructed image. Extensive experiments validate MoRE-Brain’s high reconstruction fidelity, with bottleneck analyses further demonstrating its effective utilization of fMRI signals, distinguishing genuine neural decoding from over-reliance on generative priors. Consequently, MoRE-Brain marks a substantial advance towards more generalizable and interpretable fMRI-based visual decoding.

With the abundance of medical data, computer-aided AD diagnosis using multi-source and multi-modal data is a hotspot and trend in research, which brings more possibilities for the realization of accurate assessment of cognitive impairment diseases. Currently, the AD diagnosis based on convolutional neural network (CNN) is still the main method. But clinically, part of data exists in non-imaging form, which makes CNNs have a lot of challenges in fusing imaging and non-imaging. Graph neural network (GNN), which extends classical CNN to non-Euclidean space by using graph topology, affords better flexibility for multi-modal data integration. In order to realize AD diagnosis based on multi-source and multi-modality data, this work takes the advantage of CNN in acquiring image features, and further combines image features and non-imaging information via GNN, proposed a joint CNN-GNN diagnostic framework. Through ablation experiments, we further analyzed the effects of MMSE score, and Apoe4 genotype on AD diagnosis on the basis of image and could provide clinical reference. In addition, our proposed method achieved further improvement in diagnostic performance.