In recent years, assisted diagnosis of Alzheimer’s disease (AD) based on brain imaging has become a forefront research area that integrated artificial intelligence. Structural magnetic resonance imaging (sMRI) and positron emission tomography (PET) techniques respectively reveal changes in brain structure and function. Utilizing deep learning technology, common AD features can be learned from a large amount of imaging data to provide technical support for doctors’ diagnostic evaluations. To achieve more precise AD diagnosis, this study proposed an end-to-end network framework. Firstly, the framework divided brain images into 3D brain regions in the form of patches and used an attention CNN for feature extraction. Subsequently, each brain region feature was associated and optimized with a transformer encoder block equipped with multi-head attention mechanism to better represent the brain region interdependencies for AD and normal controls (NC). Finally, through feature fusion under two sub-networks, the method achieved further improvement in diagnostic performance for AD detection. The effectiveness of our method was validated on the ADNI dataset, and patch-based visualization was used to highlight key brain regions of interest in both imaging modalities.

MRI(Magnetic Resonance Imaging), as a non-invasive imaging technology, provides rich information at both the structural and functional levels of brain, offering significant support for the screening of Alzheimer's disease (AD). However, due to the large heterogeneity in data format and spatial characteristics between sMRI and fMRI, achieving effective fusion of these two modalities remains a major challenge. To address this issue, we first designed a Transformer attention module incorporating 3D positional encoding to effectively encode 3D sMRI features. Next, we constructed a cascaded transformer module to address the feature encoding of fMRI and the multimodal feature fusion of MRI images from different spatial domains, thereby enhancing the feature representation of both modalities. Additionally, we adopted a multi-layer fused feature integration strategy to enhance the robustness of multimodal features. Visualization analysis is conducted to demonstrate the effectiveness of our method. And our method significantly outperforms single-modality methods using either sMRI or fMRI, exhibiting superior performance in the AD detection task.

The cross-modal synthesis between structural magnetic resonance imaging (sMRI) and functional network connectivity (FNC) is a relatively unexplored area in medical imaging, especially with respect to schizophrenia. This study employs conditional Vision Transformer Generative Adversarial Networks (cViT-GANs) to generate FNC data based on sMRI inputs. After training on a comprehensive dataset that included both individuals with schizophrenia and healthy control subjects, our cViT-GAN model effectively synthesized the FNC matrix for each subject, and then formed a group difference FNC matrix, obtaining a Pearson correlation of 0.73 with the actual FNC matrix. In addition, our FNC visualization results demonstrate significant correlations in particular subcortical brain regions, highlighting the model’s capability of capturing detailed structural-functional associations. This performance distinguishes our model from conditional CNN-based GAN alternatives such as Pix2Pix. Our research is one of the first attempts to link sMRI and FNC synthesis, setting it apart from other cross-modal studies that concentrate on T1- and T2-weighted MR images or the fusion of MRI and CT scans.

Alzheimer's disease (AD) is a neurodegenerative process characterized by the accumulation of amyloid-beta plaques and neurofibrillary tangles and is the most common cause of dementia. Studies have been striving to analyze the disease using available physiological and behavioral data. Functional/structural neuroimaging and genomics are complementary modalities for exploring the mechanisms subserving the development of AD. In this paper, we present a deep multimodal generative data fusion framework for integrating these sources in a classification task involving AD patients and healthy controls from the ADNI database. Biological data fusion has the potential to improve the final prediction, but at the same time, it is particularly challenging due to the unavailability of all sources of input for the entire cohort of subjects. Our proposed model allows us to perform prediction even if individuals are missing certain modalities. Our method addresses the missing modalities problem via knowledge transfer from two generative adversarial networks. The model exhibits superior performance for predicting AD versus healthy control, even with missing modalities. This could have an important impact from the patient point of view since certain clinical tests may not be necessary or available to a given individual.

Understanding the associations of the structural and functional patterns of the brain is vital. Recent studies have focused on utilizing this information within and across the different functional and anatomical domains (i.e., groups of brain networks) using neuroimaging data. In this work, we use a Bayesian optimization-based method known as the Tree Parzen Estimator (TPE) to identify variation in the nature of information encoded by different functional magnetic resonance imaging (fMRI) sub-domains of the brain. We show by repeated cross-validation on a schizophrenia classification task that specific sub-domains may require more sophisticated learning architectures to contribute optimally to classification, while others require less complicated ones. Our findings reveal the need for adaptive, hierarchical learning frameworks catering to features from different sub-domains to optimally identify features enabling the prediction of the outcome of interest.

Functional magnetic resonance imaging (fMRI) is a brain imaging technique which provides detailed insights into brain function and its disruption in various brain disorders. fMRI data can be analyzed using data-driven or region-of-interest based methods. The data-driven analysis of brain activity maps involves several steps, the first of which is identifying whether the maps capture what might be interpreted as intrinsic connectivity networks (ICNs) or artifacts. This is followed by linking the ICNs to known anatomical and/or functional parcellations. Optionally, as in the study of functional network connectivity (FNC), rearranging the connectivity graph is also necessary for systematic interpretation. Here we present a toolbox that automates all these processes under minimal or no supervision with high accuracy. We provide a pretrained cross-validated elastic-net regularized general linear model for the noisecloud toolbox to separate the ICNs from artifacts. We include several well-known anatomical and functional parcellations from which researchers can choose to label the activity maps. Finally, we integrate a method for maximizing the within-domain modularity to generate a more systematically structured FNC matrix. We improve upon and integrate existing techniques and new methods to design this toolbox which can take care of all the above needs. Specifically, we show that our pretrained model achieves 89% accuracy and 100% precision at classifying ICNs from artifacts in a validation dataset. Researchers are generating brain imaging data and analyzing brain activity at an ever-increasing rate. The Autolabeller toolbox can help automate such analyses for faster and reproducible research.