Recent progress in artificial intelligence has encouraged numerous attempts to understand and decode human visual system from brain signals. These prior works typically align neural activity independently with semantic and perceptual features extracted from images using pre-trained vision models. However, they fail to account for two key challenges: (1) the modality gap arising from the natural difference in the information level of representation between brain signals and images, and (2) the fact that semantic and perceptual features are highly entangled within neural activity. To address these issues, we utilize hyperbolic space, which is well-suited for considering differences in the amount of information and has the geometric property that geodesics between two points naturally bend toward the origin, where the representational capacity is lower. Leveraging these properties, we propose a novel framework, Hyperbolic Feature Interpolation (HyFI), which interpolates between semantic and perceptual visual features along hyperbolic geodesics. This enables both the fusion and compression of perceptual and semantic information, effectively reflecting the limited expressiveness of brain signals and the entangled nature of these features. As a result, it facilitates better alignment between brain and visual features. We demonstrate that HyFI achieves state-of-the-art performance in zero-shot brain-to-image retrieval, outperforming prior methods with Top-1 accuracy improvements of up to +17.3% on THINGS-EEG and +9.1% on THINGS-MEG.

Resting-state functional magnetic resonance imaging (rs-fMRI) measures intrinsic neural activity, and analyzing its frequency-domain characteristics provides insights into brain dynamics. Owing to these properties, rs-fMRI is widely used to investigate brain disorders such as autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD). Conventional frequency-domain analyses typically rely on the Fourier transform, which lacks flexibility in capturing non-stationary neural signals due to its fixed resolution. Furthermore, these methods primarily utilize only real-valued features, such as the magnitude or phase, derived from complex-valued spectral representations. Consequently, direct modeling of the real and imaginary components, particularly within fMRI analyses, remains largely unexplored, overlooking the distinct and complementary spectral information encoded in these components. To address these limitations, we propose a novel Transformer-based framework that explicitly models the real and imaginary components of continuous wavelet transform (CWT) coefficients from rs-fMRI signals. Our architecture integrates spectral, temporal, and spatial attention modules, employing self- and cross-attention mechanisms to jointly capture intra- and inter-component relationships. Applied to the Autism Brain Imaging Data Exchange (ABIDE)-I and ADHD-200 datasets, our approach achieved state-of-the-art classification performance compared to existing baselines. Comprehensive ablation studies demonstrated the advantages of directly utilizing real and imaginary components over conventional frequency-domain features and validate each module's contribution. Moreover, attention-based analyses revealed frequency- and region-specific patterns consistent with known neurobiological alterations in ASD and ADHD. These findings highlight that preserving and jointly leveraging the real and imaginary components of CWT-based representations not only enhances diagnostic performance but also provides interpretable insights into neurodevelopmental disorders.

Ultrahigh-field (UHF) magnetic resonance imaging (MRI), 7T MRI, provides superior anatomical details of internal brain structures thanks to its enhanced signal-to-noise ratio and susceptibility-induced contrast. However, the widespread use of 7T MRI is limited by its high cost and lower accessibility compared to low-field (LF) MRI. This study proposes a SegUHF that systematically fuses the input LF MRI feature representations with the inferred 7T-like feature representations for brain image segmentation tasks in a 7T-absent environment. Specifically, our proposed adaptive fusion module within the SegUHF aggregates 7T-like features derived from the LF image using a pre-trained network and then refines them to effectively assimilate UHF guidance into LF image features. Using intensity-guided features obtained from such aggregation and assimilation, segmentation models can recognize subtle structural representations that are usually difficult to identify when relying only on LF features. Beyond these advantages, this strategy can be seamlessly utilized by modulating the contrast of LF features in alignment with UHF guidance, even when employing arbitrary segmentation models. Extensive experiments demonstrated that our method outperformed all baselines in both brain tissue and whole-brain segmentation, while also showcasing adaptability and scalability across various models and tasks. Code is available at https://github.com/ku-milab/UHF-guided_segmentation

Accurate identification of brain disorders enables timely intervention and improved patient outcomes. While numerous studies have developed AI models for resting-state functional magnetic resonance imaging (rs-fMRI) analysis, most rely on supervised learning, which can overlook hidden patterns that are less discriminatively associated with labels and require large annotated datasets. To address these limitations, we propose leveraging normative modeling, an unsupervised approach that constructs a model of normality based on healthy controls' data. Deviations from normality indicate potential disorders. However, applying normative modeling to rs-fMRI faces two significant challenges: constructing normality and ensuring explainability. To tackle these challenges, we propose BRAINEXA, a novel framework enhancing normative modeling for rs-fMRI-based brain disorder identification. Specifically, to construct accurate and stable normality, BRAINEXA introduces a training strategy that predicts more informative regions from less informative regions, discouraging trivial self-supervised learning solutions and improving representation learning without additional overhead. Furthermore, we incorporate spatiotemporal mutual information regularization to preserve distinctiveness between more informative regions and less informative regions during latent encoding, preventing potential representational distortions. For interpretability, BRAINEXA extracts normality-defining (ND) subregions, the core regions that characterize normal brain function. By combining ND subregions with anomaly scores, BRAINEXA can offer region- and connection-wise explanations that help identify clinically meaningful disruptions of normality in an unsupervised setting. We demonstrate the effectiveness of BRAINEXA on four public rs-fMRI datasets: REST-meta-MDD, ABIDE I, ADHD-200, and OASIS-3. Our code is available at https://github.com/ku-milab/BRAINEXA.

Single-source domain generalization (SDG) in medical image segmentation (MIS) aims to generalize a model using only one source domain data to segment data from an unseen target domain. Despite substantial advances in SDG with data augmentation, existing methods often fail to fully consider the details and uncertain areas prevalent in MIS, leading to mis-segmentation. In this study, we propose a Fourier-based semantic augmentation method called FIESTA using uncertainty guidance (UG) to enhance the fundamental goals of MIS in an SDG context by manipulating the amplitude and phase components in the frequency domain. The proposed Fourier augmentative transformer (FAT) addresses semantic amplitude modulation based on meaningful angular points to induce pertinent variations and harnesses the phase spectrum to ensure structural coherence. Moreover, FIESTA employs uncertainty estimation to fine-tune the augmentation process, improving the ability of the model to adapt to diverse augmented data and concentrate on areas with higher ambiguity. Extensive experiments across three cross-domain scenarios demonstrate that FIESTA surpasses recent state-of-the-art SDG approaches in segmentation performance and significantly contributes to boosting the model's applicability in medical imaging modalities.

To understand the biological characteristics of neurological disorders with functional connectivity (FC), recent studies have widely utilized deep learning-based models to identify the disease and conducted post-hoc analyses via explainable models to discover disease-related biomarkers. Most existing frameworks consist of three stages, namely, feature selection, feature extraction for classification, and analysis, where each stage is implemented separately. However, if the results at each stage lack reliability, it can cause misdiagnosis and incorrect analysis in afterward stages. In this study, we propose a novel unified framework that systemically integrates diagnoses (i.e., feature selection and feature extraction) and explanations. Notably, we devised an adaptive attention network as a feature selection approach to identify individual-specific disease-related connections. We also propose a functional network relational encoder that summarizes the global topological properties of FC by learning the inter-network relations without pre-defined edges between functional networks. Last but not least, our framework provides a novel explanatory power for neuroscientific interpretation, also termed counter-condition analysis. We simulated the FC that reverses the diagnostic information (i.e., counter-condition FC): converting a normal brain to be abnormal and vice versa. We validated the effectiveness of our framework by using two large resting-state functional magnetic resonance imaging (fMRI) datasets, Autism Brain Imaging Data Exchange (ABIDE) and REST-meta-MDD, and demonstrated that our framework outperforms other competing methods for disease identification. Furthermore, we analyzed the disease-related neurological patterns based on counter-condition analysis.

Transfer learning has attracted considerable attention in medical image analysis because of the limited number of annotated 3-D medical datasets available for training data-driven deep learning models in the real world. We propose Medical Transformer, a novel transfer learning framework that effectively models 3-D volumetric images as a sequence of 2-D image slices. To improve the high-level representation in 3-D-form empowering spatial relations, we use a multiview approach that leverages information from three planes of the 3-D volume, while providing parameter-efficient training. For building a source model generally applicable to various tasks, we pretrain the model using self-supervised learning (SSL) for masked encoding vector prediction as a proxy task, using a large-scale normal, healthy brain magnetic resonance imaging (MRI) dataset. Our pretrained model is evaluated on three downstream tasks: 1) brain disease diagnosis; 2) brain age prediction; and 3) brain tumor segmentation, which are widely studied in brain MRI research. Experimental results demonstrate that our Medical Transformer outperforms the state-of-the-art (SOTA) transfer learning methods, efficiently reducing the number of parameters by up to approximately 92% for classification and regression tasks and 97% for segmentation task, and it also achieves good performance in scenarios where only partial training samples are used.

Large amounts of fMRI data are essential to building generalized predictive models for brain disease diagnosis. In order to conduct extensive data analysis, it is often necessary to gather data from multiple organizations. However, the site variation inherent in multisite resting-state functional magnetic resonance imaging (rs-fMRI) leads to unfavorable heterogeneity in data distribution, negatively impacting the identification of biomarkers and the diagnostic decision. Several existing methods have alleviated this shift of domain distribution (i.e., multisite problem). Statistical tuning schemes directly regress out site disparity factors from the data prior to model training. Such methods have a limitation in processing data each time through variance estimation according to the added site. In the model adjustment approaches, domain adaptation (DA) methods adjust the features or models of the source domain according to the target domain during model training. Thus, it is inevitable that it needs updating model parameters according to the samples of a target site, causing great limitations in practical applicability. Meanwhile, the approach of domain generalization (DG) aims to create a universal model that can be quickly adapted to multiple domains. In this study, we propose a novel framework for disease diagnosis that alleviates the multisite problem by adaptively calibrating site-specific features into site-invariant features. Specifically, it applies directly to samples from unseen sites without the need for fine-tuning. With a learning-to-learn strategy that learns how to calibrate the features under the various domain shift environments, our novel modulation mechanism extracts site-invariant features. In our experiments over the Autism Brain Imaging Data Exchange (ABIDE I and II) dataset, we validated the generalization ability of the proposed network by improving diagnostic accuracy in both seen and unseen multisite samples.