In-context learning (ICL) offers a promising paradigm for universal medical image analysis, enabling models to perform diverse image processing tasks without retraining. However, current ICL models for medical imaging remain limited in two critical aspects: they cannot simultaneously achieve high-fidelity predictions and global anatomical understanding, and there is no unified model trained across diverse medical imaging tasks (e.g., segmentation and enhancement) and anatomical regions. As a result, the full potential of ICL in medical imaging remains underexplored. Thus, we present Medverse, a universal ICL model for 3D medical imaging, trained on 22 datasets covering diverse tasks in universal image segmentation, transformation, and enhancement across multiple organs, imaging modalities, and clinical centers. Medverse employs a next-scale autoregressive in-context learning framework that progressively refines predictions from coarse to fine, generating consistent, full-resolution volumetric outputs and enabling multi-scale anatomical awareness. We further propose a blockwise cross-attention module that facilitates long-range interactions between context and target inputs while preserving computational efficiency through spatial sparsity. Medverse is extensively evaluated on a broad collection of held-out datasets covering previously unseen clinical centers, organs, species, and imaging modalities. Results demonstrate that Medverse substantially outperforms existing ICL baselines and establishes a novel paradigm for in-context learning.

In-context learning (ICL), a type of universal model, demonstrates exceptional generalization across a wide range of tasks without retraining by leveraging task-specific guidance from context, making it particularly effective for the intricate demands of neuroimaging. However, current ICL models, limited to 2D inputs and thus exhibiting suboptimal performance, struggle to extend to 3D inputs due to the high memory demands of ICL. In this regard, we introduce Neuroverse3D, an ICL model capable of performing multiple neuroimaging tasks in 3D (e.g., segmentation, denoising, inpainting). Neuroverse3D overcomes the large memory consumption associated with 3D inputs through adaptive parallel-sequential context processing and a U-shaped fusion strategy, allowing it to handle an unlimited number of context images. Additionally, we propose an optimized loss function to balance multi-task training and enhance focus on anatomical boundaries. Our study incorporates 43,674 3D multi-modal scans from 19 neuroimaging datasets and evaluates Neuroverse3D on 14 diverse tasks using held-out test sets. The results demonstrate that Neuroverse3D significantly outperforms existing ICL models and closely matches task-specific models, enabling flexible adaptation to medical center variations without retraining. The code and model weights are publicly available at https://github.com/jiesihu/Neuroverse3D.

EEG-based seizure prediction enables timely treatment for patients, but its performance is limited by the difficulty in effectively characterizing the temporal dynamics of epileptic brain networks. Metastability, which describes recurring topographical patterns of spontaneous neural activity over time, provides a unique perspective for capturing the dynamic evolution before seizure onset. In this study, we propose a seizure prediction model that fuses consistent epileptic network processes across subjects into a higher-order latent space. Specifically, we first construct metastable transition patterns to identify the recurrent network states over time. Through adversarial feature learning, we then impose the metastability prior on the latent embedding space encoded via a variational autoencoder (VAE), while leveraging the maximum mean discrepancy measure (MMD) to further mitigate the patient gap. The latent representation, endowed with physiological priors, is ultimately utilized for patient-independent seizure prediction. We evaluate our method on two publicly available and one clinical scalp EEG datasets. Compared to the existing methods, our method has improved AUC, sensitivity, and specificity on CHB-MIT dataset by approximately 9%, 5%, and 5%, respectively. Our method shows that combining brain network-based physiological prior with deep learning for EEG representation learning is a brand-new strategy for associating seizures with complex brain network variations, enabling reliable patient-independent seizure prediction.

The multi-modal neuroimage study has provided insights into understanding the heteromodal relationships between brain network organization and behavioral phenotypes. Integrating data from various modalities facilitates the characterization of the interplay among anatomical, functional, and physiological brain alterations or developments. Graph Neural Networks (GNNs) have recently become popular in analyzing and fusing multi-modal, graph-structured brain networks. However, effectively learning complementary representations from other modalities remains a significant challenge due to the sophisticated and heterogeneous inter-modal dependencies. Furthermore, most existing studies often focus on specific modalities (e.g., only fMRI and DTI), which limits their scalability to other types of brain networks. To overcome these limitations, we propose a HyperComplex Graph Neural Network (HC-GNN) that models multi-modal networks as hypercomplex tensor graphs. In our approach, HC-GNN is conceptualized as a dynamic spatial graph, where the attentively learned inter-modal associations are represented as the adjacency matrix. HC-GNN leverages hypercomplex operations for inter-modal intersections through cross-embedding and cross-aggregation, enriching the deep coupling of multi-modal representations. We conduct a statistical analysis on the saliency maps to associate disease biomarkers. Extensive experiments on three datasets demonstrate the superior classification performance of our method and its strong scalability to various types of modalities. Our work presents a powerful paradigm for the study of multi-modal brain networks.

In practical medical image segmentation tasks, ensuring confidence calibration is crucial. However, medical image segmentation typically relies on hard labels (one-hot vectors), and when minimizing the cross-entropy loss, the model’s softmax predictions are compelled to align with hard labels, resulting in over-confident predictions. To alleviate above problems, this study proposes a novel framework on calibration of medical image segmentation, called CALSeg. The Variational Label Smoothing (VLS) method is innovatively proposed, which learns the latent joint distribution of images and labels through variational inference to capture complex relationships between images and labels. This enables the effective estimation of latent soft labels by learning pixel-level information and semantic probability distribution features. The training of a neural network based on estimated soft labels provides a regularization effect, effectively preventing model overfitting and improving the calibration of the model. Comprehensive experiments on two medical image segmentation datasets demonstrate that CALSeg achieved optimal network calibration while also improving segmentation accuracy. The code is available at https://github.com/Guoxt/CALSeg.

Recent advancements in fMRI-based brain disorder diagnosis have shown that graph neural networks (GNNs) have been state-of-the-art methods for brain network analysis. Among them, transductive and inductive learning can be exploited by GNN. Transductive graphs, such as population graphs, take each subject as a node and use the node classification task for diagnosis. This line of work suffers from high computational costs and poor scalability to unseen data. Inductive methods, on the other hand, only consider labeled data and may suffer from overfitting and poor generalization when training with insufficient samples. To address these limitations, we propose a unified transductive-inductive network to study the properties of both transductive and inductive learning frameworks. Our approach is implemented in a self-knowledge distillation architecture, where transductive predictions are distilled from a transductive population graph network into an inductive network as a self-supervised regularization term. To preserve the topological properties within transductive graphs, i.e., inter-node similarity, we propose a topology-regularized self-knowledge distillation (Topo-KD) approach to regularize the student model’s learning. Evaluations on the ADNI dataset demonstrate the superiority of the approach in performance and scalability.

EEG signal classification using Riemannian manifolds has shown great potential. However, the huge computational cost associated with Riemannian metrics poses challenges for applying Riemannian methods, particularly in high-dimensional feature data. To address these, we propose an efficient ensemble method called MLCSP-TSE-MLP, which aims to reduce the computational cost while achieving superior performance. MLCSP of the ensemble utilizes a Riemannian graph embedding strategy to learn intrinsic low-dimensional sub-manifolds, enhancing discrimination. TSE uses the Euclidean mean as the reference point for tangent space mapping and reducing computational cost. Finally, the ensemble incorporates the MLP classifier to offer improved classification performance. Classification results conducted on three datasets demonstrate that MLCSP-TSE-MLP achieves significant superior performance compared to various competing methods. Notably, the MLCSP-TSE module achieves a remarkable increase in training speed and exhibits much lower test time compared to traditional Riemannian methods. Based on these results, we believe that the proposed MLCSP-TSE-MLP is a powerful tool for handling high-dimensional data and holds great potential for practical applications.

Source-free domain adaptation (SFDA) aims to adapt models trained on a labeled source domain to an unlabeled target domain without access to source data. In medical imaging scenarios, the practical significance of SFDA methods has been emphasized due to data heterogeneity and privacy concerns. Recent state-of-the-art SFDA methods primarily rely on self-training based on pseudo-labels (PLs). Unfortunately, the accuracy of PLs may deteriorate due to domain shift, thus limiting the effectiveness of the adaptation process. To address this issue, we propose a Chebyshev confidence guided SFDA framework to accurately assess the reliability of PLs and generate self-improving PLs for self-training. The Chebyshev confidence is estimated by calculating the probability lower bound of PL confidence, given the prediction and the corresponding uncertainty. Leveraging the Chebyshev confidence, we introduce two confidence-guided denoising methods: direct denoising and prototypical denoising. Additionally, we propose a novel teacher-student joint training scheme (TJTS) that incorporates a confidence weighting module to iteratively improve PLs' accuracy. The TJTS, in collaboration with the denoising methods, effectively prevents the propagation of noise and enhances the accuracy of PLs. Extensive experiments in diverse domain scenarios validate the effectiveness of our proposed framework and establish its superiority over state-of-the-art SFDA methods. Our paper contributes to the field of SFDA by providing a novel approach for precisely estimating the reliability of PLs and a framework for obtaining high-quality PLs, resulting in improved adaptation performance.

Recently, the study of multi-modal brain connectome has recorded a tremendous increase and facilitated the diagnosis of brain disorders. In this paradigm, functional and structural networks, e.g., functional and structural connectivity derived from fMRI and DTI, are in some manner interacted but are not necessarily linearly related. Accordingly, there remains a great challenge to leverage complementary information for brain connectome analysis. Recently, Graph Convolutional Networks (GNN) have been widely applied to the fusion of multi-modal brain connectome. However, most existing GNN methods fail to couple inter-modal relationships. In this regard, we propose a Cross-modal Graph Neural Network (Cross-GNN) that captures inter-modal dependencies through dynamic graph learning and mutual learning. Specifically, the inter-modal representations are attentively coupled into a compositional space for reasoning inter-modal dependencies. Additionally, we investigate mutual learning in explicit and implicit ways: (1) Cross-modal representations are obtained by cross-embedding explicitly based on the inter-modal correspondence matrix. (2) We propose a cross-modal distillation method to implicitly regularize latent representations with cross-modal semantic contexts. We carry out statistical analysis on the attentively learned correspondence matrices to evaluate inter-modal relationships for associating disease biomarkers. Our extensive experiments on three datasets demonstrate the superiority of our proposed method for disease diagnosis with promising prediction performance and multi-modal connectome biomarker location.

Foundation models pretrained on large-scale datasets via self-supervised learning demonstrate exceptional versatility across various tasks. Due to the heterogeneity and hard-to-collect medical data, this approach is especially beneficial for medical image analysis and neuroscience research, as it streamlines broad downstream tasks without the need for numerous costly annotations. However, there has been limited investigation into brain network foundation models, limiting their adaptability and generalizability for broad neuroscience studies. In this study, we aim to bridge this gap. In particular, 1) we curated a comprehensive dataset by collating images from 30 datasets, which comprises 70,781 samples of 46,686 participants. Moreover, we introduce pseudo-functional connectivity (pFC) to further generates millions of augmented brain networks by randomly dropping certain timepoints of the BOLD signal; 2) we propose the BrainMass framework for brain network self-supervised learning via mask modeling and feature alignment. BrainMass employs Mask-ROI Modeling (MRM) to bolster intra-network dependencies and regional specificity. Furthermore, Latent Representation Alignment (LRA) module is utilized to regularize augmented brain networks of the same participant with similar topological properties to yield similar latent representations by aligning their latent embeddings. Extensive experiments on eight internal tasks and seven external brain disorder diagnosis tasks show BrainMass's superior performance, highlighting its significant generalizability and adaptability. Nonetheless, BrainMass demonstrates powerful few/zero-shot learning abilities and exhibits meaningful interpretation to various diseases, showcasing its potential use for clinical applications.

The multi-modal neuroimage study has dramatically facilitated disease diagnosis. Tensor-based methods are commonly used to represent multi-modal data as multi-dimensional arrays and usually implement matrix decomposition. These methods can be seen as a linear algebraic way for the lossy compression of an array. However, involved lossy operations might have a negative impact on performance, and overlook underlying important complementary information between modalities. This study proposes a Tensor-based Complex-valued Graph Neural Network (TC-GNN) to model multimodal neuroimages as complex-valued tensor graphs by investigating underlying complementary associations and cross-modality message aggregation. Experiments on two real-world datasets demonstrate our method’s consistent improvements and superiority over other baseline models in multi-modal brain disease analysis.

Multivariate analysis approaches provide insights into the identification of phenotype associations in brain connectome data. In recent years, deep learning methods including convolutional neural network (CNN) and graph neural network (GNN), have shifted the development of connectome-wide association studies (CWAS) and made breakthroughs for connectome representation learning by leveraging deep embedded features. However, most existing studies remain limited by potentially ignoring the exploration of region-specific features, which play a key role in distinguishing brain disorders with high intra-class variations, such as autism spectrum disorder (ASD), and attention deficit hyperactivity disorder (ADHD). Here, we propose a multivariate distance-based connectome network (MDCN) that addresses the local specificity problem by efficient parcellation-wise learning, as well as associating population and parcellation dependencies to map individual differences. The approach incorporating an explainable method, parcellation-wise gradient and class activation map (p-GradCAM), is feasible for identifying individual patterns of interest and pinpointing connectome associations with diseases. We demonstrate the utility of our method on two largely aggregated multicenter public datasets by distinguishing ASD and ADHD from healthy controls and assessing their associations with underlying diseases. Extensive experiments have demonstrated the superiority of MDCN in classification and interpretation, where MDCN outperformed competitive state-of-the-art methods and achieved a high proportion of overlap with previous findings. As a CWAS-guided deep learning method, our proposed MDCN framework may narrow the bridge between deep learning and CWAS approaches, and provide new insights for connectome-wide association studies.

Medical image segmentation annotation suffers from annotator variation due to the inherent differences in annotators’ expertise and the inherent blurriness of medical images. In practice, using opinions from multiple annotators can effectively reduce the impact of such annotator-related biases. Meanwhile, it is common practice in deep learning to fuse multiple annotations through methods such as majority voting, but these methods ignore the rich information of annotator preferences ingrained in the original multi-annotator annotations. To address this issue, we propose a modeling annotator variation and annotator preference (AVAP) framework for multiple annotations medical image segmentation, which consists of three parts. First, the widely used encoder-decoder backbone network use to extract feature maps of the image. Second, an annotator variation modeling (AVM) module is devised to estimate the annotation variation among multiple annotators by modeling multi-annotations as a multi-class segmentation problem. Third, an annotator preference modeling (APM) module estimate each annotator’s preference-involved segmentation by annotator encoding and dynamic filter learning. The experiment on the RIGA benchmark with multiple annotations shows that our AVAP framework outperforms a range of state-of-the-art (SOTA) multiple annotations segmentation methods. Further, we are the first to introduce dynamic filter learning into the annotator preference modeling.

Multi-modal neuroimaging technology has greatly facilitated the diagnosis efficiency and diagnosis accuracy, and provides complementary information in discovering objective disease biomarkers. Conventional deep learning methods, e.g. convolutional neural networks, overlook relationships between nodes and fail to capture topological properties in graphs. Graph neural networks have been proven to be of great importance in modeling brain connectome networks and relating disease-specific patterns. However, most existing graph methods explicitly require known graph structures, which are not available in the sophisticated brain system. Especially in heterogeneous multi-modal brain networks, there exists a great challenge to model interactions among brain regions in consideration of inter-modal dependencies. In this study, we propose a Multimodal Dynamic Graph Convolution Network (MDGCN) for structural and functional brain network learning. Our method benefits from modeling inter-modal representations and relating attentive multi-model associations into dynamic graphs with a compositional correspondence matrix. Moreover, a bilateral graph convolution layer is proposed to aggregate multi-modal representations in terms of multi-modal associations. Extensive experiments on three datasets demonstrate the superiority of our proposed method in terms of disease classification, with the accuracy of 90.4%, 85.9% and 98.3% in predicting Mild Cognitive Impairment, Parkinson’s Disease, and Schizophrenia respectively. Our statistical evaluations on the correspondence matrix exhibit a high correspondence with previous evidence of biomarkers.

The brain age has been proven to be a phenotype of relevance to cognitive performance and brain disease. Achieving accurate brain age prediction is an essential prerequisite for optimizing the predicted brain-age difference as a biomarker. As a comprehensive biological characteristic, the brain age is hard to be exploited accurately with models using feature engineering and local processing such as local convolution and recurrent operations that process one local neighborhood at a time. Instead, Vision Transformers learn global attentive interaction of patch tokens, introducing less inductive bias and modeling long-range dependencies. In terms of this, we proposed a novel network for learning brain age interpreting with global and local dependencies, where the corresponding representations are captured by Successive Permuted Transformer (SPT) and convolution blocks. The SPT brings computation efficiency and locates the 3D spatial information indirectly via continuously encoding 2D slices from different views. Finally, we collect a large cohort of 22645 subjects with ages ranging from 14 to 97 and our network performed the best among a series of deep learning methods, yielding a mean absolute error (MAE) of 2.855 in validation set, and 2.911 in an independent test set.

In this article, we focus on a wireless-powered body area network in which the simultaneous wireless information and power transfer (SWIPT) technique is adopted. We consider two scenarios based on whether sensor nodes (SNs) are equipped with battery. For the first time, energy consumption minimization with throughput heterogeneity (ECM-TH) problem is addressed for both scenarios. For the battery-free scenario, a low-complexity time allocation scheme is proposed. This scheme solves the ECM-TH problem based on a hybrid method of gradient descent and bisection search algorithms. Consequently, compared with the interior-point method, our scheme has a lower computational complexity for the same energy consumption performance of the network. For the battery-assisted scenario, the nonconvex ECM-TH problem is first transformed into a convex optimization problem by introducing auxiliary variables. Then, a joint time and power allocation scheme based on the Lagrange dual subgradient method is proposed to solve it. Compared with the battery-free scenario, energy consumption and outage probability are both decreased in the battery-assisted scenario. Moreover, we address a special case wherein the feasible set of the above-mentioned ECM-TH problems may be empty owing to poor channel conditions or high throughput requirements of SNs.

Phase synchronization phenomena are directly connected with the underlying neural mechanisms of certain cognitive processes. However, only the amplitude information is utilized in most electroencephalogram (EEG)-based brain-computer interfaces (BCIs). Few of the existing methods can simultaneously measure the amplitude and phase information required for classification. In this study, a novel common amplitude-phase measurement (CAPM) method is proposed. This method is capable of jointly measuring the phase and amplitude information of EEG signals on the Riemannian manifold. The proposed CAPM method comprises a two-step approach. First, a novel Riemannian graph embedding is proposed for dimensionality reduction while performing spatial-spectral filtering. The graph embedding is excellent in capturing the intrinsic features contained by the physiological signal. Second, to enhance robustness, a novel classifier is designed to incorporate the regularized linear regression in the computation of Riemannian distance. Experimental results on two BCI competition datasets demonstrate CAPM can yield high classification performance. The proposed CAPM method is a promising tool in analyzing EEG amplitude-phase characteristics and exhibits great potential in BCI applications.