Functional magnetic resonance imaging (fMRI) allows the observation of brain functional connectivity patterns. Attention-based diagnostic models have been widely applied in fMRI data for brain disease diagnosis. However, the global attention mechanism of the Transformer faces challenges in adaptively identifying and focusing on significant brain regions and connections relevant to disease diagnosis while reducing attention to non-relevant regions and connections in fMRI data, as well as the degradation problem of the attention mechanism, thereby limiting the improvement in diagnostic accuracy. To address these problems, we propose a connection-mask-residual focused attention network (Trifocal Transformer) based on fMRI data for brain disease diagnosis. In the Trifocal Transformer, a Connection Focus Module is developed to simulate brain functional connectivity, thereby enhancing the attention mechanism's ability to focus on significant regions and connections relevant to disease diagnosis. To mitigate the potential negative impact of non-focused regions in the attention map, a learnable Mask Focus Module is designed to adaptively reduce attention to non-relevant regions and connections. To address the degradation of the attention mechanism's focusing ability, we establish Residual Focus Connections between the attention maps, which reinforce the focusing effect across layers and ensure stable attention to significant features. Comprehensive experimental results demonstrate that the Trifocal Transformer achieves superior diagnostic accuracies of 74.1% and 71.2% on ADHD-200 and ABIDE I datasets, respectively. Furthermore, our method reveals potentially disease-related regions of interest (ROIs), providing a new neuroimaging perspective for brain disease diagnosis and treatment.

Test-Time Adaptation (TTA) addresses domain shifts between training and testing. However, existing methods assume a homogeneous target domain (e.g., single domain) at any given time. They fail to handle the dynamic nature of real-world data, where single-domain and multiple-domain distributions change over time. We identify that performance drops in multiple-domain scenarios are caused by batch normalization errors and gradient conflicts, which hinder adaptation. To solve these challenges, we propose Domain Diversity Adaptive Test-Time Adaptation (DATTA), the first approach to handle TTA under dynamic domain shift data streams. It is guided by a novel domain-diversity score. DATTA has three key components: a domain-diversity discriminator to recognize single- and multiple-domain patterns, domain-diversity adaptive batch normalization to combine source and test-time statistics, and domain-diversity adaptive fine-tuning to resolve gradient conflicts. Extensive experiments show that DATTA significantly outperforms state-of-the-art methods by up to 13%. Code is available at https://github.com/DYW77/DATTA.

Despite progress, deep neural networks still suffer performance declines under distribution shifts between training and test domains, leading to a substantial decrease in Quality of Experience (QoE) for applications. Existing test-time adaptation (TTA) methods are challenged by dynamic, multiple test distributions within batches. We observe that feature distributions across different domains inherently cluster into distinct groups with varying means and variances. This divergence reveals a critical limitation of previous global normalization strategies in TTA, which inevitably distort the original data characteristics. Based on this insight, we propose Feature-based Instance Neighbor Discovery (FIND), which comprises three key components: Layer-Wise Feature Disentanglement (LFD), Feature-Aware Batch Normalization (FABN) and Selective FABN (S-FABN). LFD stably captures features with similar distributions at each layer by constructing graph structures; while FABN optimally combines source statistics with test-time distribution-specific statistics for robust feature representation. Finally, S-FABN determines which layers require feature partitioning and which can remain unified, thus enhancing the efficiency of inference. Extensive experiments demonstrate that FIND significantly outperforms existing methods, achieving up to approximately 30\% accuracy improvement in dynamic scenarios while maintaining computational efficiency. The source code is available at https://github.com/Peanut-255/FIND.

Transformers have been applied to medical image segmentation tasks owing to their excellent longrange modeling capability, compensating for the failure of Convolutional Neural Networks (CNNs) to extract global features. However, the standardized self-attention modules in Transformers, characterized by a uniform and inflexible pattern of attention distribution, frequently lead to unnecessary computational redundancy with high-dimensional data, consequently impeding the model's capacity for precise concentration on salient image regions. Additionally, achieving effective explicit interaction between the spatially detailed features captured by CNNs and the long-range contextual features provided by Transformers remains challenging. In this architecture, we propose a Hybrid Transformer and CNN architecture with Multi-scale Deformable Attention(HMDA), designed to address the aforementioned issues effectively. Specifically, we introduce a Multi-scale Spatially Adaptive Deformable Attention (MSADA) mechanism, which attends to a small set of key sampling points around a reference within the multi-scale features, to achieve better performance. In addition, we propose the Cross Attention Bridge (CAB) module, which integrates multi-scale transformer and local features through channelwise cross attention enriching feature synthesis. HMDA is validated on multiple datasets, and the results demonstrate the effectiveness of our approach, which achieves competitive results compared to the previous methods.

Diffusion magnetic resonance imaging (dMRI) is a non-invasive method for capturing the microanatomical information of tissues by measuring the diffusion weighted signals along multiple directions, which is widely used in the quantification of microstructures. Obtaining microscopic parameters requires dense sampling in the q space, leading to significant time consumption. The most popular approach to accelerating dMRI acquisition is to undersample the q-space data, along with applying deep learning methods to reconstruct quantitative diffusion parameters. However, the reliance on a predetermined q-space sampling strategy often constrains traditional deep learning-based reconstructions. The present study proposed a novel deep learning model, named attention-based q-space deep learning (aqDL), to implement the reconstruction with variable q-space sampling strategies. The aqDL maps dMRI data from different scanning strategies onto a common feature space by using a series of Transformer encoders. The latent features are employed to reconstruct dMRI parameters via a multilayer perceptron. The performance of the aqDL model was assessed utilizing the Human Connectome Project datasets at varying undersampling numbers. To validate its generalizability, the model was further tested on two additional independent datasets. Our results showed that aqDL consistently achieves the highest reconstruction accuracy at various undersampling numbers, regardless of whether variable or predetermined q-space scanning strategies are employed. These findings suggest that aqDL has the potential to be used on general clinical dMRI datasets.

Brain lesion segmentation is crucial for neurological disease research and diagnosis. As different types of lesions exhibit distinct characteristics on different imaging modalities, segmentation methods are typically developed in a task-specific manner, where each segmentation model is tailored to a specific lesion type and modality. However, the use of task-specific models requires predetermination of the lesion type and imaging modality, which complicates their deployment in real-world scenarios. In this work, we propose a universal foundation model for brain lesion segmentation on magnetic resonance imaging (MRI), which can automatically segment different types of brain lesions given input of various MRI modalities. We develop a novel Mixture of Modality Experts (MoME) framework with multiple expert networks attending to different imaging modalities. A hierarchical gating network is proposed to combine the expert predictions and foster expertise collaboration. Moreover, to avoid the degeneration of each expert network, we introduce a curriculum learning strategy during training to preserve the specialisation of each expert. In addition to MoME, to handle the combination of multiple input modalities, we propose MoME+, which uses a soft dispatch network for input modality routing. We evaluated the proposed method on nine brain lesion datasets, encompassing five imaging modalities and eight lesion types. The results show that our model outperforms state-of-the-art universal models for brain lesion segmentation and achieves promising generalisation performance onto unseen datasets.

Radiation therapy plays a crucial role in cancer treatment, necessitating precise delivery of radiation to tumors while sparing healthy tissues over multiple days. Computed tomography (CT) is integral for treatment planning, offering electron density data crucial for accurate dose calculations. However, accurately representing patient anatomy is challenging, especially in adaptive radiotherapy, where CT is not acquired daily. Magnetic resonance imaging (MRI) provides superior soft-tissue contrast. Still, it lacks electron density information, while cone beam CT (CBCT) lacks direct electron density calibration and is mainly used for patient positioning. Adopting MRI-only or CBCT-based adaptive radiotherapy eliminates the need for CT planning but presents challenges. Synthetic CT (sCT) generation techniques aim to address these challenges by using image synthesis to bridge the gap between MRI, CBCT, and CT. The SynthRAD2023 challenge was organized to compare synthetic CT generation methods using multi-center ground truth data from 1080 patients, divided into two tasks: (1) MRI-to-CT and (2) CBCT-to-CT. The evaluation included image similarity and dose-based metrics from proton and photon plans. The challenge attracted significant participation, with 617 registrations and 22/17 valid submissions for tasks 1/2. Top-performing teams achieved high structural similarity indices (≥0.87/0.90) and gamma pass rates for photon (≥98.1%/99.0%) and proton (≥97.3%/97.0%) plans. However, no significant correlation was found between image similarity metrics and dose accuracy, emphasizing the need for dose evaluation when assessing the clinical applicability of sCT. SynthRAD2023 facilitated the investigation and benchmarking of sCT generation techniques, providing insights for developing MRI-only and CBCT-based adaptive radiotherapy. It showcased the growing capacity of deep learning to produce high-quality sCT, reducing reliance on conventional CT for treatment planning.

Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.

Heart is the most important organ of the human body, and Electrocardiogram (ECG) is an essential tool for clinical monitoring of heart health and detecting cardiovascular diseases. Automatic detection of ECG anomalies is of great significance and clinical value in healthcare. However, performing automatic anomaly detection for the ECG data is challenging because we not only need to accurately detect the anomalies but also need to provide clinically meaningful interpretation of the results. Existing works on automatic ECG anomaly detection either rely on hand-crafted designs of feature extraction algorithms which are typically too simple to deliver good performance, or deep learning for automatically extracting features, which is not interpretable.

Diffusion magnetic resonance imaging (dMRI) is an important tool in characterizing tissue microstructure based on biophysical models, which are typically multi-compartmental models with mathematically complex and highly non-linear forms. Resolving microstructures from these models with conventional optimization techniques is prone to estimation errors and requires dense sampling in the q-space with a long scan time. Deep learning based approaches have been proposed to overcome these limitations. Motivated by the superior performance of the Transformer in feature extraction than the convolutional structure, in this work, we present a learning-based framework based on Transformer, namely, a Microstructure Estimation Transformer with Sparse Coding (METSC) for dMRI-based microstructural parameter estimation. To take advantage of the Transformer while addressing its limitation in large training data requirement, we explicitly introduce an inductive bias-model bias into the Transformer using a sparse coding technique to facilitate the training process. Thus, the METSC is composed with three stages, an embedding stage, a sparse representation stage, and a mapping stage. The embedding stage is a Transformer-based structure that encodes the signal in a high-level space to ensure the core voxel of a patch is represented effectively. In the sparse representation stage, a dictionary is constructed by solving a sparse reconstruction problem that unfolds the Iterative Hard Thresholding (IHT) process. The mapping stage is essentially a decoder that computes the microstructural parameters from the output of the second stage, based on the weighted sum of normalized dictionary coefficients where the weights are also learned. We tested our framework on two dMRI models with downsampled q-space data, including the intravoxel incoherent motion (IVIM) model and the neurite orientation dispersion and density imaging (NODDI) model. The proposed method achieved up to 11.25 folds of acceleration while retaining high fitting accuracy for NODDI fitting, reducing the mean squared error (MSE) up to 70% compared with the previous q-space learning approach. METSC outperformed the other state-of-the-art learning-based methods, including the model-free and model-based methods. The network also showed robustness against noise and generalizability across different datasets. The superior performance of METSC indicates its potential to improve dMRI acquisition and model fitting in clinical applications.