Neural coupling is a fundamental mechanism in neuroscience that facilitates the emergence of cognitive functions through dynamic interactions and synchronization among distributed brain regions. Inspired by this principle, we pose the question: Might the biological mechanism of neural oscillatory synchronization inspire the feature representation learning for neuroscience? By addressing this question through the Kuramoto model, renowned for simulating oscillatory dynamics, we present a novel physics-informed deep model, `SyncBrain`, it models brain regions as interacting oscillatory units and simulates their temporal dynamics and synchronization patterns to distinguish cognitive states. Furthermore, inspired by the brain's inherent ability to dynamically attend to critical temporal information, we incorporate an adaptive control module that introduces an attention-like mechanism to guide information flow. We evaluate our model on multiple functional neuroimaging datasets, it demonstrates promising performance and enhanced interpretability in both cognitive state decoding and early disease diagnosis, outperforming existing computational methods. These results demonstrate the effectiveness of neural oscillatory mechanisms in shaping robust and interpretable machine learning models for neuroscience applications.

A reliable foundation model of functional neuroimages is critical to promote clinical applications where the performance of current AI models is significantly impeded by a limited sample size. To that end, tremendous efforts have been made to pretraining large models on extensive unlabeled fMRI data using scalable self-supervised learning. Since self-supervision is not necessarily aligned with the brain-to-outcome relationship, most foundation models are suboptimal to the downstream task, such as predicting disease outcomes. By capitalizing on rich environmental variables and demographic data along with an unprecedented amount of functional neuroimages, we form the brain modeling as a multitask learning and present a scalable model architecture for (i) multitask pretraining by tokenizing multiple brain-environment interactions (BEI) and (ii) semi-supervised finetuning by assigning pseudo-labels of default BEI. We have evaluated our foundation model on a variety of applications, including sex prediction, human behavior recognition, and disease early diagnosis of Autism, Parkinson's disease, Alzheimer's disease, and Schizophrenia, where promising results indicate the great potential to facilitate current neuroimaging applications in clinical routines.

Functional Magnetic Resonance Image (fMRI) is commonly employed to study human brain activity, since it offers insight into the relationship between functional fluctuations and human behavior. To enhance analysis and comprehension of brain activity, Graph Neural Networks (GNNs) have been widely applied to the analysis of functional connectivities (FC) derived from fMRI data, due to their ability to capture the synergistic interactions among brain regions. However, in the human brain, performing complex tasks typically involves the activation of certain pathways, which could be represented as paths across graphs. As such, conventional GNNs struggle to learn from these pathways due to the long-range dependencies of multiple pathways. To address these challenges, we introduce a novel framework BrainMAP to learn multiple pathways in brain networks. BrainMAP leverages sequential models to identify long-range correlations among sequentialized brain regions and incorporates an aggregation module based on Mixture of Experts (MoE) to learn from multiple pathways. Our comprehensive experiments highlight BrainMAP's superior performance. Furthermore, our framework enables explanatory analyses of crucial brain regions involved in tasks.

Modeling the progression of neurodegenerative diseases such as Alzheimer’s disease (AD) is crucial for early detection and prevention given their irreversible nature. However, the scarcity of longitudinal data and complex disease dynamics make the analysis highly challenging. Moreover, longitudinal samples often contain irregular and large intervals between subject visits, which underscore the necessity for advanced data generation techniques that can accurately simulate disease progression over time. In this regime, we propose a novel conditional generative model for synthesizing longitudinal sequences and present its application to neurodegenerative disease data generation conditioned on multiple time-dependent ordinal factors, such as age and disease severity. Our method sequentially generates continuous data by bridging gaps between sparse data points with a diffusion model, ensuring a realistic representation of disease progression. The synthetic data are curated to integrate both cohort-level and individual-specific characteristics, where the cohort-level representations are modeled with an ordinal regression to capture longitudinally monotonic behavior. Extensive experiments on four AD biomarkers validate the superiority of our method over nine baseline approaches, highlighting its potential to be applied to a variety of longitudinal data generation.

State‑space models (SSMs) have become a cornerstone for unraveling brain dynamics, capturing how latent neural states evolve over time and give rise to observed signals. By combining deep learning’s flexibility with SSMs’ principled dynamical structure, recent studies have achieved powerful fits to functional neuroimaging data. However, most approaches still view the brain as a set of loosely connected regions or impose oversimplified network priors, falling short of a truly holistic, self‐organized dynamical system perspective. Brain functional connectivity (FC) at each time point naturally forms a symmetric positive definite (SPD) matrix, which lives on a curved Riemannian manifold rather than in Euclidean space. Capturing the trajectories of these SPD matrices is key to understanding how coordinated networks support cognition and behavior. To this end, we introduce *GeoDynamics*, a geometric state space neural network that tracks latent brain state trajectories directly on the high‑dimensional SPD manifold. *GeoDynamics* embeds each connectivity matrix into a manifold‑aware recurrent framework, learning smooth, geometry‑respecting transitions that reveal task‐driven state changes and early markers of Alzheimer’s, Parkinson’s, and autism. Beyond neuroscience, we validate *GeoDynamics* on human action recognition benchmarks (UTKinect, Florence, HDM05), demonstrating its scalability and robustness in modeling complex spatiotemporal dynamics across diverse domains.

Alzheimer’s disease (AD) is marked by cognitive decline along with the widespread of tau aggregates across the brain cortex. Due to the challenges of imaging pathology spreading flows \textit{in vivo}, however, quantitative analysis on the cortical pathways of tau propagation and its interaction with the cascade of amyloid-beta (A$\beta$) plaques lags behind the experimental insights of underlying pathophysiological mechanisms. To address this challenge, we present a physics-informed neural network, empowered by mean-field theory, to uncover the biologically meaningful spreading pathways of tau aggregates between two longitudinal snapshots. Following the notion of `prion-like' mechanism in AD, we first formulate the dynamics of tau propagation as a mean-field game (MFG), where the spread of tau aggregate at each location (aka. agent) depends on the collective behavior of the surrounding agents as well as the potential field formed by amyloid burden. Given the governing equation of propagation dynamics, MFG reaches an equilibrium that allows us to model the evolution of tau aggregates as an optimal transport with the lowest cost in \textit{Wasserstein} space. By leveraging the variational primal-dual structure in MFG, we propose a \textit{Wasserstein}-1 Lagrangian generative adversarial network (GAN), in which a Lipschitz critic seeks the appropriate transport cost at the population level and a generator parameterizes the flow fields of optimal transport across individuals. Additionally, we incorporate a symbolic regression module to derive an explicit formulation capturing the A$\beta$-tau crosstalk. Experimental results on public neuroimaging datasets demonstrate that our explainable deep model not only yields precise and reliable predictions of future tau progression for unseen new subjects but also provides a new window to uncover new understanding of pathology propagation in AD through learning-based approaches.

In typical graph neural networks (GNNs), feature representation learning naturally evolves through iteratively updating node features and exchanging information based on graph topology. In this context, we conceptualize that the learning process in GNNs is a mean-field game (MFG), where each graph node is an agent, interacting with its topologically connected neighbors. However, current GNNs often employ the identical MFG strategy across different graph datasets, regardless of whether the graph exhibits homophilic or heterophilic characteristics. To address this challenge, we propose to formulate the learning mechanism into a variational framework of the MFG inverse problem, introducing an in-context selective message passing paradigm for each agent, which promotes the best overall outcome for the graph. Specifically, we seek for the application-adaptive transportation function (controlling information exchange throughout the graph) and reaction function (controlling feature representation learning on each agent), \textit{on the fly}, which allows us to uncover the most suitable selective mechanism of message passing by solving an MFG variational problem through the lens of Hamiltonian flows. Taken together, our variational framework unifies existing GNN models into various mean-field games with distinct equilibrium states, each characterized by the learned in-context message passing operators. Furthermore, we present an agnostic end-to-end deep model, coined \textit{Game-of-GNN}, to jointly identify the message passing mechanism and fine-tune the GNN hyper-parameters on top of the elucidated message passing operators. \textit{Game-of-GNN} has achieved SOTA performance on diverse graph data, including popular benchmark datasets and human connectomes. More importantly, the mathematical insight of MFG framework provides a new window to understand the foundational principles of graph learning as an interactive dynamical system, which allows us to reshape the idea of designing next-generation GNN models.

In both neuroscience and artificial intelligence (AI), it is well-established that neural “coupling” gives rise to dynamically distributed systems. These systems exhibit self-organized spatiotemporal patterns of synchronized neural oscillations, enabling the representation of abstract concepts. By capitalizing on the unprecedented amount of human neuroimaging data, we propose that advancing the theoretical understanding of rhythmic coordination in neural circuits can offer powerful design principles for the next generation of machine learning models with improved efficiency and robustness. To this end, we introduce a physics-informed deep learning framework for \underline{B}rain \underline{R}hythm \underline{I}dentification by \underline{K}uramoto and \underline{C}ontrol (coined \textit{BRICK}) to characterize the synchronization of neural oscillations that shapes the dynamics of evolving cognitive states. Recognizing that brain networks are structurally connected yet behaviorally dynamic, we further conceptualize rhythmic neural activity as an artificial dynamical system of coupled oscillators, offering a shared mechanistic bridge to brain-inspired machine intelligence. By treating each node as an oscillator interacting with its neighbors, this approach moves beyond the conventional paradigm of graph heat diffusion and establishes a new regime of representation compression through oscillatory synchronization. Empirical evaluations demonstrate that this synchronization-driven mechanism not only mitigates over-smoothing in deep GNNs but also enhances the model’s capacity for reasoning and solving complex graph-based problems.

Generating realistic graphs faces challenges in estimating accurate distribution of graphs in an embedding space while preserving structural characteristics. However, existing graph generation methods primarily focus on approximating the joint distribution of nodes and edges, often overlooking topological properties such as connected components and loops, hindering accurate representation of global structures. To address this issue, we propose a Topology-Aware diffusion-based Graph Generation (TAGG), which aims to sample synthetic graphs that closely resemble the structural characteristics of the original graph based on persistent homology. Specifically, we suggest two core components: 1) Persistence Diagram Matching (PDM) loss which ensures high topological fidelity of generated graphs, and 2) topology-aware attention module (TAM) which induces the denoising network to capture the homological characteristics of the original graphs. Extensive experiments on conventional graph benchmarks demonstrate the effectiveness of our approach demonstrating high generation performance across various metrics, while achieving closer alignment with the distribution of topological features observed in the original graphs. Furthermore, application to real brain network data showcases its potential for complex and real graph applications.

Given the large scale of public functional Magnetic Resonance Imaging (fMRI), e.g., UK Biobank (UKB) and Human Connectome Projects (HCP), brain foundation models are emerging. Although the amount of samples under rich environmental variables is unprecedented, existing brain foundation models learn from fMRI derived from a narrow range of cognitive states stimulated by similar environments, causing the limited robustness demonstrated in various applications and datasets acquired with different pipelines and limited sample size. By capitalizing on the variety of cognitive status as subjects performing explicit tasks, we present the mixture of brain experts, namely BrainMoE, pre-training on tasking fMRI with rich behavioral tasks in addition to resting fMRI for a robust brain foundation model. Brain experts are designed to produce embeddings for different behavioral tasks related to cognition. Afterward, these cognition embeddings are mixed by a cognition adapter via cross-attention so that BrainMoE can handle orthogonal embeddings and be robust on those boutique downstream datasets. We have pre-trained two existing self-regressive architectures and one new supervised architecture as brain experts on 68,251 fMRI scans among UKB and HCP, containing 12 different cognitive states. Then, BrainMoE is evaluated on a variety of applications, including sex, age prediction, human behavior recognition, disease early diagnosis of Autism, Parkinson's disease, Alzheimer's disease, and Schizophrenia, and fMRI-EEG multimodal applications, where promising results in eight datasets from three different pipelines indicate great potential to facilitate current neuroimaging applications in clinical routines.

A fundamental challenge in cognitive neuroscience is understanding how cognition emerges from the interplay between structural connectivity (SC) and dynamic functional connectivity (FC) in the brain. Network neuroscience has emerged as a powerful framework to understand brain function through a holistic perspective on structure-function relationships. In this context, current machine learning approaches typically seek to establish direct mappings between structural connectivity (SC) and functional connectivity (FC) associated with specific cognitive states. However, these state-independent methods often yield inconsistent results due to overlapping brain networks across cognitive states. To address this limitation, we conceptualize to uncover the dendritic coupling mechanism between one static SC and multiple FCs by solving a flow problem that bridges the distribution of SC to a mixed distribution of FCs, conditioned on various cognitive states, along a Riemannian manifold of symmetric positive-definite (SPD) manifold. We further prove the equivalence between flow matching on the SPD manifold and on the computationally efficient Cholesky manifold. Since a spare of functional connections is shared across cognitive states, we introduce the notion of consensus control to promote the shared kinetic structures between multiple FC-to-SC pathways via synchronized coordination, yielding a biologically meaningful underpinning on SC-FC coupling mechanism. Together, we present BrainFlow, a reversible generative model that achieves state-of-the-art performance on not only synthetic data but also large-scale neuroimaging datasets from UK Biobank and Human Connectome Project.

Emerging researchindicates that the degenerative biomarkers associated with Alzheimer's disease (AD) exhibit a non-random distribution within the cerebral cortex, instead following the structural brain network. The alterations in brain networks occur much earlier than the onset of clinical symptoms, thereby affecting the progression of brain disease. In this context, the utilization of computational methods to ascertain the propagation patterns of neuropathological events would contribute to the comprehension of the pathophysiological mechanism involved in the evolution of AD. Despite the encouraging findings achieved by existing graph-based deep learning approaches in analyzing irregular graph data, their applications in identifying the spreading pathway of neuropathology are limited due to two disadvantages. They include (1) lack of a common brain network as an unbiased reference basis for group comparison, and (2) lack of an appropriate mechanism for the identification of propagation patterns. To this end, we propose a proof-of-concept harmonic wavelet neural network (HWNN) to predict the early stage of AD and localize disease-related significant wavelets, which can be used to characterize the spreading pathways of neuropathological events across the brain network. The extensive experiments constructed on both synthetic and real datasets demonstrate that our proposed method achieves superior performance in classification accuracy and statistical power of identifying propagation patterns, compared with other representative approaches.

Positron Emission Tomography (PET) is essential for understanding the pathophysiological mechanisms underlying neurodegenerative diseases like Alzheimer's disease (AD). However, existing approaches primarily focus on stereotypical patterns of pathology burden, lacking the ability to elucidate the underlying propagation mechanisms by which pathologies spread throughout the brain over time. Given that many neurodegenerative diseases exhibit prion-like pathology spread, it is essential to uncover the spot-to-spot flow field between consecutive PET snapshots. To address this, we reformulate the problem of identifying latent cortical propagation pathways of neuropathological burden within the well-established framework of optimal mass transport (OMT). In this formulation, the dynamic spreading of pathology across longitudinal PET scans is inherently constrained by the geometry of the brain cortex. To solve this problem, we introduce a variational framework that characterizes the dynamical system of pathology propagation in the brain, ultimately reducing to a Wasserstein geodesic between two density distributions of pathology accumulation. Furthermore, we hypothesize that a well-characterized mechanism of pathology propagation will enable the prediction of future pathology accumulation at the individual level, paving the way for personalized disease progression modeling. Building on the principles of physics-informed deep models, we derive the governing equation of the underlying OMT model and introduce an explainable, generative adversarial network-inspired framework. Our approach (1) parameterizes population-level OMT dynamics through a flow adjuster and (2) predicts the spreading flow in unseen subjects using a trained flow driver. We validate the accuracy of our model on publicly available datasets, demonstrating its effectiveness in forecasting future pathology accumulation. Since our deep model adheres to the second law of thermodynamics, we further explore the propagation dynamics of tau aggregates throughout the progression of AD. In contrast to traditional methods, our physics-informed approach enhances both accuracy and interpretability, demonstrating its potential to reveal novel neurobiological mechanisms driving disease progression.

Mounting evidence has revealed that functional brain networks are intrinsically dynamic, undergoing changes over time, even in the resting-state environment. Notably, recent studies have highlighted the existence of a small number of critical brain regions within each functional brain network that exhibit a flexible role in adapting the geometric pattern of brain connectivity over time, referred to as "temporal hub" regions. Therefore, the identification of these temporal hubs becomes pivotal for comprehending the mechanisms that underlie the dynamic evolution of brain connectivity. However, existing spatio-temporal hub identification methods rely on static network-based approaches, wherein each temporal hub region is independently inferred from individual time-segmented networks without considering their temporal consistency and consequently fails to align the evolution of hubs with the dynamic changes in brain states. To address this limitation, we propose a novel spatio-temporal hub identification method that fully leverages dynamic graph embedding to distinguish temporal hubs from peripheral nodes, in which dynamic graph embeddings are learned from both spatial and temporal dimensions. Specifically, to preserve the temporal consistency of evolving networks, we model the dynamic graph embedding as a physical model of time, where the network-to-network transition is mathematically expressed as a total variation of dynamic graph embedding with respect to time. Furthermore, a Grassmannian manifold optimization scheme is introduced to enhance graph embedding learning and capture the time-varying topology of brain networks. Experimental results on both synthetic and real fMRI data demonstrate superior temporal consistency in hub identification, surpassing conventional approaches.

One of the fundamental scientific problems in neuroscience is to have a good understanding of how cognition and behavior emerge from brain function. Since the neuroscience concept of cognitive control parallels the notion of system control in engineering, many computational models formulate the dynamics neural process into a dynamical system, where the hidden states of the complex neural system are modulated by energetic simulations. However, the human brain is a quintessential complex biological system. Current computation models either use neural networks to approximate the underlying dynamics, which makes it difficult to fully understand the system mechanics, or compromise to simplified linear models with very limited power to characterize non-linear and self-organized dynamics along with complex neural activities. To address this challenge, we devise an end-to-end deep model to identify the underlying brain dynamics based on Koopman operator theory, which allows us to model a complex non-linear system in an infinite-dimensional linear space. In the context of reverse engineering, we further propose a biology-inspired control module that adjusts the input (neural activity data) based on feedback to align brain dynamics with the underlying cognitive task. We have applied our deep model to predict cognitive states from a large scale of existing neuroimaging data by identifying the latent dynamic system of functional fluctuations. Promising results demonstrate the potential of establishing a system-level understanding of the intricate relationship between brain function and cognition through the landscape of explainable deep models.

Various Graph Neural Networks (GNN) have been successful in analyzing data in non-Euclidean spaces, however, they have limitations such as oversmoothing, i.e., information becomes excessively averaged as the number of hidden layers increases. The issue stems from the intrinsic formulation of conventional graph convolution where the nodal features are aggregated from a direct neighborhood per layer across the entire nodes in the graph. As setting different number of hidden layers per node is infeasible, recent works leverage a diffusion kernel to redefine the graph structure and incorporate information from farther nodes. Unfortunately, such approaches suffer from heavy diagonalization of a graph Laplacian or learning a large transform matrix. In this regards, we propose a diffusion learning framework where the range of feature aggregation is controlled by the scale of a diffusion kernel. For efficient computation, we derive closed-form derivatives of approximations of the graph convolution with respect to the scale, so that node-wise range can be adaptively learned.With a downstream classifier, the entire framework is made trainable in an end-to-end manner. Our model is tested on various standard datasets for node-wise classification for the state-of-the-art performance, and it is also validated on a real-world brain network data for graph classifications to demonstrate its practicality for Alzheimer classification.

Analysis of neurodegenerative diseases on brain connectomes is important in facilitating early diagnosis and predicting its onset. However, investigation of the progressive and irreversible dynamics of these diseases remains underexplored in cross-sectional studies as its diagnostic groups are considered independent. Also, as in many real-world graphs, brain networks exhibit intricate structures with both homophily and heterophily. To address these challenges, we propose Adaptive Graph diffusion network with Temporal regularization (AGT). AGT introduces node-wise convolution to adaptively capture low (i.e., homophily) and high-frequency (i.e., heterophily) characteristics within an optimally tailored range for each node. Moreover, AGT captures sequential variations within progressive diagnostic groups with a novel temporal regularization, considering the relative feature distance between the groups in the latent space. As a result, our proposed model yields interpretable results at both node-level and group-level. The superiority of our method is validated on two neurodegenerative disease benchmarks for graph classification: Alzheimer’s Disease Neuroimaging Initiative (ADNI) and Parkinson’s Progression Markers Initiative (PPMI) datasets.

The human brain is a complex inter-wired system that emerges spontaneous functional fluctuations. In spite of tremendous success in the experimental neuroscience field, a system-level understanding of how brain anatomy supports various neural activities remains elusive. Capitalizing on the unprecedented amount of neuroimaging data, we present a physics-informed deep model to uncover the coupling mechanism between brain structure and function through the lens of data geometry that is rooted in the widespread wiring topology of connections between distant brain regions. Since deciphering the puzzle of self-organized patterns in functional fluctuations is the gateway to understanding the emergence of cognition and behavior, we devise a geometric deep model to uncover manifold mapping functions that characterize the intrinsic feature representations of evolving functional fluctuations on the Riemannian manifold. In lieu of learning unconstrained mapping functions, we introduce a set of graph-harmonic scattering transforms to impose the brain-wide geometry on top of manifold mapping functions, which allows us to cast the manifold-based deep learning into a reminiscent of *MLP-Mixer* architecture (in computer vision) for Riemannian manifold. As a proof-of-concept approach, we explore a neural-manifold perspective to understand the relationship between (static) brain structure and (dynamic) function, challenging the prevailing notion in cognitive neuroscience by proposing that neural activities are essentially excited by brain-wide oscillation waves living on the geometry of human connectomes, instead of being confined to focal areas.

Human brain is a complex system composed of many components that interact with each other. A well-designed computational model, usually in the format of partial differential equations (PDEs), is vital to understand the working mechanisms that can explain dynamic and self-organized behaviors. However, the model formulation and parameters are often tuned empirically based on the predefined domain-specific knowledge, which lags behind the emerging paradigm of discovering novel mechanisms from the unprecedented amount of spatiotemporal data. To address this limitation, we sought to link the power of deep neural networks and physics principles of complex systems, which allows us to design explainable deep models for uncovering the mechanistic role of how human brain (the most sophisticated complex system) maintains controllable functions while interacting with external stimulations. In the spirit of optimal control, we present a unified framework to design an explainable deep model that describes the dynamic behaviors of underlying neurobiological processes, allowing us to understand the latent control mechanism at a system level. We have uncovered the pathophysiological mechanism of Alzheimer's disease to the extent of controllability of disease progression, where the dissected system-level understanding enables higher prediction accuracy for disease progression and better explainability for disease etiology than conventional (black box) deep models.

Mounting evidence shows that Alzheimer's disease (AD) manifests the dysfunction of the brain network much earlier before the onset of clinical symptoms, making its early diagnosis possible. Current brain network analyses treat high-dimensional network data as a regular matrix or vector, which destroys the essential network topology, thereby seriously affecting diagnosis accuracy. In this context, harmonic waves provide a solid theoretical background for exploring brain network topology. However, the harmonic waves are originally intended to discover neurological disease propagation patterns in the brain, which makes it difficult to accommodate brain disease diagnosis with high heterogeneity. To address this challenge, this article proposes a network manifold harmonic discriminant analysis (MHDA) method for accurately detecting AD. Each brain network is regarded as an instance drawn on a Stiefel manifold. Every instance is represented by a set of orthonormal eigenvectors (i.e., harmonic waves) derived from its Laplacian matrix, which fully respects the topological structure of the brain network. An MHDA method within the Stiefel space is proposed to identify the group-dependent common harmonic waves, which can be used as group-specific references for downstream analyses. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method in stratifying cognitively normal (CN) controls, mild cognitive impairment (MCI), and AD.

Graphs are ubiquitous in various domains, such as social networks and biological systems. Despite the great successes of graph neural networks (GNNs) in modeling and analyzing complex graph data, the inductive bias of locality assumption, which involves exchanging information only within neighboring connected nodes, restricts GNNs in capturing long-range dependencies and global patterns in graphs. Inspired by the classic Brachistochrone problem, we seek how to devise a new inductive bias for cutting-edge graph application and present a general framework through the lens of variational analysis. The backbone of our framework is a two-way mapping between the discrete GNN model and continuous diffusion functional, which allows us to design application-specific objective function in the continuous domain and engineer discrete deep model with mathematical guarantees. First, we address over-smoothing in current GNNs. Specifically, our inference reveals that the existing layer-by-layer models of graph embedding learning are equivalent to a ${\ell _2}$-norm integral functional of graph gradients, which is the underlying cause of the over-smoothing problem. Similar to edge-preserving filters in image denoising, we introduce the total variation (TV) to promote alignment of the graph diffusion pattern with the global information present in community topologies. On top of this, we devise a new selective mechanism for inductive bias that can be easily integrated into existing GNNs and effectively address the trade-off between model depth and over-smoothing. Second, we devise a novel generative adversarial network (GAN) to predict the spreading flows in the graph through a neural transport equation. To avoid the potential issue of vanishing flows, we tailor the objective function to minimize the transportation within each community while maximizing the inter-community flows. Our new GNN models achieve state-of-the-art (SOTA) performance on graph learning benchmarks such as Cora, Citeseer, and Pubmed.

Since brain network organization is essentially governed by the harmonic waves derived from the Eigen-system of the underlying Laplacian matrix, discovering the harmonic-based alterations provides a new window to understand the pathogenic mechanism of Alzheimer's disease (AD) in a unified reference space. However, current reference (common harmonic waves) estimation studies over the individual harmonic waves are often sensitive to outliers, which are obtained by averaging the heterogenous individual brain networks. To address this challenge, we propose a novel manifold learning approach to identify a set of outlier-immunized common harmonic waves. The backbone of our framework is calculating the geometric median of all individual harmonic waves on the Stiefel manifold, instead of Fréchet mean, thus improving the robustness of learned common harmonic waves to the outliers. A manifold optimization scheme with theoretically guaranteed convergence is tailored to solve our method. The experimental results on synthetic data and real data demonstrate that the common harmonic waves learned by our approach are not only more robust to the outliers than the state-of-the-art methods, but also provide a putative imaging biomarker to predict the early stage of AD.

Human brain is a complex yet economically organized system, where a small portion of critical hub regions support the majority of brain functions. The identification of common hub nodes in a population of networks is often simplified as a voting procedure on the set of identified hub nodes across individual brain networks, which ignores the intrinsic data geometry and partially lacks the reproducible findings in neuroscience. Hence, we propose a first-ever group-wise hub identification method to identify hub nodes that are common across a population of individual brain networks. Specifically, the backbone of our method is to learn common graph embedding that can represent the majority of local topological profiles. By requiring orthogonality among the graph embedding vectors, each graph embedding as a data element is residing on the Grassmannian manifold. We present a novel Grassmannian manifold optimization scheme that allows us to find the common graph embeddings, which not only identify the most reliable hub nodes in each network but also yield a population-based common hub node map. Results of the accuracy and replicability on both synthetic and real network data show that the proposed manifold learning approach outperforms all hub identification methods employed in this evaluation.

Image-based cell counting is a fundamental yet challenging task with wide applications in biological research. In this paper, we propose a novel unified deep network framework designed to solve this problem for various cell types in both 2D and 3D images. Specifically, we first propose SAU-Net for cell counting by extending the segmentation network U-Net with a Self-Attention module. Second, we design an extension of Batch Normalization (BN) to facilitate the training process for small datasets. In addition, a new 3D benchmark dataset based on the existing mouse blastocyst (MBC) dataset is developed and released to the community. Our SAU-Net achieves state-of-the-art results on four benchmark 2D datasets - synthetic fluorescence microscopy (VGG) dataset, Modified Bone Marrow (MBM) dataset, human subcutaneous adipose tissue (ADI) dataset, and Dublin Cell Counting (DCC) dataset, and the new 3D dataset, MBC. The BN extension is validated using extensive experiments on the 2D datasets, since GPU memory constraints preclude use of 3D datasets. The source code is available at https://github.com/mzlr/sau-net.

Functional connectivities (FC) of brain network manifest remarkable geometric patterns, which is the gateway to understanding brain dynamics. In this work, we present a novel geometric-attention neural network to characterize the time-evolving brain state change from the functional neuroimages by tracking the trajectory of functional dynamics on high-dimension Riemannian manifold of symmetric positive definite (SPD) matrices. Specifically, we put the spotlight on learning the common state-specific manifold signatures that represent the underlying cognition. In this context, the driving force of our neural network is tied up with the learning of the evolution functionals on the Riemannian manifold of SPD matrix that underlies the known evolving brain states. To do so, we train a convolution neural network (CNN) on the Riemannian manifold of SPD matrices to seek for the putative low-dimension feature representations, followed by an end-to-end recurrent neural network (RNN) to yield the time-varying mapping function of SPD matrices which fits the evolutionary trajectories of the underlying states. Furthermore, we devise a geometric attention mechanism in CNN, allowing us to discover the latent geometric patterns in SPD matrices that are associated with the underlying states. Notably, our work has the potential to understand how brain function emerges behavior by investigating the geometrical patterns from functional brain networks, which is essentially a correlation matrix of neuronal activity signals. Our proposed manifold-based neural network achieves promising results in predicting brain state changes on both simulated data and task functional neuroimaging data from Human Connectome Project, which implies great applicability in neuroscience studies.

The topology of human functional networks is assumed to oscillate during brain states changes. The functional neuroimage is employed to offer a non-invasive window to understand cognition and behaviors by characterizing the functional connections between spatially distinct brain regions. Consequently, identifying the transitions of functional connectivities is the critical step to understanding the mechanism of cognition that might be underlined with neurological disorders. However, little attention has been paid to studying the geometry of the entire functional brain network. To tackle this issue, this paper models the cognition changes on functional brain networks as a set of landmarks residing on a Riemannian manifold. Accordingly, we propose a Riemannian manifold mean shift method to detect cognition changes by identifying the representative function networks of the distribution of functional networks. The manifold mean shift (MMS) method is applied on both simulated data and real functional neuroimaging data, downloaded from Human Connectome Project (HCP). Experimental results demonstrated the MMS achieved highly accurate and consistent cognition change, by comparing three state-of-the-art methods.

BACKGROUND: Recent advances in tissue clearing techniques, combined with high-speed image acquisition through light sheet microscopy, enable rapid three-dimensional (3D) imaging of biological specimens, such as whole mouse brains, in a matter of hours. Quantitative analysis of such 3D images can help us understand how changes in brain structure lead to differences in behavior or cognition, but distinguishing densely packed features of interest, such as nuclei, from background can be challenging. Recent deep learning-based nuclear segmentation algorithms show great promise for automated segmentation, but require large numbers of accurate manually labeled nuclei as training data. RESULTS: We present Segmentor, an open-source tool for reliable, efficient, and user-friendly manual annotation and refinement of objects (e.g., nuclei) within 3D light sheet microscopy images. Segmentor employs a hybrid 2D-3D approach for visualizing and segmenting objects and contains features for automatic region splitting, designed specifically for streamlining the process of 3D segmentation of nuclei. We show that editing simultaneously in 2D and 3D using Segmentor significantly decreases time spent on manual annotations without affecting accuracy as compared to editing the same set of images with only 2D capabilities. CONCLUSIONS: Segmentor is a tool for increased efficiency of manual annotation and refinement of 3D objects that can be used to train deep learning segmentation algorithms, and is available at https://www.nucleininja.org/ and https://github.com/RENCI/Segmentor .

Converging evidence shows that disease-relevant brain alterations do not appear in random brain locations, instead, their spatial patterns follow large-scale brain networks. In this context, a powerful network analysis approach with a mathematical foundation is indispensable to understand the mechanisms of neuropathological events as they spread through the brain. Indeed, the topology of each brain network is governed by its native harmonic waves, which are a set of orthogonal bases derived from the Eigen-system of the underlying Laplacian matrix. To that end, we propose a novel connectome harmonic analysis framework that provides enhanced mathematical insights by detecting frequency-based alterations relevant to brain disorders. The backbone of our framework is a novel manifold algebra appropriate for inference across harmonic waves. This algebra overcomes the limitations of using classic Euclidean operations on irregular data structures. The individual harmonic differences are measured by a set of common harmonic waves learned from a population of individual Eigen-systems, where each native Eigen-system is regarded as a sample drawn from the Stiefel manifold. Specifically, a manifold optimization scheme is tailored to find the common harmonic waves, which reside at the center of the Stiefel manifold. To that end, the common harmonic waves constitute a new set of neurobiological bases to understand disease progression. Each harmonic wave exhibits a unique propagation pattern of neuropathological burden spreading across brain networks. The statistical power of our novel connectome harmonic analysis approach is evaluated by identifying frequency-based alterations relevant to Alzheimer's disease, where our learning-based manifold approach discovers more significant and reproducible network dysfunction patterns than Euclidean methods.

The functional connectomic profile is one of the non-invasive imaging biomarkers in the computer-assisted diagnostic system for many neuro-diseases. However, the diagnostic power of functional connectivity is challenged by mixed frequency-specific neuronal oscillations in the brain, which makes the single Functional Connectivity Network (FCN) often underpowered to capture the disease-related functional patterns. To address this challenge, we propose a novel functional connectivity analysis framework to conduct joint feature learning and personalized disease diagnosis, in a semi-supervised manner, aiming at focusing on putative multi-band functional connectivity biomarkers from functional neuroimaging data. Specifically, we first decompose the Blood Oxygenation Level Dependent (BOLD) signals into multiple frequency bands by the discrete wavelet transform, and then cast the alignment of all fully-connected FCNs derived from multiple frequency bands into a parameter-free multi-band fusion model. The proposed fusion model fuses all fully-connected FCNs to obtain a sparsely-connected FCN (sparse FCN for short) for each individual subject, as well as lets each sparse FCN be close to its neighbored sparse FCNs and be far away from its furthest sparse FCNs. Furthermore, we employ the$\ell _{{1}}$-SVM to conduct joint brain region selection and disease diagnosis. Finally, we evaluate the effectiveness of our proposed framework on various neuro-diseases,i.e.,Fronto-Temporal Dementia (FTD), Obsessive-Compulsive Disorder (OCD), and Alzheimer’s Disease (AD), and the experimental results demonstrate that our framework shows more reasonable results, compared to state-of-the-art methods, in terms of classification performance and the selected brain regions. The source code can be visited by the urlhttps://github.com/reynard-hu/mbbna.

Currently, many studies of Alzheimer's disease (AD) are investigating the neurobiological factors behind the acquisition of beta-amyloid (A), pathologic tau (T), and neurodegeneration ([N]) biomarkers from neuroimages. However, a system-level mechanism of how these neuropathological burdens promote neurodegeneration and why AD exhibits characteristic progression is largely elusive. In this study, we combined the power of systems biology and network neuroscience to understand the dynamic interaction and diffusion process of AT[N] biomarkers from an unprecedented amount of longitudinal Amyloid PET scan, MRI imaging, and DTI data. Specifically, we developed a network-guided biochemical model to jointly (1) model the interaction of AT[N] biomarkers at each brain region and (2) characterize their propagation pattern across the fiber pathways in the structural brain network, where the brain resilience is also considered as a moderator of cognitive decline. Our biochemical model offers a greater mathematical insight to understand the physiopathological mechanism of AD progression by studying the system dynamics and stability. Thus, an in-depth system-level analysis allows us to gain a new understanding of how AT[N] biomarkers spread throughout the brain, capture the early sign of cognitive decline, and predict the AD progression from the preclinical stage.

Brain functional connectivity analysis on fMRI data could improve the understanding of human brain function. However, due to the influence of the inter-subject variability and the heterogeneity across subjects, previous methods of functional connectivity analysis are often insufficient in capturing disease-related representation so that decreasing disease diagnosis performance. In this paper, we first propose a new multi-graph fusion framework to fine-tune the original representation derived from Pearson correlation analysis, and then employ L1-SVM on fine-tuned representations to conduct joint brain region selection and disease diagnosis for avoiding the issue of the curse of dimensionality on high-dimensional data. The multi-graph fusion framework automatically learns the connectivity number for every node (i.e., brain region) and integrates all subjects in a unified framework to output homogenous and discriminative representations of all subjects. Experimental results on two real data sets, i.e., fronto-temporal dementia (FTD) and obsessive-compulsive disorder (OCD), verified the effectiveness of our proposed framework, compared to state-of-the-art methods.

Accurate segmentation of anatomical brain structures is crucial for many neuroimaging applications, e.g., early brain development studies and the study of imaging biomarkers of neurodegenerative diseases. Although multi-atlas segmentation (MAS) has achieved many successes in the medical imaging area, this approach encounters limitations in segmenting anatomical structures associated with poor image contrast. To address this issue, we propose a new MAS method that uses a hypergraph learning framework to model the complex subject-within and subject-to-atlas image voxel relationships and propagate the label on the atlas image to the target subject image. To alleviate the low-image contrast issue, we propose two strategies equipped with our hypergraph learning framework. First, we use a hierarchical strategy that exploits high-level context features for hypergraph construction. Because the context features are computed on the tentatively estimated probability maps, we can ultimately turn the hypergraph learning into a hierarchical model. Second, instead of only propagating the labels from the atlas images to the target subject image, we use a dynamic label propagation strategy that can gradually use increasing reliably identified labels from the subject image to aid in predicting the labels on the difficult-to-label subject image voxels. Compared with the state-of-the-art label fusion methods, our results show that the hierarchical hypergraph learning framework can substantially improve the robustness and accuracy in the segmentation of anatomical brain structures with low image contrast from magnetic resonance (MR) images.

Discriminative methods commonly produce models with relatively good generalization abilities. However, this advantage is challenged in real-world applications (e.g., medical image analysis problems), in which there often exist outlier data points (sample-outliers) and noises in the predictor values (feature-noises). Methods robust to both types of these deviations are somewhat overlooked in the literature. We further argue that denoising can be more effective, if we learn the model using all the available labeled and unlabeled samples, as the intrinsic geometry of the sample manifold can be better constructed using more data points. In this paper, we propose a semi-supervised robust discriminative classification method based on the least-squares formulation of linear discriminant analysis to detect sample-outliers and feature-noises simultaneously, using both labeled training and unlabeled testing data. We conduct several experiments on a synthetic, some benchmark semi-supervised learning, and two brain neurodegenerative disease diagnosis datasets (for Parkinson's and Alzheimer's diseases). Specifically for the application of neurodegenerative diseases diagnosis, incorporating robust machine learning methods can be of great benefit, due to the noisy nature of neuroimaging data. Our results show that our method outperforms the baseline and several state-of-the-art methods, in terms of both accuracy and the area under the ROC curve.

Hyper-graph techniques have been widely investigated in computer vision and medical imaging applications, showing superior performance for modeling complex subject-wise relationships and sufficient flexibility to deal with missing data from multi-modal neuroimaging data. Existing hyper-graph methods, however, are inadequate for two reasons. First, representations are generated only from the observed imaging data, a process that is completely independent of the subsequent data label inference/ classification step. Thus, hyper-graph results constructed in this way may not be consistent with phenotype data such as clinical labels or scores. More critically, it might generate sub-optimal predictions in relation to clinical labels/scores. Second, current hyper-graph inference methods rely on two sequential steps: 1) building the hyper-graph for each individual modality and then predicted latent labels for new subjects upon each constructed hyper-graph and 2) a voting procedure to incorporate inference results across different hyper-graphs. This approach, however, is limited by failing to consider the complex and complementary relationships of multi-modal imaging data with respect to hyper-graph inference procedure. To address these two issues, we propose a novel dynamic hyper-graph inference method supported by a semi-supervised framework. Our method iteratively estimates and adjusts the hyper-graph structures from multi-modal imaging data until consistency between the learned hyper-graph and the observed clinical labels and scores is achieved. This hyper-graph inference framework also eases the integration process of classification (identifying individuals having neurodegenerative disease) and regression (predicting the clinical scores) within the same framework. The experimental results on identifying mild cognition impairment (MCI) subjects and the fine grained recognition of MCI progression stages show improved performance using our proposed hyper-graph inference method compared with conventional methods.

Computed tomography (CT) is widely used for dose planning in the radiotherapy of prostate cancer. However, CT has low tissue contrast, thus making manual contouring difficult. In contrast, magnetic resonance (MR) image provides high tissue contrast and is thus ideal for manual contouring. If MR image can be registered to CT image of the same patient, the contouring accuracy of CT could be substantially improved, which could eventually lead to high treatment efficacy. In this paper, we propose a learning-based approach for multimodal image registration. First, to fill the appearance gap between modalities, a structured random forest with auto-context model is learnt to synthesize MRI from CT and vice versa. Then, MRI-to-CT registration is steered in a dual manner of registering images with same appearances, i.e., (1) registering the synthesized CT with CT, and (2) also registering MRI with the synthesized MRI. Next, a dual-core deformation fusion framework is developed to iteratively and effectively combine these two registration results. Experiments on pelvic CT and MR images have shown the improved registration performance by our proposed method, compared with the existing non-learning based registration methods.

Brain atlases are an essential component in understanding the dynamic cerebral development, especially for the early postnatal period. However, longitudinal atlases are rare for infants, and the existing ones are generally limited by their fuzzy appearance. Moreover, since longitudinal atlas construction is typically performed independently over time, the constructed atlases often fail to preserve temporal consistency. This problem is further aggravated for infant images since they typically have low spatial resolution and insufficient tissue contrast. In this paper, we propose a novel framework for consistent spatial-temporal construction of longitudinal atlases for developing infant brain MR images. Specifically, for preserving structural details, the atlas construction is performed in spatial-temporal wavelet domain simultaneously. This is achieved by a patch-based combination of results from each frequency subband. Compared with the existing infant longitudinal atlases, our experimental results indicate that our approach is able to produce longitudinal atlases with richer structural details and also better longitudinal consistency, thus leading to higher performance when used for spatial normalization of a group of infant brain images.

Recently, multi-atlas segmentation (MAS) has achieved a great success in the medical imaging area. The key assumption of MAS is that multiple atlases encompass richer anatomical variability than a single atlas. Therefore, we can label the target image more accurately by mapping the label information from the appropriate atlas images that have the most similar structures. The problem of atlas selection, however, still remains unexplored. Current state-of-the-art MAS methods rely on image similarity to select a set of atlases. Unfortunately, this heuristic criterion is not necessarily related to segmentation performance and, thus may undermine segmentation results. To solve this simple but critical problem, we propose a learning-based atlas selection method to pick up the best atlases that would eventually lead to more accurate image segmentation. Our idea is to learn the relationship between the pairwise appearance of observed instances (a pair of atlas and target images) and their final labeling performance (in terms of Dice ratio). In this way, we can select the best atlases according to their expected labeling accuracy. It is worth noting that our atlas selection method is general enough to be integrated with existing MAS methods. As is shown in the experiments, we achieve significant improvement after we integrate our method with 3 widely used MAS methods on ADNI and LONI LPBA40 datasets.

Lung 4D-CT provides important anatomical structure and motion information, which can be crucial in radiation therapy for lung cancer. However, radiation dose concerns limit the number of axial slices in 4D-CT, resulting in low superior-inferior resolution. We propose an approach to estimate the intermediate slices for resolution enhancement of 4D-CT. We explore the lung-motion-induced locally complimentary sampling information across respiratory phases, by using the deformation fields between 3D phase-volumes. For better robustness to noise and registration errors, we estimate the unknown intermediate slices in a patch-wise manner. To this end, we compute candidate patches from the available slices in different phases, based on the deformation field estimates. We then linearly combine the candidate patches, using weights computed by solving an h minimization problem. Unlike state-of-the-art methods, our deformation-driven patch-based approach requires a small number of inter-phase candidate patches, and yet outperforms these methods. This highlights the usefulness of considering deformation information in resolution enhancement of lung 4D-CT.

Recently, multiple-atlas segmentation (MAS) has achieved a great success in the medical imaging area. The key assumption is that multiple atlases have greater chances of correctly labeling a target image than a single atlas. However, the problem of atlas selection still remains unexplored. Traditionally, image similarity is used to select a set of atlases. Unfortunately, this heuristic criterion is not necessarily related to the final segmentation performance. To solve this seemingly simple but critical problem, we propose a learning-based atlas selection method to pick up the best atlases that would lead to a more accurate segmentation. Our main idea is to learn the relationship between the pairwise appearance of observed instances (i.e., a pair of atlas and target images) and their final labeling performance (e.g., using the Dice ratio). In this way, we select the best atlases based on their expected labeling accuracy. Our atlas selection method is general enough to be integrated with any existing MAS method. We show the advantages of our atlas selection method in an extensive experimental evaluation in the ADNI, SATA, IXI, and LONI LPBA40 datasets. As shown in the experiments, our method can boost the performance of three widely used MAS methods, outperforming other learning-based and image-similarity-based atlas selection methods.

In the above paper (ibid., vol. 33, no. 10, pp. 1939-1953, Oct. 2014), Gerard Sanroma was incorrectly indicated as the corresponding author. Dinggang Shen should have been indicated as the corresponding author.

Recently, group wise registration has been investigated for simultaneous alignment of all images without selecting any individual image as the template, thus avoiding the potential bias in image registration. However, none of current group wise registration method fully utilizes the image distribution to guide the registration. Thus, the registration performance usually suffers from large inter-subject variations across individual images. To solve this issue, we propose a novel group wise registration algorithm for large population dataset, guided by the image distribution on the manifold. Specifically, we first use a graph to model the distribution of all image data sitting on the image manifold, with each node representing an image and each edge representing the geodesic pathway between two nodes (or images). Then, the procedure of warping all images to their population center turns to the dynamic shrinking of the graph nodes along their graph edges until all graph nodes become close to each other. Thus, the topology of image distribution on the image manifold is always preserved during the group wise registration. More importantly, by modeling the distribution of all images via a graph, we can potentially reduce registration error since every time each image is warped only according to its nearby images with similar structures in the graph. We have evaluated our proposed group wise registration method on both synthetic and real datasets, with comparison to the two state-of-the-art group wise registration methods. All experimental results show that our proposed method achieves the best performance in terms of registration accuracy and robustness.

Robust anatomical correspondence detection is a key step in many medical image applications such as image registration and motion correction. In the computer vision field, graph matching techniques have emerged as a powerful approach for correspondence detection. By considering potential correspondences as graph nodes, graph edges can be used to measure the pairwise agreement between possible correspondences. In this paper, we present a novel, hierarchical graph matching method with sparsity constraint to further augment the power of conventional graph matching methods in establishing anatomical correspondences, especially for the cases of large inter-subject variations in medical applications. Specifically, we first propose to measure the pairwise agreement between potential correspondences along a sequence of intensity profiles which reduces the ambiguity in correspondence matching. We next introduce the concept of sparsity on the fuzziness of correspondences to suppress the distraction from misleading matches, which is very important for achieving the accurate, one-to-one correspondences. Finally, we integrate our graph matching method into a hierarchical correspondence matching framework, where we use multiple models to deal with the large inter-subject anatomical variations and gradually refine the correspondence matching results between the tentatively deformed model images and the underlying subject image. Evaluations on both synthetic data and public hand X-ray images indicate that the proposed hierarchical sparse graph matching method yields the best correspondence matching performance in terms of both accuracy and robustness when compared with several conventional graph matching methods.

4D-CT plays an important role in lung cancer treatment. However, due to the inherent high-dose exposure associated with CT, dense sampling along superior-inferior direction is often not practical. As a result, artifacts such as lung vessel discontinuity and partial volume are typical in 4D-CT images and might mislead dose administration in radiation therapy. In this paper, we present a novel patch-based technique for super-resolution enhancement of the 4D-CT images along the superior-inferior direction. Our working premise is that the anatomical information that is missing at one particular phase can be recovered from other phases. Based on this assumption, we employ a patch-based mechanism for guided reconstruction of super-resolution axial slices. Specifically, to reconstruct each targeted super-resolution slice for a CT image at a particular phase, we agglomerate a dictionary of patches from images of all other phases in the 4D-CT sequence. Then we perform a sparse combination of the patches in this dictionary to reconstruct details of a super-resolution patch, under constraint of similarity to the corresponding patches in the neighboring slices. By iterating this procedure over all possible patch locations, a superresolution 4D-CT image sequence with enhanced anatomical details can be eventually reconstructed. Our method was extensively evaluated using a public dataset. In all experiments, our method outperforms the conventional linear and cubic-spline interpolation methods in terms of preserving image details and suppressing misleading artifacts.

In this paper, we propose a general framework for performance improvement of the current state-of-the-art registration algorithms in terms of both accuracy and computation time. The key concept involves rapid prediction of a deformation field for registration initialization, which is achieved by a statistical correlation model learned between image appearances and deformation fields. This allows us to immediately bring a template image as close as possible to a subject image that we need to register. The task of the registration algorithm is hence reduced to estimating small deformation between the subject image and the initially warped template image, i.e., the intermediate template (IT). Specifically, to obtain a good subject-specific initial deformation, support vector regression is utilized to determine the correlation between image appearances and their respective deformation fields. When registering a new subject onto the template, an initial deformation field is first predicted based on the subject's image appearance for generating an IT. With the IT, only the residual deformation needs to be estimated, presenting much less challenge to the existing registration algorithms. Our learning-based framework affords two important advantages: 1) by requiring only the estimation of the residual deformation between the IT and the subject image, the computation time can be greatly reduced; 2) by leveraging good deformation initialization, local minima giving suboptimal solution could be avoided. Our framework has been extensively evaluated using medical images from different sources, and the results indicate that, on top of accuracy improvement, significant registration speedup can be achieved, as compared with the case where no prediction of initial deformation is performed.

4D-CT plays an important role in lung cancer treatment because of its capability in providing a comprehensive characterization of respiratory motion for high-precision radiation therapy. However, due to the inherent high-dose exposure associated with CT, dense sampling along superior-inferior direction is often not practical, thus resulting in an inter-slice thickness that is much greater than in-plane voxel resolutions. As a consequence, artifacts such as lung vessel discontinuity and partial volume effects are often observed in 4D-CT images, which may mislead dose administration in radiation therapy. In this paper, we present a novel patch-based technique for resolution enhancement of 4D-CT images along the superior-inferior direction. Our working premise is that anatomical information that is missing in one particular phase can be recovered from other phases. Based on this assumption, we employ a hierarchical patch-based sparse representation mechanism to enhance the superior-inferior resolution of 4D-CT by reconstructing additional intermediate CT slices. Specifically, for each spatial location on an intermediate CT slice that we intend to reconstruct, we first agglomerate a dictionary of patches from images of all other phases in the 4D-CT. We then employ a sparse combination of patches from this dictionary, with guidance from neighboring (upper and lower) slices, to reconstruct a series of patches, which we progressively refine in a hierarchical fashion to reconstruct the final intermediate slices with significantly enhanced anatomical details. Our method was extensively evaluated using a public dataset. In all experiments, our method outperforms the conventional linear and cubic-spline interpolation methods in preserving image details and also in suppressing misleading artifacts, indicating that our proposed method can potentially be applied to better image-guided radiation therapy of lung cancer in the future.

A novel groupwise registration framework, called Atlas Building by Self-Organized Registration and Bundling (ABSORB), is proposed in this paper. In this framework, the global structure of relative subject image distribution is preserved during the registration by constraining each subject to deform locally within the learned manifold. A self-organized registration is employed to deform each subject towards a subset of its neighbors that are closer to the global center. Some subjects close enough in the manifold will be bundled into a subgroup during the registration, and then deformed together in the subsequent registration process. This framework performs groupwise registration in a hierarchical way. Specifically, in the higher level, it will perform on a much smaller dataset formed by the representative subjects of all subgroups generated in the previous levels of registration. The atlas image can be eventually built once the registration arrives at the upmost level. Experimental results on both synthetic and real datasets show that the proposed framework can achieve substantial improvements, compared to the other two widely used groupwise methods, in terms of both registration accuracy and robustness.

We propose a novel diffusion tensor imaging (DTI) registration algorithm, called fast tensor image morphing for elastic registration (F-TIMER). F-TIMER leverages multiscale tensor regional distributions and local boundaries for hierarchically driving deformable matching of tensor image volumes. Registration is achieved by utilizing a set of automatically determined structural landmarks, via solving a soft correspondence problem. Based on the estimated correspondences, thin-plate splines are employed to generate a smooth, topology preserving, and dense transformation, and to avoid arbitrary mapping of nonlandmark voxels. To mitigate the problem of local minima, which is common in the estimation of high dimensional transformations, we employ a hierarchical strategy where a small subset of voxels with more distinctive attribute vectors are first deployed as landmarks to estimate a relatively robust low-degrees-of-freedom transformation. As the registration progresses, an increasing number of voxels are permitted to participate in refining the correspondence matching. A scheme as such allows less conservative progression of the correspondence matching towards the optimal solution, and hence results in a faster matching speed. Compared with its predecessor TIMER, which has been shown to outperform state-of-the-art algorithms, experimental results indicate that F-TIMER is capable of achieving comparable accuracy at only a fraction of the computation cost.

This paper presents a learning-based method for deformable registration of magnetic resonance (MR) brain images. There are two novelties in the proposed registration method. First, a set of best-scale geometric features are selected for each point in the brain, in order to facilitate correspondence detection during the registration procedure. This is achieved by optimizing an energy function that requires each point to have its best-scale geometric features consistent over the corresponding points in the training samples, and at the same time distinctive from those of nearby points in the neighborhood. Second, the active points used to drive the brain registration are hierarchically selected during the registration procedure, based on their saliency and consistency measures. That is, the image points with salient and consistent features (across different individuals) are considered for the initial registration of two images, while other less salient and consistent points join the registration procedure later. By incorporating these two novel strategies into the framework of the HAMMER registration algorithm, the registration accuracy has been improved according to the results on simulated brain data, and also visible improvement is observed particularly in the cortical regions of real brain data.