The understanding of the convoluted evolution of infant brain networks during the first postnatal year is pivotal for identifying the dynamics of early brain connectivity development. Thanks to the valuable insights into the brain's anatomy, existing deep learning frameworks focused on forecasting the brain evolution trajectory from a single baseline observation. While yielding remarkable results, they suffer from three major limitations. First, they lack the ability to generalize to multi-trajectory prediction tasks, where each graph trajectory corresponds to a particular imaging modality or connectivity type (e.g., T1-w MRI). Second, existing models require extensive training datasets to achieve satisfactory performance which are often challenging to obtain. Third, they do not efficiently utilize incomplete time series data. To address these limitations, we introduce FedGmTE-Net++, a federated graph-based multi-trajectory evolution network. Using the power of federation, we aggregate local learnings among diverse hospitals with limited datasets. As a result, we enhance the performance of each hospital's local generative model, while preserving data privacy. The three key innovations of FedGmTE-Net++ are: (i) presenting the first federated learning framework specifically designed for brain multi-trajectory evolution prediction in a data-scarce environment, (ii) incorporating an auxiliary regularizer in the local objective function to exploit all the longitudinal brain connectivity within the evolution trajectory and maximize data utilization, (iii) introducing a two-step imputation process, comprising a preliminary K-Nearest Neighbours based precompletion followed by an imputation refinement step that employs regressors to improve similarity scores and refine imputations. Our comprehensive experimental results showed the outperformance of FedGmTE-Net++ in brain multi-trajectory prediction from a single baseline graph in comparison with benchmark methods. Our source code is available at https://github.com/basiralab/FedGmTE-Net-plus.

Computational methods for prediction of the dynamic and complex development of the infant cerebral cortex are critical and highly desired for a better understanding of early brain development in health and disease. Although a few methods have been proposed, they are limited to predicting cortical surface maps at predefined ages and require a large amount of strictly paired longitudinal data at these ages for model training. However, longitudinal infant images are typically acquired at highly irregular and nonuniform scanning ages, thus leading to limited training data for these methods and low flexibility and accuracy. To address these issues, we propose a flexible framework for individualized prediction of cortical surface maps at arbitrary ages during infancy. The central idea is that a cortical surface map can be considered as an entangled representation of two distinct components: 1) the identity-related invariant features, which preserve the individual identity and 2) the age-related features, which reflect the developmental patterns. Our framework, called intensive triplet autoencoder, extracts the mixed latent feature and further disentangles it into two components with an attention-based module. Identity recognition and age estimation tasks are introduced as supervision for a reliable disentanglement. Thus, we can obtain the target individualized cortical property maps with disentangled identity-related information with specific age-related information. Moreover, an adversarial learning strategy is integrated to achieve a vivid and realistic prediction. Extensive experiments validate our method's superior capability in predicting early developing cortical surface maps flexibly and precisely, in comparison with existing methods.

Resting-state functional magnetic resonance imaging (rs-fMRI) is a commonly used functional neuroimaging technique to investigate the functional brain networks. However, rs-fMRI data are often contaminated with noise and artifacts that adversely affect the results of rs-fMRI studies. Several machine/deep learning methods have achieved impressive performance to automatically regress the noise-related components decomposed from rs-fMRI data, which are expressed as the pairs of a spatial map and its associated time series. However, most of the previous methods individually analyze each modality of the noise-related components and simply aggregate the decision-level information (or knowledge) extracted from each modality to make a final decision. Moreover, these approaches consider only the limited modalities making it difficult to explore class-discriminative spectral information of noise-related components. To overcome these limitations, we propose a unified deep attentive spatio-spectral-temporal feature fusion framework. We first adopt a learnable wavelet transform module at the input-level of the framework to elaborately explore the spectral information in subsequent processes. We then construct a feature-level multi-modality fusion module to efficiently exchange the information from multi-modality inputs in the feature space. Finally, we design confidence-based voting strategies for decision-level fusion at the end of the framework to make a robust final decision. In our experiments, the proposed method achieved remarkable performance for noise-related component detection on various rs-fMRI datasets.

In this paper, we present a robust reconstruction scheme for diffusion MRI (dMRI) data acquired using slice-interleaved diffusion encoding (SIDE). When combined with SIDE undersampling and simultaneous multi-slice (SMS) imaging, our reconstruction strategy is capable of significantly reducing the amount of data that needs to be acquired, enabling high-speed diffusion imaging for pediatric, elderly, and claustrophobic individuals. In contrast to the conventional approach of acquiring a full diffusion-weighted (DW) volume per diffusion wavevector, SIDE acquires in each repetition time (TR) a volume that consists of interleaved slice groups, each group corresponding to a different diffusion wavevector. This strategy allows SIDE to rapidly acquire data covering a large number of wavevectors within a short period of time. The proposed reconstruction method uses a diffusion spectrum model and multi-dimensional total variation to recover full DW images from DW volumes that are slice-undersampled due to unacquired SIDE volumes. We formulate an inverse problem that can be solved efficiently using the alternating direction method of multipliers (ADMM). Experiment results demonstrate that DW images can be reconstructed with high fidelity even when the acquisition is accelerated by 25 folds.

Studies have shown that there is a tight connection between cognition skills and brain morphology during infancy. Nonetheless, it is still a great challenge to predict individual cognitive scores using their brain morphological features, considering issues like the excessive feature dimension, small sample size and missing data. Due to the limited data, a compact but expressive feature set is desirable as it can reduce the dimension and avoid the potential overfitting issue. Therefore, we pioneer the path signature method to further explore the essential hidden dynamic patterns of longitudinal cortical features. To form a hierarchical and more informative temporal representation, in this work, a novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores. By introducing the existence embedding in path generation, we can improve the robustness against the missing data. Benefiting from the global temporal receptive field of CF-PSNet, characteristics consisted in the existing data can be fully leveraged. Further, as there is no need for the whole brain to work for a certain cognitive ability, a top K selection module is used to select the most influential brain regions, decreasing the model size and the risk of overfitting. Extensive experiments are conducted on an in-house longitudinal infant dataset within 9 time points. By comparing with several recent algorithms, we illustrate the state-of-the-art performance of our CF-PSNet (i.e., root mean square error of 0.027 with the time latency of 518 milliseconds for each sample).

Infancy is a critical period for the human brain development, and brain age is one of the indices for the brain development status associated with neuroimaging data. The difference between the predicted age based on neuroimaging and the chronological age can provide an important early indicator of deviation from the normal developmental trajectory. In this study, we utilize the Graph Convolutional Network (GCN) to predict the infant brain age based on resting-state fMRI data. The brain connectivity obtained from rs-fMRI can be represented as a graph with brain regions as nodes and functional connections as edges. However, since the brain connectivity is a fully connected graph with features on edges, current GCN cannot be directly used for it is a node-based method for sparse graphs. Hence, we propose an edge-based Graph Path Convolution (GPC) method, which aggregates the information from different paths and can be naturally applied on dense graphs. We refer the whole model as Brain Connectivity Graph Convolutional Networks (BC-GCN). Further, two upgraded network structures are proposed by including the residual and attention modules, referred as BC-GCN-Res and BC-GCN-SE to emphasize the information of the original data and enhance influential channels. Moreover, we design a two-stage coarse-to-fine framework, which determines the age group first and then predicts the age using group-specific BC-GCN-SE models. To avoid accumulated errors from the first stage, a cross-group training strategy is adopted for the second stage regression models. We conduct experiments on infant fMRI scans from 6 to 811 days of age. The coarse-to-fine framework shows significant improvements when being applied to several models (reducing error over 10 days). Comparing with state-of-the-art methods, our proposed model BC-GCN-SE with coarse-to-fine framework reduces the mean absolute error of the prediction from >70 days to 49.9 days. The code is now available at https://github.com/SCUT-Xinlab/BC-GCN.

Deformable registration is fundamental to longitudinal and population-based image analyses. However, it is challenging to precisely align longitudinal infant brain MR images of the same subject, as well as cross-sectional infant brain MR images of different subjects, due to fast brain development during infancy. In this paper, we propose a recurrently usable deep neural network for the registration of infant brain MR images. There are three main highlights of our proposed method. (i) We use brain tissue segmentation maps for registration, instead of intensity images, to tackle the issue of rapid contrast changes of brain tissues during the first year of life. (ii) A single registration network is trained in a one-shot manner, and then recurrently applied in inference for multiple times, such that the complex deformation field can be recovered incrementally. (iii) We also propose both the adaptive smoothing layer and the tissue-aware anti-folding constraint into the registration network to ensure the physiological plausibility of estimated deformations without degrading the registration accuracy. Experimental results, in comparison to the state-of-the-art registration methods, indicate that our proposed method achieves the highest registration accuracy while still preserving the smoothness of the deformation field. The implementation of our proposed registration network is available onlinehttps://github.com/Barnonewdm/ACTA-Reg-Net.

To better understand early brain development in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; h owever, one of the major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, the iSeg-2019 challenge (http://iseg2019.web.unc.edu) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. T raining/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participated in the iSeg-2019. In this article, 8 top-ranked methods were reviewed by detailing their pipelines/implementations, presenting experimental results, and evaluating performance across different sites in terms of whole brain, regions of interest, and gyral landmark curves. We further pointed out their limitations and possible directions for addressing the multi-site issue. We find that multi-site consistency is still an open issue. We hope that the multi-site dataset in the iSeg-2019 and this review article will attract more researchers to address the challenging and critical multi-site issue in practice.

Convolutional Neural Networks (CNNs) have achieved overwhelming success in learning-related problems for 2D/3D images in the Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have an inherent spherical topology in a manifold space, e.g., the convoluted brain cortical surfaces represented by triangular meshes. There is no consistent neighborhood definition and thus no straightforward convolution/pooling operations for such cortical surface data. In this paper, leveraging the regular and hierarchical geometric structure of the resampled spherical cortical surfaces, we create the 1-ring filter on spherical cortical triangular meshes and accordingly develop convolution/pooling operations for constructing Spherical U-Net for cortical surface data. However, the regular nature of the 1-ring filter makes it inherently limited to model fixed geometric transformations. To further enhance the transformation modeling capability of Spherical U-Net, we introduce the deformable convolution and deformable pooling to cortical surface data and accordingly propose the Spherical Deformable U-Net (SDU-Net). Specifically, spherical offsets are learned to freely deform the 1-ring filter on the sphere to adaptively localize cortical structures with different sizes and shapes. We then apply the SDU-Net to two challenging and scientifically important tasks in neuroimaging: cortical surface parcellation and cortical attribute map prediction. Both applications validate the competitive performance of our approach in accuracy and computational efficiency in comparison with state-of-the-art methods.

Cortical surface registration is an essential step and prerequisite for surface-based neuroimaging analysis. It aligns cortical surfaces across individuals and time points to establish cross-sectional and longitudinal cortical correspondences to facilitate neuroimaging studies. Though achieving good performance, available methods are either time consuming or not flexible to extend to multiple or high dimensional features. Considering the explosive availability of large-scale and multimodal brain MRI data, fast surface registration methods that can flexibly handle multimodal features are desired. In this study, we develop a Superfast Spherical Surface Registration (S3Reg) framework for the cerebral cortex. Leveraging an end-to-end unsupervised learning strategy, S3Reg offers great flexibility in the choice of input feature sets and output similarity measures for registration, and meanwhile reduces the registration time significantly. Specifically, we exploit the powerful learning capability of spherical Convolutional Neural Network (CNN) to directly learn the deformation fields in spherical space and implement diffeomorphic design with "scaling and squaring" layers to guarantee topology-preserving deformations. To handle the polar-distortion issue, we construct a novel spherical CNN model using three orthogonal Spherical U-Nets. Experiments are performed on two different datasets to align both adult and infant multimodal cortical features. Results demonstrate that our S3Reg shows superior or comparable performance with state-of-the-art methods, while improving the registration time from 1 min to 10 sec.

In this paper, we introduce an image quality assessment (IQA) method for pediatric T1- and T2-weighted MR images. IQA is first performed slice-wise using a nonlocal residual neural network (NR-Net) and then volume-wise by agglomerating the slice QA results using random forest. Our method requires only a small amount of quality-annotated images for training and is designed to be robust to annotation noise that might occur due to rater errors and the inevitable mix of good and bad slices in an image volume. Using a small set of quality-assessed images, we pre-train NR-Net to annotate each image slice with an initial quality rating (i.e., pass, questionable, fail), which we then refine by semi-supervised learning and iterative self-training. Experimental results demonstrate that our method, trained using only samples of modest size, exhibit great generalizability, capable of real-time (milliseconds per volume) large-scale IQA with nearperfect accuracy.

Prediction of the chronological age based on neuroimaging data is important for brain development analysis and brain disease diagnosis. Although many researches have been conducted for age prediction of older children and adults, little work has been dedicated to infants. To this end, this paper focuses on predicting infant age from birth to 2-year old using brain MR images, as well as identifying some related biomarkers. However, brain development during infancy is too rapid and heterogeneous to be accurately modeled by the conventional regression models. To address this issue, a two-stage prediction method is proposed. Specifically, our method first roughly predicts the age range of an infant and then finely predicts the accurate chronological age based on a learned, age-group-specific regression model. Combining this two-stage prediction method with another complementary one-stage prediction method, a hierarchical rough-to-fine (HRtoF) model is built. HRtoF effectively splits the rapid and heterogeneous changes during a long time period into several short time ranges and further mines the discrimination capability of cortical features, thus reaching high accuracy in infant age prediction. Taking 8 types of cortical morphometric features from structural MRI as predictors, the effectiveness of our proposed HRtoF model is validated using an infant dataset including 50 healthy subjects with 251 longitudinal MRI scans from 14 to 797 days. Comparing with five state-of-the-art regression methods, HRtoF model reduces the mean absolute error of the prediction from >48 days to 32.1 days. The correlation coefficient of the predicted age and the chronological age reaches 0.963. Moreover, based on HRtoF, the relative contributions of the eight types of cortical features for age prediction are also studied.

Effective fusion of structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) data has the potential to boost the accuracy of infant age prediction thanks to the complementary information provided by different imaging modalities. However, functional connectivity measured by fMRI during infancy is largely immature and noisy compared to the morphological features from sMRI, thus making the sMRI and fMRI fusion for infant brain analysis extremely challenging. With the conventional multimodal fusion strategies, adding fMRI data for age prediction has a high risk of introducing more noises than useful features, which would lead to reduced accuracy than that merely using sMRI data. To address this issue, we develop a novel model termed as disentangled-multimodal adversarial autoencoder (DMM-AAE) for infant age prediction based on multimodal brain MRI. Specifically, we disentangle the latent variables of autoencoder into common and specific codes to represent the shared and complementary information among modalities, respectively. Then, cross-reconstruction requirement and common-specific distance ratio loss are designed as regularizations to ensure the effectiveness and thoroughness of the disentanglement. By arranging relatively independent autoencoders to separate the modalities and employing disentanglement under cross-reconstruction requirement to integrate them, our DMM-AAE method effectively restrains the possible interference cross modalities, while realizing effective information fusion. Taking advantage of the latent variable disentanglement, a new strategy is further proposed and embedded into DMM-AAE to address the issue of incompleteness of the multimodal neuroimages, which can also be used as an independent algorithm for missing modality imputation. By taking six types of cortical morphometric features from sMRI and brain functional connectivity from fMRI as predictors, the superiority of the proposed DMM-AAE is validated on infant age (35 to 848 days after birth) prediction using incomplete multimodal neuroimages. The mean absolute error of the prediction based on DMM-AAE reaches 37.6 days, outperforming state-of-the-art methods. Generally, our proposed DMM-AAE can serve as a promising model for prediction with multimodal data.

During the first years of life, the human brain undergoes dynamic spatially-heterogeneous changes, invo- lving differentiation of neuronal types, dendritic arbori- zation, axonal ingrowth, outgrowth and retraction, synaptogenesis, and myelination. To better quantify these changes, this article presents a method for probing tissue microarchitecture by characterizing water diffusion in a spectrum of length scales, factoring out the effects of intra-voxel orientation heterogeneity. Our method is based on the spherical means of the diffusion signal, computed over gradient directions for a set of diffusion weightings (i.e., b -values). We decompose the spherical mean profile at each voxel into a spherical mean spectrum (SMS), which essentially encodes the fractions of spin packets undergoing fine- to coarse-scale diffusion proce- sses, characterizing restricted and hindered diffusion stemming respectively from intra- and extra-cellular water compartments. From the SMS, multiple orientation distribution invariant indices can be computed, allowing for example the quantification of neurite density, microscopic fractional anisotropy ( μ FA), per-axon axial/radial diffusivity, and free/restricted isotropic diffusivity. We show that these indices can be computed for the developing brain for greater sensitivity and specificity to development related changes in tissue microstructure. Also, we demonstrate that our method, called spherical mean spectrum imaging (SMSI), is fast, accurate, and can overcome the biases associated with other state-of-the-art microstructure models.

Fast and automated image quality assessment (IQA) of diffusion MR images is crucial for making timely decisions for rescans. However, learning a model for this task is challenging as the number of annotated data is limited and the annotation labels might not always be correct. As a remedy, we will introduce in this paper an automatic image quality assessment (IQA) method based on hierarchical non-local residual networks for pediatric diffusion MR images. Our IQA is performed in three sequential stages, i.e., 1) slice-wise IQA, where a nonlocal residual network is first pre-trained to annotate each slice with an initial quality rating (i.e., pass/questionable/fail), which is subsequently refined via iterative semi-supervised learning and slice self-training; 2) volume-wise IQA, which agglomerates the features extracted from the slices of a volume, and uses a nonlocal network to annotate the quality rating for each volume via iterative volume self-training; and 3) subject-wise IQA, which ensembles the volumetric IQA results to determine the overall image quality pertaining to a subject. Experimental results demonstrate that our method, trained using only samples of modest size, exhibits great generalizability, and is capable of conducting rapid hierarchical IQA with near-perfect accuracy.

Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.

Diffusion magnetic resonance imaging (DMRI) suffers from lower signal-to-noise-ratio (SNR) due to MR signal attenuation associated with the motion of water molecules. To improve SNR, the non-local means (NLM) algorithm has demonstrated state-of-the-art performance in noise reduction. However, existing NLM algorithms do not take into account explicitly the fact that DMRI signal can vary significantly with local fiber orientations. Applying NLM naïvely can hence blur subtle structures and aggravate partial volume effects. To overcome this limitation, we improve NLM by performing neighborhood matching in non-flat domains and removing noise with information from both x -space (spatial domain) and q -space (wavevector domain). Specifically, we first encode the q -space sampling domain using a graph. We then perform graph framelet transforms to extract robust rotation-invariant features for each sampling point in x-q space. The resulting features are employed for robust neighborhood matching to locate recurrent information. Finally, we remove noise via an NLM framework. To adapt to the various types of noise in multi-coil MR imaging, we transform the signal before denoising so that it is Gaussian-distributed, allowing noise removal to be carried out in an unbiased manner. Our method is able to more effectively locate recurrent information in white matter structures with different orientations, avoiding the blurring effects caused by naïvely applying NLM. Experiments on synthetic, repetitively-acquired, and infant DMRI data demonstrate that our method is able to preserve subtle structures while effectively removing noise.

Acquiring images of the same anatomy with multiple different contrasts increases the diversity of diagnostic information available in an MR exam. Yet, the scan time limitations may prohibit the acquisition of certain contrasts, and some contrasts may be corrupted by noise and artifacts. In such cases, the ability to synthesize unacquired or corrupted contrasts can improve diagnostic utility. For multi-contrast synthesis, the current methods learn a nonlinear intensity transformation between the source and target images, either via nonlinear regression or deterministic neural networks. These methods can, in turn, suffer from the loss of structural details in synthesized images. Here, in this paper, we propose a new approach for multi-contrast MRI synthesis based on conditional generative adversarial networks. The proposed approach preserves intermediate-to-high frequency details via an adversarial loss, and it offers enhanced synthesis performance via pixel-wise and perceptual losses for registered multi-contrast images and a cycle-consistency loss for unregistered images. Information from neighboring cross-sections are utilized to further improve synthesis quality. Demonstrations on T1- and T2- weighted images from healthy subjects and patients clearly indicate the superior performance of the proposed approach compared to the previous state-of-the-art methods. Our synthesis approach can help improve the quality and versatility of the multi-contrast MRI exams without the need for prolonged or repeated examinations.

Missing data is a common problem in longitudinal studies due to subject dropouts and failed scans. We present a graph-based convolutional neural network to predict missing diffusion MRI data. In particular, we consider the relationships between sampling points in the spatial domain and the diffusion wave-vector domain to construct a graph. We then use a graph convolutional network to learn the non-linear mapping from available data to missing data. Our method harnesses a multi-scale residual architecture with adversarial learning for prediction with greater accuracy and perceptual quality. Experimental results show that our method is accurate and robust in the longitudinal prediction of infant brain diffusion MRI data.

Accurate segmentation of infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is an indispensable foundation for early studying of brain growth patterns and morphological changes in neurodevelopmental disorders. Nevertheless, in the isointense phase (approximately 6-9 months of age), due to inherent myelination and maturation process, WM and GM exhibit similar levels of intensity in both T1-weighted (T1w) and T2-weighted (T2w) MR images, making tissue segmentation very challenging. Despite many efforts were devoted to brain segmentation, only few studies have focused on the segmentation of 6-month infant brain images. With the idea of boosting methodological development in the community, iSeg-2017 challenge (http://iseg2017.web.unc.edu) provides a set of 6-month infant subjects with manual labels for training and testing the participating methods. Among the 21 automatic segmentation methods participating in iSeg-2017, we review the 8 top-ranked teams, in terms of Dice ratio, modified Hausdorff distance and average surface distance, and introduce their pipelines, implementations, as well as source codes. We further discuss limitations and possible future directions. We hope the dataset in iSeg-2017 and this review article could provide insights into methodological development for the community.

Positron emission tomography (PET) has been substantially used recently. To minimize the potential health risk caused by the tracer radiation inherent to PET scans, it is of great interest to synthesize the high-quality PET image from the low-dose one to reduce the radiation exposure. In this paper, we propose a 3D auto-context-based locality adaptive multi-modality generative adversarial networks model (LA-GANs) to synthesize the high-quality FDG PET image from the low-dose one with the accompanying MRI images that provide anatomical information. Our work has four contributions. First, different from the traditional methods that treat each image modality as an input channel and apply the same kernel to convolve the whole image, we argue that the contributions of different modalities could vary at different image locations, and therefore a unified kernel for a whole image is not optimal. To address this issue, we propose a locality adaptive strategy for multi-modality fusion. Second, we utilize 1 ×1 ×1 kernel to learn this locality adaptive fusion so that the number of additional parameters incurred by our method is kept minimum. Third, the proposed locality adaptive fusion mechanism is learned jointly with the PET image synthesis in a 3D conditional GANs model, which generates high-quality PET images by employing large-sized image patches and hierarchical features. Fourth, we apply the auto-context strategy to our scheme and propose an auto-context LA-GANs model to further refine the quality of synthesized images. Experimental results show that our method outperforms the traditional multi-modality fusion methods used in deep networks, as well as the state-of-the-art PET estimation approaches.

The early postnatal period witnesses rapid and dynamic brain development. However, the relationship between brain anatomical structure and cognitive ability is still unknown. Currently, there is no explicit model to characterize this relationship in the literature. In this paper, we explore this relationship by investigating the mapping between morphological features of the cerebral cortex and cognitive scores. To this end, we introduce a multi-view multi-task learning approach to intuitively explore complementary information from different time-points and handle the missing data issue in longitudinal studies simultaneously. Accordingly, we establish a novel model, latent partial multi-view representation learning. Our approach regards data from different time-points as different views and constructs a latent representation to capture the complementary information from incomplete time-points. The latent representation explores the complementarity across different time-points and improves the accuracy of prediction. The minimization problem is solved by the alternating direction method of multipliers. Experimental results on both synthetic and real data validate the effectiveness of our proposed algorithm.

Positron emission tomography (PET) images are widely used in many clinical applications, such as tumor detection and brain disorder diagnosis. To obtain PET images of diagnostic quality, a sufficient amount of radioactive tracer has to be injected into a living body, which will inevitably increase the risk of radiation exposure. On the other hand, if the tracer dose is considerably reduced, the quality of the resulting images would be significantly degraded. It is of great interest to estimate a standard-dose PET (S-PET) image from a low-dose one in order to reduce the risk of radiation exposure and preserve image quality. This may be achieved through mapping both S-PET and low-dose PET data into a common space and then performing patch-based sparse representation. However, a one-size-fits-all common space built from all training patches is unlikely to be optimal for each target S-PET patch, which limits the estimation accuracy. In this paper, we propose a data-driven multi-level canonical correlation analysis scheme to solve this problem. In particular, a subset of training data that is most useful in estimating a target S-PET patch is identified in each level, and then used in the next level to update common space and improve estimation. In addition, we also use multi-modal magnetic resonance images to help improve the estimation with complementary information. Validations on phantom and real human brain data sets show that our method effectively estimates S-PET images and well preserves critical clinical quantification measures, such as standard uptake value.

We consider the problem of statistical computations with persistence diagrams, a summary representation of topological features in data. These diagrams encode persistent homology, a widely used invariant in topological data analysis. While several avenues towards a statistical treatment of the diagrams have been explored recently, we follow an alternative route that is motivated by the success of methods based on the embedding of probability measures into reproducing kernel Hilbert spaces. In fact, a positive definite kernel on persistence diagrams has recently been proposed, connecting persistent homology to popular kernel-based learning techniques such as support vector machines. However, important properties of that kernel which would enable a principled use in the context of probability measure embeddings remain to be explored. Our contribution is to close this gap by proving universality of a variant of the original kernel, and to demonstrate its effective use in two-sample hypothesis testing on synthetic as well as real-world data.

Twin imaging studies have been valuable for understanding the relative contribution of the environment and genes on brain structures and their functions. Conventional analyses of twin imaging data include three sequential steps: spatially smoothing imaging data, independently fitting a structural equation model at each voxel, and finally correcting for multiple comparisons. However, conventional analyses are limited due to the same amount of smoothing throughout the whole image, the arbitrary choice of smoothing extent, and the decreased power in detecting environmental and genetic effects introduced by smoothing raw images. The goal of this paper is to develop a two-stage multiscale adaptive regression method (TwinMARM) for spatial and adaptive analysis of twin neuroimaging and behavioral data. The first stage is to establish the relationship between twin imaging data and a set of covariates of interest, such as age and gender. The second stage is to disentangle the environmental and genetic influences on brain structures and their functions. In each stage, TwinMARM employs hierarchically nested spheres with increasing radii at each location and then captures spatial dependence among imaging observations via consecutively connected spheres across all voxels. Simulation studies show that our TwinMARM significantly outperforms conventional analyses of twin imaging data. Finally, we use our method to detect statistically significant effects of genetic and environmental variations on white matter structures in a neonatal twin study.

White matter fiber tractography plays a key role in the in vivo understanding of brain circuitry. For tract-based comparison of a population of images, a common approach is to first generate an atlas by averaging, after spatial normalization, all images in the population, and then perform tractography using the constructed atlas. The reconstructed fiber trajectories form a common geometry onto which diffusion properties of each individual subject can be projected based on the corresponding locations in the subject native space. However, in the case of high angular resolution diffusion imaging (HARDI), where modeling fiber crossings is an important goal, the above-mentioned averaging method for generating an atlas results in significant error in the estimation of local fiber orientations and causes a major loss of fiber crossings. These limitatitons have significant impact on the accuracy of the reconstructed fiber trajectories and jeopardize subsequent tract-based analysis. As a remedy, we present in this paper a more effective means of performing tractography at a population level. Our method entails determining a bipolar Watson distribution at each voxel location based on information given by all images in the population, giving us not only the local principal orientations of the fiber pathways, but also confidence levels of how reliable these orientations are across subjects. The distribution field is then fed as an input to a probabilistic tractography framework for reconstructing a set of fiber trajectories that are consistent across all images in the population. We observe that the proposed method, called PopTract, results in significantly better preservation of fiber crossings, and hence yields better trajectory reconstruction in the atlas space.

We propose a novel diffusion tensor imaging (DTI) registration algorithm, called fast tensor image morphing for elastic registration (F-TIMER). F-TIMER leverages multiscale tensor regional distributions and local boundaries for hierarchically driving deformable matching of tensor image volumes. Registration is achieved by utilizing a set of automatically determined structural landmarks, via solving a soft correspondence problem. Based on the estimated correspondences, thin-plate splines are employed to generate a smooth, topology preserving, and dense transformation, and to avoid arbitrary mapping of nonlandmark voxels. To mitigate the problem of local minima, which is common in the estimation of high dimensional transformations, we employ a hierarchical strategy where a small subset of voxels with more distinctive attribute vectors are first deployed as landmarks to estimate a relatively robust low-degrees-of-freedom transformation. As the registration progresses, an increasing number of voxels are permitted to participate in refining the correspondence matching. A scheme as such allows less conservative progression of the correspondence matching towards the optimal solution, and hence results in a faster matching speed. Compared with its predecessor TIMER, which has been shown to outperform state-of-the-art algorithms, experimental results indicate that F-TIMER is capable of achieving comparable accuracy at only a fraction of the computation cost.

Diffusion tensor imaging (DTI) provides important information on the structure of white matter fiber bundles as well as detailed tissue properties along these fiber bundles in vivo. This paper presents a functional regression framework, called FRATS, for the analysis of multiple diffusion properties along fiber bundle as functions in an infinite dimensional space and their association with a set of covariates of interest, such as age, diagnostic status and gender, in real applications. The functional regression framework consists of four integrated components: the local polynomial kernel method for smoothing multiple diffusion properties along individual fiber bundles, a functional linear model for characterizing the association between fiber bundle diffusion properties and a set of covariates, a global test statistic for testing hypotheses of interest, and a resampling method for approximating the p-value of the global test statistic. The proposed methodology is applied to characterizing the development of five diffusion properties including fractional anisotropy, mean diffusivity, and the three eigenvalues of diffusion tensor along the splenium of the corpus callosum tract and the right internal capsule tract in a clinical study of neurodevelopment. Significant age and gestational age effects on the five diffusion properties were found in both tracts. The resulting analysis pipeline can be used for understanding normal brain development, the neural bases of neuropsychiatric disorders, and the joint effects of environmental and genetic factors on white matter fiber bundles.

Diffusion tensors are estimated from magnetic resonance images (MRIs) that are diffusion-weighted, and those images inherently contain noise. Therefore, noise in the diffusion-weighted images produces uncertainty in estimation of the tensors and their derived parameters, which include eigenvalues, eigenvectors, and the trajectories of fiber pathways that are reconstructed from those eigenvalues and eigenvectors. Although repetition and wild bootstrap methods have been widely used to quantify the uncertainty of diffusion tensors and their derived parameters, we currently lack theoretical derivations that would validate the use of these two bootstrap methods for the estimation of statistical parameters of tensors in the presence of noise. The aim of this paper is to examine theoretically and numerically the repetition and wild bootstrap methods for approximating uncertainty in estimation of diffusion tensor parameters under two different schemes for acquiring diffusion weighted images. Whether these bootstrap methods can be used to quantify uncertainty in some diffusion tensor parameters, such as fractional anisotropy (FA), depends critically on the morphology of the diffusion tensor that is being estimated. The wild and repetition bootstrap methods in particular cannot quantify uncertainty in the principal direction (PD) of isotropic (or oblate) tensor. We also examine the use of bootstrap methods in estimating tensors in a voxel containing multiple tensors, demonstrating their limitations when quantifying the uncertainty of tensor parameters in those locations. Simulation studies are also used to understand more thoroughly our theoretical results. Our findings raise serious concerns about the use of bootstrap methods to quantify the uncertainty of fiber pathways when those pathways pass through voxels that contain either isotropic tensors, oblate tensors, or multiple tensors.

In dynamic susceptibility contrast perfusion weighted imaging, the recirculation effect is normally removed by gamma-variate fitting from concentration curves before estimating hemodynamic parameters. At lower SNR, however, many fitting failures may result. Moreover, when cerebral hemodynamics is compromised e.g., cerebral ischemia, a substantially broadened concentration curve is anticipated, resulting in the first passage overlapping with recirculation, which again causes a gamma-fit to fail to consistently discern recirculation contributions from the first passage. We propose to exploit independent component analysis to obviate the recirculation effect. We demonstrate that such a technique can remove recirculation in normal and ischemic brain tissues while preserving the first passage. This in turn allows for accurate recirculation elimination and hence improved estimation of cerebral blood volume particularly when overlapping between first passage and recirculation is suspected as in the case of an ischemic lesion.

The clinical recognition of abnormal vascular tortuosity, or excessive bending, twisting, and winding, is important to the diagnosis of many diseases. Automated detection and quantitation of abnormal vascular tortuosity from three-dimensional (3-D) medical image data would, therefore, be of value. However, previous research has centered primarily upon two-dimensional (2-D) analysis of the special subset of vessels whose paths are normally close to straight. This report provides the first 3-D tortuosity analysis of clusters of vessels within the normally tortuous intracerebral circulation. We define three different clinical patterns of abnormal tortuosity. We extend into 3-D two tortuosity metrics previously reported as useful in analyzing 2-D images and describe a new metric that incorporates counts of minima of total curvature. We extract vessels from MRA data, map corresponding anatomical regions between sets of normal patients and patients with known pathology, and evaluate the three tortuosity metrics for ability to detect each type of abnormality within the region of interest. We conclude that the new tortuosity metric appears to be the most effective in detecting several types of abnormalities. However, one of the other metrics, based on a sum of curvature magnitudes, may be more effective in recognizing tightly coiled, "corkscrew" vessels associated with malignant tumors.