Time-series data such as fMRI and MEG carry a wealth of inherent spatio-temporal coupling relationship, and their modeling via deep learning is essential for uncovering biological mechanisms. However, current machine learning models for mining spatio-temporal information usually overlook this intrinsic coupling association, in addition to poor explainability. In this paper, we present an explainable learning framework for spatio-temporal coupling. Specifically, this framework constructs a deep learning network based on spatio-temporal correlation, which can well integrate the time-varying coupled relationships between node representation and inter-node connectivity. Furthermore, it explores spatio-temporal evolution at each time step, providing a better explainability of the analysis results. Finally, we apply the proposed framework to brain dynamic functional connectivity (dFC) analysis. Experimental results demonstrate that it can effectively capture the variations in dFC during brain development and the evolution of spatio-temporal information at the resting state. Two distinct developmental functional connectivity (FC) patterns are identified. Specifically, the connectivity among regions related to emotional regulation decreases, while the connectivity associated with cognitive activities increases. In addition, children and young adults display notable cyclic fluctuations in resting-state brain dFC.

Graph convolutional deep learning has emerged as a promising method to explore the functional organization of the human brain in neuroscience research. This paper presents a novel framework that utilizes the gated graph transformer (GGT) model to predict individuals' cognitive ability based on functional connectivity (FC) derived from fMRI. Our framework incorporates prior spatial knowledge and uses a random-walk diffusion strategy that captures the intricate structural and functional relationships between different brain regions. Specifically, our approach employs learnable structural and positional encodings (LSPE) in conjunction with a gating mechanism to efficiently disentangle the learning of positional encoding (PE) and graph embeddings. Additionally, we utilize the attention mechanism to derive multi-view node feature embeddings and dynamically distribute propagation weights between each node and its neighbors, which facilitates the identification of significant biomarkers from functional brain networks and thus enhances the interpretability of the findings. To evaluate our proposed model in cognitive ability prediction, we conduct experiments on two large-scale brain imaging datasets: the Philadelphia Neurodevelopmental Cohort (PNC) and the Human Connectome Project (HCP). The results show that our approach not only outperforms existing methods in prediction accuracy but also provides superior explainability, which can be used to identify important FCs underlying cognitive behaviors.

The reliability of functional network connectivity (FNC) measured using independent component analysis (ICA) has frequently been explored within the literature, with results displaying varying levels of reliability and demonstrating that minor changes in data preprocessing procedures can significantly alter FC results and reliability. However, one important avenue of research that has not been explored within the current literature is the effect of spatial normalization techniques on FNC reliability. Spatially constrained independent component analysis techniques such as multi-objective optimization with reference (MOO-ICAR) is one of many methods used to study brain functional connectivity (FC) using fMRI that is theoretically robust to variations which may arise in data as a result of normalization procedures. In this work, we deploy MOO-ICAR across 30 different spatial normalization pipelines varying across participant template, normalization modality (anatomical vs functional), and one vs. two-stage warps to MNI space. Most components display relatively high consistency intraclass-correlation coefficients (ICCs), with the vast majoritv (~80%) ereater than 0.5.

Functional magnetic resonance imaging (fMRI) is a powerful technique with the potential to estimate individual variations in behavioral and cognitive traits. Joint learning of multiple datasets can utilize their complementary information so as to improve learning performance, but it also gives rise to the challenge for data fusion to effectively integrate brain patterns elicited by multiple fMRI data. However, most of the current data fusion methods analyze each single dataset separately and further infer the relationship among them, which fail to utilize the multidimensional structure inherent across modalities and may ignore complex but important interactions. To address this issue, we propose a novel sparse tensor decomposition method to integrate multiple task-stimulus (paradigm) fMRI data. Seeing each paradigm fMRI as one modality, our proposed method considers the relationships across subjects and modalities simultaneously. In specific, a third-order tensor is first modeled by using the functional network connectivity (FNC) of subjects in multiple fMRI paradigms. A novel sparse tensor decomposition with the regularization terms is designed to factorize the tensor into a series of rank-one components, which can extract the shared components across modalities as the embedded features. The L2,1-norm regularizer (i.e., group sparsity) is enforced to select a few common features among multiple subjects. Validation of the proposed method is performed on realistic three paradigm fMRI datasets from the Philadelphia Neurodevelopmental Cohort (PNC) study, for the study of the relationship between the FNC and human cognitive abilities. Experimental results show our method outperforms several other competing methods in the prediction of individuals with different cognitive behaviors via the wide range achievement test (WRAT). Furthermore, our method discovers the FNC related to the cognitive behaviors, such as the connectivity associated with the default mode network (DMN) for three paradigms, and the connectivity between DMN and visual (VIS) domains within the emotion task.

The combination of multimodal imaging and genomics provides a more comprehensive way for the study of mental illnesses and brain functions. Deep network-based data fusion models have been developed to capture their complex associations, resulting in improved diagnosis of diseases. However, deep learning models are often difficult to interpret, bringing about challenges for uncovering biological mechanisms using these models. In this work, we develop an interpretable multimodal fusion model to perform automated diagnosis and result interpretation simultaneously. We name it Grad-CAM guided convolutional collaborative learning (gCAM-CCL), which is achieved by combining intermediate feature maps with gradient-based weights. The gCAM-CCL model can generate interpretable activation maps to quantify pixel-level contributions of the input features. Moreover, the estimated activation maps are class-specific, which can therefore facilitate the identification of biomarkers underlying different groups. We validate the gCAM-CCL model on a brain imaging-genetic study, and demonstrate its applications to both the classification of cognitive function groups and the discovery of underlying biological mechanisms. Specifically, our analysis results suggest that during task-fMRI scans, several object recognition related regions of interests (ROIs) are activated followed by several downstream encoding ROIs. In addition, the high cognitive group may have stronger neurotransmission signaling while the low cognitive group may have problems in brain/neuron development due to genetic variations.

Human brain development is a complex and dynamic process caused by several factors such as genetics, sex hormones, and environmental changes. A number of recent studies on brain development have examined functional connectivity (FC) defined by the temporal correlation between time series of different brain regions. We propose to add the directional flow of information during brain maturation. To do so, we extract effective connectivity (EC) through Granger causality (GC) for two different groups of subjects, i.e., children and young adults. The motivation is that the inclusion of causal interaction may further discriminate brain connections between two age groups and help to discover new connections between brain regions. The contributions of this study are threefold. First, there has been a lack of attention to EC-based feature extraction in the context of brain development. To this end, we propose a new kernel-based GC (KGC) method to learn nonlinearity of complex brain network, where a reduced Sine hyperbolic polynomial (RSP) neural network was used as our proposed learner. Second, we used causality values as the weight for the directional connectivity between brain regions. Our findings indicated that the strength of connections was significantly higher in young adults relative to children. In addition, our new EC-based feature outperformed FC-based analysis from Philadelphia neurocohort (PNC) study with better discrimination of different age groups. Moreover, the fusion of these two sets of features (FC + EC) improved brain age prediction accuracy by more than 4%, indicating that they should be used together for brain development studies.

Recently, a hypergraph constructed from functional magnetic resonance imaging (fMRI) was utilized to explore brain functional connectivity networks (FCNs) for the classification of neurodegenerative diseases. Each edge of a hypergraph (called hyperedge) can connect any number of brain regions-of-interest (ROIs) instead of only two ROIs, and thus characterizes high-order relations among multiple ROIs that cannot be uncovered by a simple graph in the traditional graph based FCN construction methods. Unlike the existing hypergraph based methods where all hyperedges are assumed to have equal weights and only certain topological features are extracted from the hypergraphs, we propose a hypergraph learning based method for FCN construction in this paper. Specifically, we first generate hyperedges from fMRI time series based on sparse representation, then employ hypergraph learning to adaptively learn hyperedge weights, and finally define a hypergraph similarity matrix to represent the FCN. In our proposed method, weighting hyperedges results in better discriminative FCNs across subjects, and the defined hypergraph similarity matrix can better reveal the overall structure of brain network than using those hypergraph topological features. Moreover, we propose a multi-hypergraph learning based method by integrating multi-paradigm fMRI data, where the hyperedge weights associated with each fMRI paradigm are jointly learned and then a unified hypergraph similarity matrix is computed to represent the FCN. We validate the effectiveness of the proposed method on the Philadelphia Neurodevelopmental Cohort dataset for the classification of individuals' learning ability from three paradigms of fMRI data. Experimental results demonstrate that our proposed approach outperforms the traditional graph based methods (i.e., Pearson's correlation and partial correlation with the graphical Lasso) and the existing unweighted hypergraph based methods, which sheds light on how to optimize estimation of FCNs for cognitive and behavioral study.

Adolescence is a transitional period between the childhood and adulthood with physical changes, as well as increasing emotional development. Studies have shown that the emotional sensitivity is related to a second period of rapid brain growth. However, there is little focus on the trend of brain development during this period. In this paper, we aim to track functional brain connectivity development from late childhood to young adulthood. Mathematically, this problem can be modeled via the estimation of multiple Gaussian graphical models (GGMs). However, most existing methods either require the graph sequence to be fairly long or are only applicable to small graphs. In this paper, we adapted a Bayesian approach incorporating joint estimation of multiple GGMs to overcome the short sequence difficulty, which is also computationally efficient. The data used are the functional magnetic resonance imaging (fMRI) images obtained from the publicly available Philadelphia Neurodevelopmental Cohort (PNC). They include 855 individuals aged 8-22 years who were divided into five different adolescent stages. We summarized the networks with global measurements and applied a hypothesis test across age groups to detect the developmental patterns. Three patterns were detected and defined as consistent development, late puberty, and temporal change. We also discovered several anatomical areas, such as the middle frontal gyrus, putamen gyrus, right lingual gyrus, and right cerebellum crus 2 that are highly involved in the brain functional development. The functional networks, including the salience, subcortical, and auditory networks are significantly developing during the adolescent period.

Estimating dynamic functional network connectivity (dFNC) of the brain from functional magnetic resonance imaging (fMRI) data can reveal both spatial and temporal organization and can be applied to track the developmental trajectory of brain maturity as well as to study mental illness. Resting state fMRI (rs-fMRI) is regarded as a promising task since it reflects the spontaneous brain activity without an external stimulus. The sliding window method has been successfully used to extract dFNC but typically assumes a fixed window size. The hidden Markov model (HMM) based method is an alternative approach for estimating time-varying connectivity. In this paper, we propose a sparse HMM based on Gaussian HMM and Gaussian graphical model (GGM). In this model, the time-varying neural processes are represented as discrete brain states which are described with functional connectivity networks. By enforcing the sparsity on the precision matrix, we can get interpretable connectivity between different functional regions. The optimization of our model can be realized with the expectation maximization (EM) and graphical least absolute shrinkage and selection operator (glasso) algorithms. The proposed model is validated on both simulated blood oxygenation-level dependent (BOLD) time series and rs-fMRI data. Results indicate that the proposed model can capture both stationary and abrupt brain activity fluctuations. We also compare dFNC patterns between children and young adults from the Philadelphia Neurodevelopmental Cohort (PNC) study. Both spatial and temporal behavior of the dFNC are analyzed and compared. The results provide insight into the developmental trajectory across childhood and motivate further research on brain connectivity.

Functional connectivity (FC) estimated from functional magnetic resonance imaging (fMRI) time series, especially during resting state periods, provides a powerful tool to assess human brain functional architecture in health, disease, and developmental states. Recently, the focus of connectivity analysis has shifted toward the subnetworks of the brain, which reveals co-activating patterns over time. Most prior works produced a dense set of high-dimensional vectors, which are hard to interpret. In addition, their estimations to a large extent were based on an implicit assumption of spatial and temporal stationarity throughout the fMRI scanning session. In this paper, we propose an approach called dynamic sparse connectivity patterns (dSCPs), which takes advantage of both matrix factorization and time-varying fMRI time series to improve the estimation power of FC. The feasibility of analyzing dynamic FC with our model is first validated through simulated experiments. Then, we use our framework to measure the difference between young adults and children with real fMRI data set from the Philadelphia Neurodevelopmental Cohort (PNC). The results from the PNC data set showed significant FC differences between young adults and children in four different states. For instance, young adults had reduced connectivity between the default mode network and other subnetworks, as well as hyperconnectivity within the visual system in states 1 and 3, and hypoconnectivity in state 2. Meanwhile, they exhibited temporal correlation patterns that changed over time within functional subnetworks. In addition, the dSCPs model indicated that older people tend to spend more time within a relatively connected FC pattern. Overall, the proposed method provides a valid means to assess dynamic FC, which could facilitate the study of brain networks.

In this paper, we consider the problem of estimating multiple sparse, co-activated brain regions from functional magnetic resonance imaging (fMRI) observations belonging to different classes. More precisely, we propose a method to analyze similarities and differences in functional connectivity between children and young adults. Often, analysis is conducted on each class separately, and differences across classes are identified with an additional postprocessing step using adequate statistical tools. Here, we propose to rely on a generalized fused Lasso penalty, which allows us to make use of the entire data set in order to estimate connectivity patterns that are either shared across classes, or specific to a given group. By using the entire population during the estimation, we hope to increase the power of our analysis. The proposed model falls in the category of population-wise matrix decomposition, and a simple and efficient alternating direction method of multipliers algorithm is introduced to solve the associated optimization problem. After validating our approach on simulated data, experiments are performed on resting-state fMRI imaging from the Philadelphia neurodevelopmental cohort data set, comprised of normally developing children from ages 8 to 21. Developmental differences were observed in various brain regions, as a total of three class-specific resting-state components were identified. Statistical analysis of the estimated subject-specific features, as well as classification results (based on age groups, up to 81% accuracy, samples) related to these components demonstrate that the proposed method is able to properly extract meaningful shared and class-specific sub-networks.

Functional magnetic resonance imaging (fMRI) is a powerful tool to analyze brain development and neuronal activity. Identifying discriminative brain regions between various groups within a population has generated great interest in recent years. In this work, we consider the problem of estimating multiple sparse, co-activated brain regions from fMRI observations belonging to different classes. More precisely, we propose a method to analyze functional connectivity differences between children and young adults. Often, analysis is conducted on each class separately. Here, we propose to rely on a generalized fused Lasso penalty to extract both class-specific and shared co-expressed regions. In order to validate our method, experiments are performed on an fMRI dataset comprised of normally developing children from 8 to 21. The results demonstrate that the proposed method is able to properly extract meaningful sub-networks, which results in improved classification accuracy between the two classes.