Ample studies across various neuroimaging modalities have suggested that the human cortex at rest is hierarchically organized along the spectral and functional axes. However, the relationship between the spectral and functional organizations of the human cortex remains largely unexplored. Here, we reveal the confluence of functional and spectral cortical organizations by examining the functional specialization in spectral gradients of the cortex. These spectral gradients, derived from functional magnetic resonance imaging data at rest using our temporal de-correlation method to enhance spectral resolution, demonstrate regional frequency biases. The grading of spectral gradients across the cortex - aligns with many existing brain maps - is found to be highly functionally specialized through discovered frequency-specific resting-state functional networks, functionally distinctive spectral profiles, and an intrinsic coordinate system that is functionally specialized. By demonstrating the functionally specialized spectral gradients of the cortex, we shed light on the close relation between functional and spectral organizations of the resting human cortex.

Objective classification biomarkers that are developed using resting-state functional magnetic resonance imaging (rs-fMRI) data are expected to contribute to more effective treatment for psychiatric disorders. Unfortunately, no widely accepted biomarkers are available at present, partially because of the large variety of analysis pipelines for their development. In this study, we comprehensively evaluated analysis pipelines using a large-scale, multi-site fMRI dataset for major depressive disorder (MDD). We explored combinations of options in four sub-processes of the analysis pipelines: six types of brain parcellation, four types of functional connectivity (FC) estimations, three types of site-difference harmonization, and five types of machine-learning methods. A total of 360 different MDD classification biomarkers were constructed using the SRPBS dataset acquired with unified protocols (713 participants from four sites) as the discovery dataset, and datasets from other projects acquired with heterogeneous protocols (449 participants from four sites) were used for independent validation. We repeated the procedure after swapping the roles of the two datasets to identify superior pipelines, regardless of the discovery dataset. The classification results of the top 10 biomarkers showed high similarity, and weight similarity was observed between eight of the biomarkers, except for two that used both data-driven parcellation and FC computation. We applied the top 10 pipelines to the datasets of other psychiatric disorders (autism spectrum disorder and schizophrenia), and eight of the biomarkers exhibited sufficient classification performance for both disorders. Our results will be useful for establishing a standardized pipeline for classification biomarkers.

The recent success of sequential learning models, such as deep recurrent neural networks, is largely due to their superior representation-learning capability for learning the informative representation of a targeted time series. The learning of these representations is generally goal-directed, resulting in their task-specific nature, giving rise to excellent performance in completing a single downstream task but hindering between-task generalisation. Meanwhile, with increasingly intricate sequential learning models, learned representation becomes abstract to human knowledge and comprehension. Hence, we propose a unified local predictive model based on the multi-task learning paradigm to learn the task-agnostic and interpretable subsequence-based time series representation, allowing versatile use of learned representations in temporal prediction, smoothing, and classification tasks. The targeted interpretable representation could convey the spectral information of the modelled time series to the level of human comprehension. Through a proof-of-concept evaluation study, we demonstrate the empirical superiority of learned task-agnostic and interpretable representation over task-specific and conventional subsequence-based representation, such as symbolic and recurrent learning-based representation, in solving temporal prediction, smoothing, and classification tasks. These learned task-agnostic representations can also reveal the ground-truth periodicity of the modelled time series. We further propose two applications of our unified local predictive model in functional magnetic resonance imaging (fMRI) analysis to reveal the spectral characterisation of cortical areas at rest and reconstruct more smoothed temporal dynamics of cortical activations in both resting-state and task-evoked fMRI data, giving rise to robust decoding.

In neuroscience, the functional magnetic resonance imaging (fMRI) is a vital tool to non-invasively access brain activity. Using fMRI, the functional connectivity (FC) between brain regions can be inferred, which has contributed to a number of findings of the fundamental properties of the brain. As an important clinical application of FC, clustering of subjects based on FC recently draws much attention, which can potentially reveal important heterogeneity in subjects such as subtypes of psychiatric disorders. In particular, a multiple clustering method is a powerful analytical tool, which identifies clustering patterns of subjects depending on their FC in specific brain areas. However, when one applies an existing multiple clustering method to fMRI data, there is a need to simplify the data structure, independently dealing with elements in a FC matrix, i.e., vectorizing a correlation matrix. Such a simplification may distort the clustering results. To overcome this problem, we propose a novel multiple clustering method based on Wishart mixture models, which preserves the correlation matrix structure without vectorization. The uniqueness of this method is that the multiple clustering of subjects is based on particular networks of nodes (or regions of interest, ROIs), optimized in a data-driven manner. Hence, it can identify multiple underlying pairs of associations between a subject cluster solution and a ROI sub-network. The key assumption of the method is independence among sub-networks, which is effectively addressed by whitening correlation matrices. We applied the proposed method to synthetic and fMRI data, demonstrating the usefulness and power of the proposed method.

To uncover the rich temporal dynamics from the noisy fMRI time series, we propose a data-driven time series model: temporal reconstruction model, targets on reconstructing subsequences of fMRI time series via encoded contextual representations. This novel reconstruction model is further perfected by learning the commonality among multiple reconstructed temporal resolutions. Through an empirical validation on a synthetic noisy time series, we demonstrate the superior denoising capacity of our reconstruction model. Implementing this reconstruction model on a real fMRI dataset, it reveals the rich temporal dynamics of reconstructed fMRI time series are revealed, assisting the subsequent neuroscientific analysis on discovering more consistent subject-level temporal independent functional modes.

This article presents a review of computational methods for connectivity inference from neural activity data derived from multi-electrode recordings or fluorescence imaging. We first identify biophysical and technical challenges in connectivity inference along the data processing pipeline. We then review connectivity inference methods based on two major mathematical foundations, namely, descriptive model-free approaches and generative model-based approaches. We investigate representative studies in both categories and clarify which challenges have been addressed by which method. We further identify critical open issues and possible research directions.

BACKGROUND: Advance in high-throughput technologies in genomics, transcriptomics, and metabolomics has created demand for bioinformatics tools to integrate high-dimensional data from different sources. Canonical correlation analysis (CCA) is a statistical tool for finding linear associations between different types of information. Previous extensions of CCA used to capture nonlinear associations, such as kernel CCA, did not allow feature selection or capturing of multiple canonical components. Here we propose a novel method, two-stage kernel CCA (TSKCCA) to select appropriate kernels in the framework of multiple kernel learning. RESULTS: TSKCCA first selects relevant kernels based on the HSIC criterion in the multiple kernel learning framework. Weights are then derived by non-negative matrix decomposition with L1 regularization. Using artificial datasets and nutrigenomic datasets, we show that TSKCCA can extract multiple, nonlinear associations among high-dimensional data and multiplicative interactions among variables. CONCLUSIONS: TSKCCA can identify nonlinear associations among high-dimensional data more reliably than previous nonlinear CCA methods.

In this paper, we present a reinforcement learning model of the shepherding of a flock of sheep by a dog. The shepherding task, a heuristic model originally proposed by Strombom, et al., describes the dynamics of the sheep while being herded by a dog to a predefined target. This study recreates the proposed model using SARSA, an algorithm for learning the optimal policy in reinforcement learning. Results show that with a discretized state and action space, the dog is able to successfully herd a flock of a sheep to the target position by first learning to reach a subgoal. A reward is awarded when the dog reaches the neighbourhood of a subgoal, while a penalty is incurred for each time the shepherding task is not completed. The stochasticity of the interaction among sheep and dog, including the existence of multiple subgoals affect the learning time of the agent. Finally, we present an example of the learned shepherding task which shows the agent's continuous success after the 350th episode.

Recent studies have revealed that resting state functional connectivity is associated with major depressive disorder (MDD). However, the relationship between functional connectivity and clinical measures for the detailed assessment of depression remains unclear. The objective of our study is thus to associate functional connectivity of depressed patients and healthy controls with their individual clinical measures, using a statistical method called partial least squares analysis (PLS). We demonstrated that this method could predict certain clinical measures based on a limited number of functional connections and provided benefits to the prediction performance through incorporation of the subject's age and the estimation of multiple measures simultaneously. Generalizability of the prediction model was assured through leave one out cross validation. The results showed that for BDI-II and SHAPS the most contributing connections concerned cuneus, precuneus and middle frontal cortex and areas of the cerebellum. While the relationship was similar for PANAS(n), it showed its strongest relation with functional connection between calcarine and insula.

Most conventional policy gradient reinforcement learning (PGRL) algorithms neglect (or do not explicitly make use of) a term in the average reward gradient with respect to the policy parameter. That term involves the derivative of the stationary state distribution that corresponds to the sensitivity of its distribution to changes in the policy parameter. Although the bias introduced by this omission can be reduced by setting the forgetting rate gamma for the value functions close to 1, these algorithms do not permit gamma to be set exactly at gamma = 1. In this article, we propose a method for estimating the log stationary state distribution derivative (LSD) as a useful form of the derivative of the stationary state distribution through backward Markov chain formulation and a temporal difference learning framework. A new policy gradient (PG) framework with an LSD is also proposed, in which the average reward gradient can be estimated by setting gamma = 0, so it becomes unnecessary to learn the value functions. We also test the performance of the proposed algorithms using simple benchmark tasks and show that these can improve the performances of existing PG methods.

Corticostriatal synapse plasticity of medium spiny neurons is regulated by glutamate input from the cortex and dopamine input from the substantia nigra. While cortical stimulation alone results in long-term depression (LTD), the combination with dopamine switches LTD to long-term potentiation (LTP), which is known as dopamine-dependent plasticity. LTP is also induced by cortical stimulation in magnesium-free solution, which leads to massive calcium influx through NMDA-type receptors and is regarded as calcium-dependent plasticity. Signaling cascades in the corticostriatal spines are currently under investigation. However, because of the existence of multiple excitatory and inhibitory pathways with loops, the mechanisms regulating the two types of plasticity remain poorly understood. A signaling pathway model of spines that express D1-type dopamine receptors was constructed to analyze the dynamic mechanisms of dopamine- and calcium-dependent plasticity. The model incorporated all major signaling molecules, including dopamine- and cyclic AMP-regulated phosphoprotein with a molecular weight of 32 kDa (DARPP32), as well as AMPA receptor trafficking in the post-synaptic membrane. Simulations with dopamine and calcium inputs reproduced dopamine- and calcium-dependent plasticity. Further in silico experiments revealed that the positive feedback loop consisted of protein kinase A (PKA), protein phosphatase 2A (PP2A), and the phosphorylation site at threonine 75 of DARPP-32 (Thr75) served as the major switch for inducing LTD and LTP. Calcium input modulated this loop through the PP2B (phosphatase 2B)-CK1 (casein kinase 1)-Cdk5 (cyclin-dependent kinase 5)-Thr75 pathway and PP2A, whereas calcium and dopamine input activated the loop via PKA activation by cyclic AMP (cAMP). The positive feedback loop displayed robust bi-stable responses following changes in the reaction parameters. Increased basal dopamine levels disrupted this dopamine-dependent plasticity. The present model elucidated the mechanisms involved in bidirectional regulation of corticostriatal synapses and will allow for further exploration into causes and therapies for dysfunctions such as drug addiction.

Policy gradient Reinforcement Learning (RL) algorithms have received much attention in seeking stochastic policies that maximize the average rewards. In addition, extensions based on the concept of the Natural Gradient (NG) show promising learning efficiency because these regard metrics for the task. Though there are two candidate metrics, Kakades Fisher Information Matrix (FIM) and Morimuras FIM, all RL algorithms with NG have followed the Kakades approach. In this paper, we describe a generalized Natural Gradient (gNG) by linearly interpolating the two FIMs and propose an efficient implementation for the gNG learning based on a theory of the estimating function, generalized Natural Actor-Critic (gNAC). The gNAC algorithm involves a near optimal auxiliary function to reduce the variance of the gNG estimates. Interestingly, the gNAC can be regarded as a natural extension of the current state-of-the-art NAC algorithm, as long as the interpolating parameter is appropriately selected. Numerical experiments showed that the proposed gNAC algorithm can estimate gNG efficiently and outperformed the NAC algorithm.

We propose a novel approach to acquire the optimal policy for a continuous Markov decision process. Based on an analogy from statistical mechanics, we introduce a variational free energy over a policy. A good policy can be obtained by minimizing the variational free energy. According to our approach, the optimal policy in linear quadratic regulator problems can be obtained by using Kalman filtering and smoothing techniques. Even in non-linear problems, a semi-optimal policy can be obtained by Monte Carlo technique with a Gaussian process method.

In this article, we propose a new reinforcement learning (RL) method for a system having continuous state and action spaces. Our RL method has an architecture like the actor-critic model. The critic tries to approximate the Q-function, which is the expected future return for the current state-action pair. The actor tries to approximate a stochastic soft-max policy defined by the Q-function. The soft-max policy is more likely to select an action that has a higher Q-function value. The online EM algorithm is used to train the critic and the actor. We apply this method to two control problems. Computer simulations show that our method is able to acquire fairly good control in the two tasks after a few learning trials.