While the human brain efficiently handles various computations with a limited number of neurons, traditional deep learning networks require a significant increase in parameters to improve performance. Yet, these parameters are used inefficiently as the networks employ the same amount of computation for inputs of the same size, regardless of the input's complexity. We address this inefficiency by introducing self-introspection capabilities to the network, enabling it to adjust the number of used parameters based on the internal representation of the task and adapt the computation time based on the task complexity. This enables the network to adaptively reuse parameters across tasks, dynamically adjusting the computational effort to match the complexity of the input. We demonstrate the effectiveness of this method on language modeling and computer vision tasks. Notably, our model achieves 96.62\% accuracy on ImageNet with just a three-layer network, surpassing much larger ResNet-50 and EfficientNet. When applied to a transformer architecture, the approach achieves 95.8\%/88.7\% F1 scores on the SQuAD v1.1/v2.0 datasets at negligible parameter cost. These results showcase the potential for dynamic and reflective computation, contributing to the creation of intelligent systems that efficiently manage resources based on input data complexity.

The limited availability of diverse, high-quality datasets is a significant challenge in applying deep learning to neuroimaging research. Although synthetic data generation can potentially address this issue, on-the-fly generation is computationally demanding, while training on pre-generated data is inflexible and may incur high storage costs. We introduce Wirehead, a scalable in-memory data pipeline that significantly improves the performance of on-the-fly synthetic data generation for deep learning in neuroimaging. Wirehead's architecture decouples data generation from training by running multiple generators in independent parallel processes, facilitating near-linear performance gains proportional to the number of generators used. It efficiently handles terabytes of data using MongoDB, greatly minimizing prohibitive storage costs. The robust, modular design enables flexible pipeline configurations and fault-tolerant operation. We evaluated Wirehead with SynthSeg, a synthetic brain segmentation data generation tool that requires 7 days to train a model. When deployed in parallel, Wirehead achieved a near-linear 15.7x increase in throughput with 16 generators. With 20 generators, we can train a model in 9 hours instead of 7 days. This demonstrates Wirehead's ability to greatly accelerate experimentation cycles. While Wirehead represents a substantial step forward, it also reveals opportunities for future research in optimizing generation-training balance and resource allocation. Its ability to facilitate distributed deep learning has significant implications for enabling more ambitious neuroimaging research.

Federated analysis can help perform large-scale analyses using neuroimaging datasets across various research groups overcoming the limitations of institutional data-sharing policies, privacy or regulatory concerns as it requires no data sharing. In this work, we employ a federated neuromark algorithm to generate independent component analysis (ICA) time courses data from functional magnetic resonance imaging (fMRI) data and feed it to a federated deep neural network (DNN) model to perform classification of schizophrenia patients versus controls.

Federated learning (FL) for distributed data has gained significant attention by enabling model training on local data without transferring it to a central system. While this approach protects sensitive information, risks of data leakage still persist, necessitating the integration of privacy-preserving techniques such as differential privacy. In many FL applications, tasks like exploratory data analysis or tracking and monitoring data that change over time are essential. For these purposes, analysts rely on data visualizations to make decisions or draw conclusions. This vision paper emphasizes the importance of federated privacy-preserving visualization and outlines a general pipeline for its implementation. We discuss the challenges of integrating federated visualizations with differential privacy and demonstrate the feasibility of this approach through examples, such as federated privacy-preserving boxplots, scatterplots, and correlation visualizations in neuroimaging. This highlights the need for further research in this promising field.

Reinforcement learning (RL) algorithms have been very successful at tackling complex control problems, such as AlphaGo or fusion control. However, current research mainly emphasizes solution quality, often achieved by using large models trained on large amounts of data, and does not account for the financial, environmental, and societal costs associated with developing and deploying such models. Modern neural networks are often overparameterized and a significant number of parameters can be pruned without meaningful loss in performance, resulting in more efficient use of the model's capacity lottery ticket. We present a methodology for identifying sub-networks within a larger network in reinforcement learning (RL). We call such sub-networks, neural pathways. We show empirically that even very small learned sub-networks, using less than 5% of the large network's parameters, can provide very good quality solutions. We also demonstrate the training of multiple pathways within the same networks in a multitask setup, where each pathway is encouraged to tackle a separate task. We evaluate empirically our approach on several continuous control tasks, in both online and offline training

Deep learning models can perform as well or better than humans in many tasks, especially vision related. Almost exclusively, these models are used to perform classification or prediction. However, deep learning models are usually of black-box nature, and it is often difficult to interpret the model or the features. The lack of interpretability causes a restrain from applying deep learning to fields such as neuroimaging, where the results must be transparent, and interpretable. Therefore, we present a 'glass-box' deep learning model and apply it to the field of neuroimaging. Our model mixes spatial and temporal dimensions in succession to estimate dynamic connectivity between the brain's intrinsic networks. The interpretable connectivity matrices produced by our model result in beating state-of-the-art models on many tasks using multiple functional MRI datasets. More importantly, our model estimates task-based flexible connectivity matrices, unlike static methods such as Pearson's correlation coefficients.

Data scarcity is a notable problem, especially in the medical domain, due to patient data laws. Therefore, efficient Pre-Training techniques are required to combat this problem. In this paper, we demonstrate that a model trained on the time direction of functional neuro-imaging data could help in the downstream classification tasks, for example, classifying diseases from healthy controls in fMRI data. We train a Deep Neural Network on Independent components derived from fMRI data using Independent component analysis (ICA) to learn time direction. This pre-trained model is further trained to classify brain disorders in different datasets. Through various experiments, we show that learning time direction helps a model learn some causal relation in fMRI data that helps in faster convergence and better generalization across various datasets, even with fewer data records available for training.

Recently, methods that represent data as a graph, such as graph neural networks (GNNs) have been successfully used to learn data representations and structures to solve classification and link prediction problems. The applications of such methods are vast and diverse, but most of the current work relies on the assumption of a static graph. This assumption does not hold for many highly dynamic systems, where the underlying connectivity structure is non-stationary and is mostly unobserved. Using a static model in these situations may result in suboptimal performance. In contrast, modeling changes in graph structure with time can provide information about the system whose applications go beyond classification. Most work of this type does not learn effective connectivity and focuses on cross-correlation between nodes to generate undirected graphs. An undirected graph is unable to capture direction of an interaction which is vital in many fields, including neuroscience. To bridge this gap, we developed dynamic effective connectivity estimation via neural network training (DECENNT), a novel model to learn an interpretable directed and dynamic graph induced by the downstream classification/prediction task. DECENNT outperforms state-of-the-art (SOTA) methods on five different tasks and infers interpretable task-specific dynamic graphs. The dynamic graphs inferred from functional neuroimaging data align well with the existing literature and provide additional information. Additionally, the temporal attention module of DECENNT identifies time-intervals crucial for predictive downstream task from multivariate time series data.

Privacy protection is one of the most crucial factors when sharing MR images between researchers. There are many defacing software programs that can blur or remove the face of MR images. However, there are reasons to believe that the brain and other remaining features are not only identifiable but also can be used for facial reconstruction to fill the face of the subject back into the image, and it is possible to rebuild the facial part of the images using recently developed machine learning models. A demonstration of this is of practical significance as it could convince the community to adopt stricter data-sharing standards. Additionally, even if the reconstructed faces are not identifiable, smooth completion of the “damaged” MRI images may improve methods that depend on the head modeling, such as source localization approaches. Recent work has been focusing on using the generative adversarial networks (GAN) for this task, which is generally believed to be the best method. Because we hope the model can generate the face entirely depending on the information from the brain, we show here an alternative approach, pixel constrained CNN, which is a purely supervised facial reconstruction. We simulated the rebuild process and showed convincing results.

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

Objective: A graph theoretical approach provides a powerful framework for discovering potential biomarkers of psychotic disorders. Comparing the brain graphs of the control and patient groups can help us to discover changes in mental disorders in a more convenient way. In this paper, we propose a novel tool to identify missing links associated with blocked paths (segregation) and new links associated with additional paths (abnormal integration) in estimated patient group's time-varying graphs. We highlight the approach in an example application to the resting-state functional magnetic resonance imaging (fMRI) data of schizophrenia (SZ) patients. Methods: We first estimated whole-brain functional connectivity dynamics using a combination of spatial independent component analysis (ICA), sliding time window, and k-means clustering of windowed correlation matrices on resting-state fMRI data. The clusters are regarded as functional connectivity states, and each of them includes time intervals exhibiting similar connectivity patterns. We then estimated a Gaussian graphical model (GGM) for each state for both groups. To evaluate this approach, we compared different paths between brain components (nodes) of SZ and control groups' graphs within each state by using the concept of connected components in graph theory. Results: We identified missing edges associated with disconnectivity (disconnectors) and showed there are additional edges in the SZ group graph that contribute to creating new paths in brain graphs. Conclusion: The proposed approach provides a tool for extracting time-varying graphs and identifying disconnectors associated with disconnectivity (absence of paths) and also connectors associated with abnormal integration (additional paths) in patient group graphs. Significance: We detected several missing links in SZ, both within and between functional domains, in particular within the subcortical (3 links) and somatomotor (4 links) domains. Interestingly, our proposed method identified new links within the somatomotor domain which may be related to a compensatory response in patients that warrants future study.

While the majority of exploratory approaches search for correlations among features of different modalities, indirect/nonlinear relations between structure and function have not yet been fully investigated. In this work, we employ a neural machine translation model [1] to relate two modalities: structural MRI (sMRI) spatial components and functional MRI (fMRI) brain states estimated using a dynamic connectivity model. We consider each of the modalities as different “languages” of the same brain and fit a translation model to estimate a model for how structure influences function. Results identify multiple aligned aspects of brain structure and functional brain states showing significantly more or less alignment in the patient group as well as interesting links to other variables such as cognitive scores and symptom assessments. Our novel approach provides a new perspective on combining brain structure and function by incorporating indirect/nonlinear effects and enabling the algorithm to learn the interplay between structural and the functional networks.

Independent vector analysis (IVA) is an approach for joint blind source separation of several data sets that learns simultaneous unmixing transforms for each set. It assumes corresponding sources from different data sets to be statistically dependent. One of the main advantages is IVA's ability to retain subject-specific differences while simplifying comparison across subjects as the resulting components have the same order. The latter is an instrumental property for enabling collaboration between remote sites without sharing their data, which may be required because of ethical, privacy or efficiency concerns. This paper proposes a new decentralized algorithm for IVA that exploits the structure of the objective function. A centralized aggregator coordinates IVA algorithms at multiple sites using message passing, parallelizing the computation and limiting the amount of communication. Thus, the algorithm enables a plausibly private collaboration across multiple sites. Besides enabling analysis of decentralized data, our approach improves the running time of IVA when used locally.

Visualization of high dimensional large-scale datasets via an embedding into a 2D map is a powerful exploration tool for assessing latent structure in the data and detecting outliers. There are many methods developed for this task but most assume that all pairs of samples are available for common computation. Specifically, the distances between all pairs of points need to be directly computable. In contrast, we work with sensitive neuroimaging data, when local sites cannot share their samples and the distances cannot be easily computed across the sites. Yet, the desire is to let all the local data participate in collaborative computation without leaving their respective sites. In this scenario, a quality control tool that visualizes decentralized dataset in its entirety via global aggregation of local computations is especially important as it would allow screening of samples that cannot be evaluated otherwise. This paper introduces an algorithm to solve this problem: decentralized data stochastic neighbor embedding (dSNE). Based on the MNIST dataset we introduce metrics for measuring the embedding quality and use them to compare dSNE to its centralized counterpart. We also apply dSNE to a multi-site neuroimaging dataset with encouraging results.

Segmenting a structural magnetic resonance imaging (MRI) scan is an important pre-processing step for analytic procedures and subsequent inferences about longitudinal tissue changes. Manual segmentation defines the current gold standard in quality but is prohibitively expensive. Automatic approaches are computationally intensive, incredibly slow at scale, and error prone due to usually involving many potentially faulty intermediate steps. In order to streamline the segmentation, we introduce a deep learning model that is based on volumetric dilated convolutions, subsequently reducing both processing time and errors. Compared to its competitors, the model has a reduced set of parameters and thus is easier to train and much faster to execute. The contrast in performance between the dilated network and its competitors becomes obvious when both are tested on a large dataset of unprocessed human brain volumes. The dilated network consistently outperforms not only another state-of-the-art deep learning approach, the up convolutional network, but also the ground truth on which it was trained. Not only can the incredible speed of our model make large scale analyses much easier but we also believe it has great potential in a clinical setting where, with little to no substantial delay, a patient and provider can go over test results.

Researchers often wish to study data stored in separate locations, such as when several research entities wish to make inferences from their combined data. The most common solution is to centralize the data in one location. However, certain types of data can be difficult to transfer between entities due to legal or practical reasons. This makes centralizing these types of data problematic. A possible solution is the use of methods that learn from data without moving them to a central location: decentralized algorithms. Only a few algorithms emphasizing that property are known to us, and even fewer are used in the biomedical domain. In this paper, we propose a decentralized neural network that allows data analysis without transferring the data from the sites that host them. Instead, this method only transfers the gradients (or their parts) calculated via back-propagation. Our approach allows us to learn a classifier even when class examples are located at different sites, enabling privacy-aware collaboration across groups with specific research interests. We validate the method in several experiments to test stability, compare performance to a network trained on the centralized data, and investigate the ability to reduce size of data transfer. Our experiments on simulated, benchmark, and neuroimaging addiction data provide strong evidence that the proposed model works as effectively as a pooled centralized model.

In collaborative medical research settings, a moderate number of groups (sites) may wish to merge local analyses of private subject data. Differential privacy offers one way to guarantee privacy for these local analyses. We describe a novel ensemble learning method that we call the "feature method" for aggregating binary classifiers or regressors trained on local data. Our method leverages a public data set available at the aggregator to optimize a linear combination of local predictors. We provide some analysis of the method and show how it is effective when the local sites are required to learn classifiers that are differentially private. We prove that this method has near-optimal performance when local data sets are large enough under certain requirements on the parameters. Experimentally, we give a comparison of the feature method and the standard approach of averaging the local classifiers.

Causal structure learning from time series data is a major scientific challenge. Existing algorithms assume that measurements occur sufficiently quickly; more precisely, they assume that the system and measurement timescales are approximately equal. In many scientific domains, however, measurements occur at a significantly slower rate than the underlying system changes. Moreover, the size of the mismatch between timescales is often unknown. This paper provides three distinct causal structure learning algorithms, all of which discover all dynamic graphs that could explain the observed measurement data as arising from undersampling at some rate. That is, these algorithms all learn causal structure without assuming any particular relation between the measurement and system timescales; they are thus rate-agnostic. We apply these algorithms to data from simulations. The results provide insight into the challenge of undersampling.

Despite its multivariate nature, independent component analysis (ICA) is generally limited to univariate latents in the sense that each latent component is a scalar process. Independent subspace analysis (ISA), or multidimensional ICA (MICA), is a generalization of ICA which identifies latent independent vector components instead. While ISA/MICA considers multidimensional latent components within a single dataset, our work specifically considers the case of multiple datasets. Independent vector analysis (IVA) is a related technique that also considers multiple datasets explicitly but with a fixed and constrained model. Here, we first show that 1) ISA/MICA naturally extends to the case of multiple datasets (which we call MISA), and that 2) IVA is a special case of this extension. Then we develop an algorithm for MISA and demonstrate its performance on both IVA- and MISA-type problems. The benefit of these extensions is that the vector sources (or subspaces) capture higher order statistical dependence across datasets while retaining independence between subspaces. This is a promising model that can explore complex latent relations across multiple datasets and help identify novel biological traits for intricate mental illnesses such as schizophrenia.

Nonnegative Least Squares (NNLS) is a general form for many important problems. We consider a special case of NNLS where the input is nonnegative. It is called Totally Nonnegative Least Squares (TNNLS) in the literature. We show a reduction of TNNLS to a single class Support Vector Machine (SVM), thus relating the sparsity of a TNNLS solution to the sparsity of supports in a SVM. This allows us to apply any SVM solver to the TNNLS problem. We get an order of magnitude improvement in running time by first obtaining a smaller version of our original problem with the same solution using a fast approximate SVM solver. Second, we use an exact NNLS solver to obtain the solution. We present experimental evidence that this approach improves the performance of state-of-the-art NNLS solvers by applying it to both randomly generated problems as well as to real datasets, calculating radiation therapy dosages for cancer patients.

Distributions over permutations arise in applications ranging from multi-object tracking to ranking of instances. The difficulty of dealing with these distributions is caused by the size of their domain, which is factorial in the number of considered entities (n!). It makes the direct definition of a multinomial distribution over permutation space impractical for all but a very small n. In this work we propose an embedding of all n! permutations for a given n in a surface of a hyper sphere defined in ℝ(n-1). As a result of the embedding, we acquire ability to define continuous distributions over a hyper sphere with all the benefits of directional statistics. We provide polynomial time projections between the continuous hyper sphere representation and the n!-element permutation space. The framework provides a way to use continuous directional probability densities and the methods developed thereof for establishing densities over permutations. As a demonstration of the benefits of the framework we derive an inference procedure for a state-space model over permutations. We demonstrate the approach with simulations on a large number of objects hardly manageable by the state of the art inference methods, and an application to a real flight traffic control dataset.

The dual formulation of the support vector machine (SVM) objective function is an instance of a nonnegative quadratic programming problem. We reformulate the SVM objective function as a matrix factorization problem which establishes a connection with the regularized nonnegative matrix factorization (NMF) problem. This allows us to derive a novel multiplicative algorithm for solving hard and soft margin SVM. The algorithm follows as a natural extension of the updates for NMF and semi-NMF. No additional parameter setting, such as choosing learning rate, is required. Exploiting the connection between SVM and NMF formulation, we show how NMF algorithms can be applied to the SVM problem. Multiplicative updates that we derive for SVM problem also represent novel updates for semi-NMF. Further this unified view yields algorithmic insights in both directions: we demonstrate that the Kernel Adatron algorithm for solving SVMs can be adapted to NMF problems. Experiments demonstrate rapid convergence to good classifiers. We analyze the rates of asymptotic convergence of the updates and establish tight bounds. We test them on several datasets using various kernels and report equivalent classification performance to that of a standard SVM.