Catherine D. Schuman

dblp:42/3029 · DBLP profile ↗
← Back
41ranked-venue papers
7as first author
17since 2021 · last 2025
0000-0002-4264-8097ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 25 · 7 first-author · 8 since 2021Systems, architecture and hardware · 14 · 8 since 2021Databases, data management, data science and information retrieval · 5Applied, interdisciplinary, general and emerging computing · 5 · 1 since 2021Security and privacy · 1Software engineering, systems software and programming languages · 1 · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 1
YearPublicationVenuePosition
2025 AI-Guided Codesign for Novel Computing Paradigms
abstract
Microelectronics design is often a labor-intensive process involving extensive simulations, fabrication, and testing, particularly in analog design, which demands a skilled workforce with specialized knowledge. Emerging computing paradigms, such as neuromorphic and probabilistic computing, aim to harness the analog characteristics of devices for significant performance improvements over traditional methods. This presents a unique codesign challenge across the design stack, encompassing analog, mixed-signal, and beyond-CMOS devices. In this work, we introduce AI-guided codesign automation techniques, for the design of novel devices and circuits tailored for these cutting-edge computing paradigms, facilitating innovative solutions and hardware-aware algorithms for next-generation heterogeneous architectures.
Suma Cardwell, J. Darby Smith, Karan Patel, Andrew Maicke, Jared Arzate, Samuel Liu, Jaesuk Kwon, Christopher Allemang, Douglas Cale Crowder, Shashank Misra, Frances S. Chance, Catherine D. Schuman, Jean Anne C. Incorvia, James B. Aimone
ASP-DAC12
2025 An Exploration of a Heterogeneous Neural Configuration of SNNs
George Evans, Karan Patel, Catherine D. Schuman, Garrett S. Rose, Srutarshi Banerjee, Hritom Das
ACM Great Lakes Symposium on VLSI3
2025 Evolving Collective Robot Swarm Behavior in Neuromorphic Systems
abstract
Neuromorphic systems are a natural hardware system for robotic control because of their low size, weight, and power implementations, as well as their temporal processing capabilities. In this work, we evaluate how well neuromorphic systems are able to control and work together in a collective swarm of robots. We design simple robot structures, and we evolve and evaluate spiking neural networks to deploy to neuromorphic implementations on those robots.
Andrew Friend, Meghan Brandt, Luke McCombs, James S. Plank, Catherine D. Schuman
IJCNN5
2025 Exploring Spiking Neural Networks for Binary Classification in Multivariate Time Series at the Edge
abstract
We present a general framework for training spiking neural networks (SNNs) to perform binary classification on multivariate time series, with a focus on step-wise prediction and high precision at low false alarm rates. The approach uses the Evolutionary Optimization of Neuromorphic Systems (EONS) algorithm to evolve sparse, stateful SNNs by jointly optimizing their architectures and parameters. Inputs are encoded into spike trains, and predictions are made by thresholding a single output neuron’s spike counts. We also incorporate simple voting ensemble methods to improve performance and robustness.To evaluate the framework, we apply it with application-specific optimizations to the task of detecting low signal-to-noise ratio radioactive sources in gamma-ray spectral data. The resulting SNNs, with as few as 49 neurons and 66 synapses, achieve a 51.8% true positive rate (TPR) at a false alarm rate of 1/hr, outperforming PCA (42.7%) and deep learning (49.8%) baselines. A three-model any-vote ensemble increases TPR to 67.1% at the same false alarm rate. Hardware deployment on the μCaspian neuromorphic platform demonstrates 2 mW power consumption and 20.2 ms inference latency.We also demonstrate generalizability by applying the same framework, without domain-specific modification, to seizure detection in EEG recordings. An ensemble achieves 95% TPR with a 16% false positive rate, comparable to recent deep learning approaches with significant reduction in parameter count.
James Ghawaly, Andrew D. Nicholson, Catherine D. Schuman, Dalton Diez, Aaron R. Young, Brett Witherspoon
IJCNN3
2024 Transductive Spiking Graph Neural Networks for Loihi
abstract
Graph neural networks have emerged as a specialized branch of deep learning, designed to address problems where pairwise relations between objects are crucial. Recent advancements utilize graph convolutional neural networks to extract features within graph structures. Despite promising results, these methods face challenges in real-world applications due to sparse features, resulting in inefficient resource utilization. Recent studies draw inspiration from the mammalian brain and employ spiking neural networks to model and learn graph structures. However, these approaches are limited to traditional Von Neumann-based computing systems, which still face hardware inefficiencies. In this study, we present a fully neuromorphic implementation of spiking graph neural networks designed for Loihi 2. We optimize network parameters using Lava Bayesian Optimization, a novel hyperparameter optimization system compatible with neuromorphic computing architectures. We showcase the performance benefits of combining neuromorphic Bayesian optimization with our approach for citation graph classification using fixed-precision spiking neurons. Our results demonstrate the capability of integer-precision, Loihi 2 compatible spiking neural networks in performing citation graph classification with comparable accuracy to existing floating point implementations.
Shay Snyder, Victoria Clerico, Guojing Cong, Shruti R. Kulkarni, Catherine D. Schuman, Sumedh R. Risbud, Maryam Parsa
ACM Great Lakes Symposium on VLSI5
2024 Device Codesign using Reinforcement Learning
abstract
We demonstrate device codesign using reinforcement learning for probabilistic computing applications. We use a spin orbit torque magnetic tunnel junction model (SOT-MTJ) as the device exemplar. We leverage reinforcement learning (RL) to vary key device and material properties of the SOT-MTJ device for stochastic operation. Our RL method generated different candidate devices capable of generating stochastic samples for a given exponential distribution.
Suma Cardwell, Karan Patel, Catherine D. Schuman, J. Darby Smith, Jaesuk Kwon, Andrew Maicke, Jared Arzate, Jean Anne C. Incorvia
ISCAS3
2023 Hyperparameter Optimization and Feature Inclusion in Graph Neural Networks for Spiking Implementation
abstract
Graph convolutional networks leverage both graph structures and features on nodes and edges for improved learning performance in comparison with classical machine learning approaches. Spiking neuromorphic computers natively implement network-like computation and have been shown to be successful at implementing graph learning without features. Incorporating graph features brings the challenge of efficient feature representation and balancing the contribution of topology and features in learning. In this work, we present our design of a simulated network of spiking neurons to perform semi-supervised learning on graph data using both the graph structure and the node features. We explore various design choices, present preliminary results, and discuss the opportunities for using neuromorphic computers for this task in the future.
Guojing Cong, Shruti R. Kulkarni, Seung-Hwan Lim, Prasanna Date, Shay Snyder, Maryam Parsa, Dominic Kennedy, Catherine D. Schuman
ICMLA8
2023 Composable Workflow for Accelerating Neural Architecture Search Using In Situ Analytics for Protein Classification
abstract
Neural architecture search (NAS), which automates the design of neural network (NN) architectures for scientific datasets, requires significant computational resources and time — often on the order of days or weeks of GPU hours and training time. We design the Analytics for Neural Network (A4NN) workflow, a composable workflow that significantly reduces the time and resources required to design accurate and efficient NN architectures. We introduce a parametric fitness prediction strategy and distribute training across multiple accelerators to decrease the aggregated NN training time. A4NN rigorously record neural architecture histories, model states, and metadata to reproduce the search for near-optimal NNs. We demonstrate A4NN’s ability to reduce training time and resource consumption on a dataset generated by an X-ray Free Electron Laser (XFEL) experiment simulation. When deploying A4NN, we decrease training time by up to 37% and epochs required by up to 38%.
Georgia Channing, Ria Patel, Paula Olaya, Ariel Keller Rorabaugh, Osamu Miyashita, Silvina Caíno-Lores, Catherine D. Schuman, Florence Tama, Michela Taufer
ICPP7
2023 Avoiding excess computation in asynchronous evolutionary algorithms
abstract
Abstract Asynchronous evolutionary algorithms are becoming increasingly popular as a means of making full use of many processors while solving computationally expensive search and optimization problems. These algorithms excel at keeping large clusters fully utilized, but may sometimes inefficiently sample an excess of fast‐evaluating solutions at the expense of higher‐quality, slow‐evaluating ones. We have previously introduced a steady‐state parent selection strategy, SWEET (“Selection whilE EvaluaTing”), that sometimes selects individuals that are still being evaluated and allows them to reproduce early. We perform a takeover‐time analysis that confirms that this strategy gives slow‐evaluating individuals that have higher fitnesses an increased ability to multiply in the population. We also find that SWEET appears effective at improving optimization performance on problems in which solution quality is positively correlated with evaluation time. We evaluate our approach on six simulated real‐valued optimization problems and three real‐world applications: an autonomous vehicle controller problem that involves tuning a spiking neural network and two adversarial EA problems. We further evaluate SWEET versus a basic asynchronous process in a simulated setting. We present evidence that SWEET outperforms basic asynchronous processes in a use‐case in which performance is positively correlated with evaluation time, and performs comparably (and often better) than basic asynchronous processes in several use‐cases where performance is negatively correlated with evaluation time. That said, in the cases where performance and evaluation time are negatively correlated the variance of outcomes for SWEET is notably high.
Eric O. Scott, Mark Coletti, Catherine D. Schuman, Bill Kay, Shruti R. Kulkarni, Maryam Parsa, Chathika Gunaratne, Kenneth A. De Jong
Expert Syst. J. Knowl. Eng.3
2022 A Methodology to Generate Efficient Neural Networks for Classification of Scientific Datasets
abstract
Neural networks (NNs) are increasingly utilized in high-throughput scientific workflows. In this context, NN efficiency is essential for successful workflow management. We use a multi-objective Neural Architecture Search (NAS), NSGA-Net, to search for highly accurate NNs while optimizing for efficient use of computational resources by minimizing FLoating-point Operations Per Second (FLOPS). We define a domain-agnostic methodology to generate NNs with the support of NSGA-Net, select promising NNs that balance accuracy and FLOPS usage, and refine a subset of NNs in order to curate networks suitable for efficient data analysis. We apply this methodology to a protein diffraction use case. Preliminary results show NNs that efficiently classify conformation of proteins with a final accuracy of 97.7% or higher and using only 187 FLOPS.
Ria Patel, Ariel Keller Rorabaugh, Paula Olaya, Silvina Caíno-Lores, Georgia Channing, Catherine D. Schuman, Osamu Miyashita, Florence Tama, Michela Taufer
e-Science6
2022 Unsupervised Digit Recognition Using Cosine Similarity In A Neuromemristive Competitive Learning System
abstract
This work addresses how to naturally adopt the l 2 -norm cosine similarity in the neuromemristive system and studies the unsupervised learning performance on handwritten digit image recognition. Proposed architecture is a two-layer fully connected neural network with a hard winner-take-all (WTA) learning module. For input layer, we propose single-spike temporal code that transforms input stimuli into the set of single spikes with different latencies and voltage levels. For a synapse model, we employ a compound memristor where stochastically switching binary-state memristors connected in parallel, which offers a reliable and scalable multi-state solution for synaptic weight storage. Hardware-friendly synaptic adaptation mechanism is proposed to realize spike-timing-dependent plasticity learning. Input spikes are sent out through those memristive synapses to each and every integrate-and-fire neuron in the fully connected output layer, where the hard WTA network motif introduces the competition based on cosine similarity for the given input stimuli. Finally, we present 92.64% accuracy performance on unsupervised digit recognition with only single-epoch MNIST dataset training via high-level simulations, including extensive analysis on the impact of system parameters.
Bon Woong Ku, Catherine D. Schuman, Md Musabbir Adnan, Tiffany M. Mintz, Raphael C. Pooser, Kathleen E. Hamilton, Garrett S. Rose, Sung Kyu Lim
ACM J. Emerg. Technol. Comput. Syst.2
2022 Guest Editorial: Special Section on Parallel and Distributed Computing Techniques for Non-Von Neumann Technologies
Scott Pakin, Christof Teuscher, Catherine D. Schuman
IEEE Trans. Parallel Distributed Syst.3
2021 Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution
abstract
Neuroevolution has had significant success over recent years, but there has been relatively little work applying neuroevolution approaches to spiking neural networks (SNNs). SNNs are a type of neural networks that include temporal processing component, are not easily trained using other methods because of their lack of differentiable activation functions, and can be deployed into energy-efficient neuromorphic hardware. In this work, we investigate two evolutionary approaches for training SNNs. We explore the impact of the hyperparameters of the evolutionary approaches, including tournament size, population size, and representation type, on the performance of the algorithms. We present a multi-objective Bayesian-based hyperparameter optimization approach to tune the hyperparameters to produce the most accurate and smallest SNNs. We show that the hyperparameters can significantly affect the performance of these algorithms. We also perform sensitivity analysis and demonstrate that every hyperparameter value has the potential to perform well, assuming other hyperparameter values are set correctly.
Maryam Parsa, Shruti R. Kulkarni, Mark Coletti, Jeffrey K. Bassett, J. Parker Mitchell, Catherine D. Schuman
CEC6
2021 A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems
abstract
There are a wide variety of training approaches for spiking neural networks for neuromorphic deployment. However, it is often not clear how these training algorithms perform or compare when applied across multiple neuromorphic hardware platforms and multiple datasets. In this work, we present a software framework for comparing performance across four neuromorphic training algorithms across three neuromorphic simulators and four simple classification tasks. We introduce an approach for training a spiking neural network using a decision tree, and we compare this approach to training algorithms based on evolutionary algorithms, back-propagation, and reservoir computing. We present a hyperparameter optimization approach to tune the hyperparameters of the algorithm, and show that these optimized hyperparameters depend on the processor, algorithm, and classification task. Finally, we compare the performance of the optimized algorithms across multiple metrics, including accuracy, training time, and resulting network size, and we show that there is not one best training algorithm across all datasets and performance metrics.
Catherine D. Schuman, James S. Plank, Maryam Parsa, Shruti R. Kulkarni, Nicholas D. Skuda, J. Parker Mitchell
IJCNN1
2021 Benchmarking the performance of neuromorphic and spiking neural network simulators
Shruti R. Kulkarni, Maryam Parsa, J. Parker Mitchell, Catherine D. Schuman
Neurocomputing4
2021 Stochasticity and robustness in spiking neural networks
Wilkie Olin-Ammentorp, Karsten Beckmann, Catherine D. Schuman, James S. Plank, Nathaniel C. Cady
Neurocomputing3
2021 Design of a Robust Memristive Spiking Neuromorphic System with Unsupervised Learning in Hardware
abstract
Spiking neural networks (SNN) offer a power efficient, biologically plausible learning paradigm by encoding information into spikes. The discovery of the memristor has accelerated the progress of spiking neuromorphic systems, as the intrinsic plasticity of the device makes it an ideal candidate to mimic a biological synapse. Despite providing a nanoscale form factor, non-volatility, and low-power operation, memristors suffer from device-level non-idealities, which impact system-level performance. To address these issues, this article presents a memristive crossbar-based neuromorphic system using unsupervised learning with twin-memristor synapses, fully digital pulse width modulated spike-timing-dependent plasticity, and homeostasis neurons. The implemented single-layer SNN was applied to a pattern-recognition task of classifying handwritten-digits. The performance of the system was analyzed by varying design parameters such as number of training epochs, neurons, and capacitors. Furthermore, the impact of memristor device non-idealities, such as device-switching mismatch, aging, failure, and process variations, were investigated and the resilience of the proposed system was demonstrated.
Md Musabbir Adnan, Sagarvarma Sayyaparaju, Samuel D. Brown, Shamim Ara Shawkat, Catherine D. Schuman, Garrett S. Rose
ACM J. Emerg. Technol. Comput. Syst.5
2020 GRANT: Ground-Roaming Autonomous Neuromorphic Targeter
abstract
In this work we describe the design, implementation, and testing of the first neuromorphic robot capable of obstacle avoidance, grid coverage, and targeting controlled by the second generation Dynamic Adaptive Neural Network Array (DANNA2) digital spiking neuromorphic processor. The simplicity of the DANNA2 processor along with the TENNLab hardware/software co-design framework allows for compact spiking networks that can run efficiently on a small, resource-constrained, platform such as a Xilinx Artix-7 field-programmable gate array. Additionally, we present the dynamic reconfigurability of DANNA2 arrays as a method of realizing complex, multi-objective tasks on hardware that is restricted to relatively small networks.
Jonathan D. Ambrose, Adam Z. Foshie, Mark E. Dean, James S. Plank, Garrett S. Rose, J. Parker Mitchell, Catherine D. Schuman, Grant Bruer
IJCNN7
2020 Hyperparameter Optimization in Binary Communication Networks for Neuromorphic Deployment
abstract
Training neural networks for neuromorphic deployment is non-trivial. There have been a variety of approaches proposed to adapt back-propagation or back-propagation-like algorithms appropriate for training. Considering that these networks often have very different performance characteristics than traditional neural networks, it is often unclear how to set either the network topology or the hyperparameters to achieve optimal performance. In this work, we introduce a Bayesian approach for optimizing the hyperparameters of an algorithm for training binary communication networks that can be deployed to neuromorphic hardware. We show that by optimizing the hyperparameters on this algorithm for each dataset, we can achieve improvements in accuracy over the previous state-of-the-art for this algorithm on each dataset (by up to 15 percent). This jump in performance continues to emphasize the potential when converting traditional neural networks to binary communication applicable to neuromorphic hardware.
Maryam Parsa, Catherine D. Schuman, Prasanna Date, Derek C. Rose, Bill Kay, J. Parker Mitchell, Steven R. Young, Ryan Dellana, William Severa, Thomas E. Potok, Kaushik Roy 0001
IJCNN2
2020 Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems
abstract
Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear. In this work, we study the effect of failures on the performance of four different training algorithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). We show that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. We then analyze an approach for training more resilient spiking neural networks using the evolutionary optimization approach. We show how this approach produces more resilient networks and discuss how it can be extended to other spiking neural network training approaches as well.
Catherine D. Schuman, J. Parker Mitchell, J. Travis Johnston, Maryam Parsa, Bill Kay, Prasanna Date, Robert M. Patton
IJCNN1
2020 Automated Design of Neuromorphic Networks for Scientific Applications at the Edge
abstract
Designing spiking neural networks for neuromorphic deployment is a non-trivial task. It is further complicated when there are resource constraints for the neuromorphic implementation, such as size or power constraints, that may be present in edge applications. In this work, we utilize a previously presented approach, EONS, to design spiking neural networks for a memristive neuromorphic implementation for scientific data applications. We specifically use a multi-objective approach in EONS to maximize network accuracy on the scientific data application task, but also to minimize network size and energy. We illustrate that EONS determines both the network structure and the parameters, removing the burden from the user on determining the appropriate spiking neural network structure, and we show that the resulting networks are very different from the layered structure of typical neural networks. Finally, we show that the multi-objective approach produces smaller, more energy efficient networks than the original EONS approach and produces comparable accuracy to a back-propagation style training approach.
Catherine D. Schuman, J. Parker Mitchell, Maryam Parsa, James S. Plank, Samuel D. Brown, Garrett S. Rose, Robert M. Patton, Thomas E. Potok
IJCNN1
2020 Scaled-up Neuromorphic Array Communications Controller (SNACC) for Large-scale Neural Networks
abstract
Neuromorphic computing is one promising post-Moore's law era technology, which takes inspiration from biological brains to perform computing tasks. The human brain contains billions of neurons with trillions of synapses and as neuromorphic hardware systems scale to larger and larger sizes, the communication system used to transfer information between neuromorphic elements and traditional computers must scale to keep up. In prior work, we describe the use of a separate neuromorphic array communications controller to support low-latency, high-throughput communication between our neuromorphic systems and a traditional computer. In this work, the neuromorphic array communications controller is used to support the scaling of a neuromorphic development system which uses multiple neuromorphic processors arranged in a two-dimensional array. The neuromorphic array communications controller, along with scalable local connections, is used to create a scalable neuromorphic platform to enable the development and testing of large neuromorphic network arrays.
Aaron R. Young, Adam Z. Foshie, Mark E. Dean, James S. Plank, Garrett S. Rose, J. Parker Mitchell, Catherine D. Schuman
IJCNN7
2019 Visualization System for Evolutionary Neural Networks for Deep Learning
abstract
Deep learning is actively used in a wide range of fields for scientific discovery. To effectively apply deep learning to a particular problem, it is important to select an appropriate network architecture and other hyper-parameters (at each layer). Evolving architectures and hyper-parameters using a genetic algorithm is one current approach to search the huge space of all possible configurations to find those more optimal for the problem. However, examining an evolutionary process and tuning the genetic algorithm are challenging, pushing most users to treat the process as a black box. To address this challenge, we propose a visualization system for evolutionary neural networks for deep learning. The key feature of our visualization system is to provide a visual analytics environment for evaluating a genetic algorithm in order to improve the underlying operations to reduce time to find good solutions. Our system is able to not only visualize how a genetic algorithm traverses its search space but also allows users to examine evolving networks in-depth to get insights to improve performance through interactive visualization components.
Junghoon Chae, Catherine D. Schuman, Steven R. Young, J. Travis Johnston, Derek C. Rose, Robert M. Patton, Thomas E. Potok
IEEE BigData2
2019 Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems
abstract
Designing a neuromorphic computing system involves selection of several hyperparameters that not only affect the accuracy of the framework, but also the energy efficiency and speed of inference and training. These hyperparameters might be inherent to the training of the spiking neural network (SNN), the input/output encoding of the real-world data to spikes, or the underlying neuromorphic hardware. In this work, we present a Bayesian-based hyperparameter optimization approach for spiking neuromorphic systems, and we show how this optimization framework can lead to significant improvement in designing accurate neuromorphic computing systems. In particular, we show that this hyperparameter optimization approach can discover the same optimal hyperparameter set for input encoding as a grid search, but with far fewer evaluations and far less time. We also show the impact of hardware-specific hyperparameters on the performance of the system, and we demonstrate that by optimizing these hyperparameters, we can achieve significantly better application performance.
Maryam Parsa, J. Parker Mitchell, Catherine D. Schuman, Robert M. Patton, Thomas E. Potok, Kaushik Roy 0001
IEEE BigData3
2019 Exascale Deep Learning to Accelerate Cancer Research
abstract
Deep learning, through the use of neural networks, has demonstrated remarkable ability to automate many routine tasks when presented with sufficient data for training. The neural network architecture (e.g. number of layers, types of layers, connections between layers, etc.) plays a critical role in determining what, if anything, the neural network is able to learn from the training data. The trend for neural network architectures, especially those trained on ImageNet, has been to grow ever deeper and more complex. The result has been ever increasing accuracy on benchmark datasets with the cost of increased computational demands. In this paper we demonstrate that neural network architectures can be automatically generated, tailored for a specific application, with dual objectives: accuracy of prediction and speed of prediction. Using MENNDL- an HPC-enabled software stack for neural architecture search-we generate a neural network with comparable accuracy to state-of-the-art networks on a cancer pathology dataset that is also 16× faster at inference. The speedup in inference is necessary because of the volume and velocity of cancer pathology data; specifically, the previous state-of-the-art networks are too slow for individual researchers without access to HPC systems to keep pace with the rate of data generation. Our new model enables researchers with modest computational resources to analyze newly generated data faster than it is collected.
Robert M. Patton, Shahira Abousamra, Dimitris Samaras, Joel H. Saltz, J. Travis Johnston, Steven R. Young, Catherine D. Schuman, Thomas E. Potok, Derek C. Rose, Seung-Hwan Lim, Junghoon Chae, Le Hou
IEEE BigData7
2019 Evolving Energy Efficient Convolutional Neural Networks
abstract
As deep neural networks have been deployed in more and more applications over the past half decade and are finding their way into an ever increasing number of operational systems, their energy consumption becomes a concern whether running in the datacenter or on edge devices. Hyperparameter optimization and automated network design for deep learning is a quickly growing field, but much of the focus has remained only on optimizing for the performance of the machine learning task. In this work, we demonstrate that the best performing networks created through this automated network design process have radically different computational characteristics (e.g. energy usage, model size, inference time), presenting the opportunity to utilize this optimization process to make deep learning networks more energy efficient and deployable to smaller devices. Optimizing for these computational characteristics is critical as the number of applications of deep learning continues to expand.
Steven R. Young, Pravallika Devineni, Maryam Parsa, J. Travis Johnston, Bill Kay, Robert M. Patton, Catherine D. Schuman, Derek C. Rose, Thomas E. Potok
IEEE BigData7
2019 Deep Learning for Vertex Reconstruction of Neutrino-nucleus Interaction Events with Combined Energy and Time Data
abstract
We present a deep learning approach for vertex reconstruction of neutrino-nucleus interaction events, a problem in the domain of high energy physics. In this approach, we combine both energy and timing data that are collected in the MIN-ERvA detector to perform classification and regression tasks. We show that the resulting network achieves higher accuracy than previous results while requiring a smaller model size and less training time. In particular, the proposed model outperforms the state-of-the-art by 4.00% on classification accuracy. For the regression task, our model achieves 0.9919 on the coefficient of determination, higher than the previous work (0.96).
Linghao Song, Fan Chen 0001, Steven R. Young, Catherine D. Schuman, Gabriel N. Perdue, Thomas E. Potok
ICASSP4
2019 Intelligent Reservoir Generation for Liquid State Machines using Evolutionary Optimization
abstract
Neuromorphic Computing is a burgeoning field of research. Many groups are exploring hardware architectures and theoretical ideas about spiking recurrent neural networks. The overarching goal is to exploit the low power promise of these neuromorphic systems. However, it is difficult to train spiking recurrent neural networks (SRNNs) to perform tasks and make efficient use of neuromorphic hardware. Reservoir Computing is an attractive methodology because it requires no tuning of weights for the reservoir itself. Yet, to find optimal reservoirs, manual tuning of hyperparameters such as hidden neurons, synaptic density, and natural structure is still required. Because of this, researchers often have to generate and evaluate many networks, which can result in non-trivial amounts of computation. This paper employs the reservoir computing technique (specifically liquid state machines) and genetic algorithms in order to develop useful networks that can be deployed on neuromorphic hardware. We build on past work in reservoir computing and genetic algorithms to demonstrate the power of combining these two techniques and the advantage it can provide over manually tuning reservoirs for use on classification tasks. We discuss the complexities of determining whether or not to use the genetic algorithms approach for liquid state machine generation.
John Reynolds 0001, James S. Plank, Catherine D. Schuman
IJCNN3
2019 Non-Traditional Input Encoding Schemes for Spiking Neuromorphic Systems
abstract
A key challenge for utilizing spiking neural networks or spiking neuromorphic systems for most applications is translating numerical data into spikes that are appropriate to apply as input to a spiking neural network. In this work, we present several approaches for encoding numerical values as spikes, including binning, spike-count encoding, and charge-injection encoding, and we show how these approaches can be combined hierarchically to form more complex encoding schemes. We demonstrate how these different encoding approaches perform on four different applications, running on four different neuromorphic systems that are based on spiking neural networks. We show that the input encoding method can have a significant effect on application performance and that the best input encoding method is application-specific.
Catherine D. Schuman, James S. Plank, Grant Bruer, Jeremy Anantharaj
IJCNN1
2018 Energy and Area Efficiency in Neuromorphic Computing for Resource Constrained Devices
abstract
Resource constrained devices are the building blocks of the internet of things (IoT) era. Since the idea behind IoT is to develop an interconnected environment where the devices are tiny enough to operate with limited resources, several control systems have been built to maintain low energy and area consumption while operating as IoT edge devices. Several researchers have begun work on implementing control systems built from resource constrained devices using machine learning. However, there are many ways such devices can achieve lower power consumption and area utilization while maximizing application efficiency. Spiky neuromorphic computing (SNC) is an emerging paradigm that can be leveraged in resource constrained devices for several emerging applications. While delivering the benefits of machine learning, SNC also helps minimize power consumption. For example, low energy memory devices (memristors) are often used to achieve low power operation and also help in reducing system area. In total, we anticipate SNC will provide computational efficiency approaching that of deep learning while using low power, resource constrained devices.
Gangotree Chakma, Nicholas D. Skuda, Catherine D. Schuman, James S. Plank, Mark E. Dean, Garrett S. Rose
ACM Great Lakes Symposium on VLSI3
2018 Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks
abstract
Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.
Catherine D. Schuman, Grant Bruer, Aaron R. Young, Mark E. Dean, James S. Plank
IJCNN1
2018 Neuromorphic Array Communications Controller to Support Large-Scale Neural Networks
abstract
Neuromorphic computing is one promising post-Moore's law era technology. In order to develop and use neuromorphic systems, traditional von Neumann-based computers must be able to communicate with neuromorphic hardware to support functionality such as monitoring the state of the network, optimizing the array to better perform the task, and input/output data processing. In this paper, we describe our use of a separate neuromorphic array communications controller to support highthroughput, low-latency communication between a traditional computer and our implementations of neuromorphic systems. The goal of the communications controller is to provide enough performance to facilitate the desired interaction between the systems and to enable scaling of the neuromorphic systems to larger sizes.
Aaron R. Young, Mark E. Dean, James S. Plank, Garrett S. Rose, Catherine D. Schuman
IJCNN5
2018 High-Level Simulation for Spiking Neuromorphic Computing Systems
abstract
Neuromorphic computing systems are alternatives to conventional microprocessors, often built from unconventional hardware. Designing and evaluating these systems requires multiple levels of simulation, from the device level to the circuit level to the system level. In this paper, we describe the system level simulator of a neuromorphic computing system based on memristors. We compare it to a circuit level simulator of the same system, both verifying its accuracy and demonstrating its performance improvement. We argue that system level simulation is an essential part of the design process of neuromorphic systems.
Nicholas D. Skuda, Catherine D. Schuman, Gangotree Chakma, James S. Plank, Garrett S. Rose
ISCAS2
2018 167-PFlops deep learning for electron microscopy: from learning physics to atomic manipulation
Robert M. Patton, J. Travis Johnston, Steven R. Young, Catherine D. Schuman, Don D. March, Thomas E. Potok, Derek C. Rose, Seung-Hwan Lim, Thomas P. Karnowski, Maxim A. Ziatdinov, Sergei V. Kalinin
SC4
2018 A Study of Complex Deep Learning Networks on High-Performance, Neuromorphic, and Quantum Computers
abstract
Current deep learning approaches have been very successful using convolutional neural networks trained on large graphical-processing-unit-based computers. Three limitations of this approach are that (1) they are based on a simple layered network topology, i.e., highly connected layers, without intra-layer connections; (2) the networks are manually configured to achieve optimal results, and (3) the implementation of the network model is expensive in both cost and power. In this article, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. We use the MNIST dataset for our experiment, due to input size limitations of current quantum computers. Our results show the feasibility of using the three architectures in tandem to address the above deep learning limitations. We show that a quantum computer can find high quality values of intra-layer connection weights in a tractable time as the complexity of the network increases, a high performance computer can find optimal layer-based topologies, and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware.
Thomas E. Potok, Catherine D. Schuman, Steven R. Young, Robert M. Patton, Federico M. Spedalieri, Jeremy Liu, Ke-Thia Yao, Garrett S. Rose, Gangotree Chakma
ACM J. Emerg. Technol. Comput. Syst.2
2017 Structure-based fitness prediction for the variable-structure DANNA neuromorphic architecture
abstract
In recent years, research on neuromporphic computing platforms has focused on variable-structure, spiking network models. An important methodology for programming these networks is evoluationary optimization (EO), where thousands of networks are generated and then evaluated by determining fitness scores on specific tasks. Fitness scores guide the generation of new networks until a target fitness is achieved. One source of performance overhead during EO is the simulation of the task on each network to determing its fitness. To mitigate this source of overhead, we formulate the Static Fitness Prediction Task (SFPT), for predicting a network's fitness without direct simulation. Our hypothesis is that we can use SFPT to predict a network's fitness sufficiently accurately to reject a significant portion of networks during EO without having to simulate them, thereby making the EO more efficient. We propose a data-driven approach to the SFPT on the neuromorphic model DANNA [1]. Our approach transforms networks into directed graphs and extracts structural features to train an ancillary model for predicting the fitness of new networks. We analyze the extracted features and evaluate several predictive models to predict the fitness of networks for five tasks. Our results demonstrate a predictive capacity in these features and models. Our primary contribution is to demonstrate the utility of graph-level features extracted from variable-structure networks to predict network fitness and circumvent expensive simulations.
Aleksander Klibisz, Grant Bruer, James S. Plank, Catherine D. Schuman
IJCNN4
2017 The effect of biologically-inspired mechanisms in spiking neural networks for neuromorphic implementation
abstract
An open question in neuromorphic computing is what neuron and synapse models should be used and what level of biological detail should be included in those models. For neuromorphic systems, the complexity level of the neuron and synapse model have a corresponding effect on the complexity of the hardware, and thus potentially affect the scale of network that can be feasibly implemented as well as the device's efficiency. In this work, we seek to determine what effect the inclusion of two biological mechanisms have on performance. We start with a basic spiking neural network model that currently has multiple hardware implementations and examine the effect of including two additional biologically-inspired mechanisms (simple potentiation/depression weight-change mechanisms on the synapses and leak on the neurons). We compare the performance of networks with and without these mechanisms on three simple applications. We found that, in general, these two mechanisms had relatively little effect on training and generalization performance, indicating that they may be able to be omitted in future neuromorphic implementations.
Catherine D. Schuman
IJCNN1
2016 An Application Development Platform for neuromorphic computing
abstract
Dynamic Adaptive Neural Network Arrays (DANNAs) are neuromorphic computing systems developed as a hardware based approach to the implementation of neural networks. They feature highly adaptive and programmable structural elements, which model artificial neural networks with spiking behavior. We design them to solve problems using evolutionary optimization. In this paper, we highlight the current hardware and software implementations of DANNA, including their features, functionalities and performance. We then describe the development of an Application Development Platform (ADP) to support efficient application implementation and testing of DANNA based solutions. We conclude with future directions.
Mark E. Dean, Christopher Daffron, Adam Disney, John Reynolds 0001, Garrett S. Rose, James S. Plank, J. Douglas Birdwell, Catherine D. Schuman
IJCNN9
2016 An evolutionary optimization framework for neural networks and neuromorphic architectures
abstract
As new neural network and neuromorphic architectures are being developed, new training methods that operate within the constraints of the new architectures are required. Evolutionary optimization (EO) is a convenient training method for new architectures. In this work, we review a spiking neural network architecture and a neuromorphic architecture, and we describe an EO training framework for these architectures. We present the results of this training framework on four classification data sets and compare those results to other neural network and neuromorphic implementations. We also discuss how this EO framework may be extended to other architectures.
Catherine D. Schuman, James S. Plank, Adam Disney, John Reynolds 0001
IJCNN1
2012 Heuristics for optimizing matrix-based erasure codes for fault-tolerant storage systems
abstract
Large scale, archival and wide-area storage systems use erasure codes to protect users from losing data due to the inevitable failures that occur. All but the most basic erasure codes employ bit-matrices so that encoding and decoding may be effected solely with the bitwise exclusive-OR (XOR) operation. There are CPU savings that can result from strategically scheduling these XOR operations so that fewer XOR's are performed. It is an open problem to derive a schedule from a bit-matrix that minimizes the number of XOR operations. We attack this open problem, deriving two new heuristics called Uber-CHRS and X-Sets to schedule encoding and decoding bit-matrices with reduced XOR operations. We evaluate these heuristics in a variety of realistic erasure coding settings and demonstrate that they are a significant improvement over previously published heuristics. We provide an open-source implementation of these heuristics so that practitioners may leverage our work.
James S. Plank, Catherine D. Schuman, B. Devin Robison
DSN2
2009 A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries for Storage
James S. Plank, Jianqiang Luo, Catherine D. Schuman, Lihao Xu, Zooko Wilcox-O'Hearn
FAST3