ReRAM-based compute-in-memory (CiM) architectures present an attractive design choice for accelerating deep neural network (DNN) training. However, these architectures are susceptible to stuck-at faults (SAFs) in ReRAM cells, which arise from manufacturing defects and cell wearout over time, particularly due to the continuous weight updates during DNN training. These faults significantly degrade accuracy and compromise dependability. To address this issue, we propose DEAR: dependable 3D architecture for robust DNN training. DEAR introduces a novel online compensation method that employs a digital compensation unit to correct SAF-induced errors dynamically during both forward and backward propagation. This approach mitigates errors induced by SAFs during both the forward and backward phases of DNN training. Additionally, DEAR leverages an HBM-based 3D memory structure to store fault-related error information efficiently. Experimental results show that DEAR limits inferencing accuracy loss to under 2% even when up to 10% of cells are faulty with uniformly distributed faults, and under 2% for up to 5% faulty cells in clustered distributions. This high fault tolerance is achieved with an area overhead of 11.5% and energy overhead of less than 6% for VGG networks and less than 12% for ResNet networks.

Hardware attacks present a new and easy way for malicious actors to compromise model parameters in machine learning (ML) enabled cyber-physical systems (CPS). This can have severe consequences for many safety-critical cyber-physical applications such as power systems, self-driving cars, healthcare, security, and so on. Prior works have proposed several pre-emptive mitigation approaches for hardware attacks that can be adopted. However, adversarial attacks can bypass existing pre-emptive attack detection methods. Existing defense setups offer no further protection once the detection is bypassed. The attacker can then cause damage without getting noticed easily. In this work, we propose a new diagnosis method to search for compromised weights in real-time even when detection is bypassed considering fault-injection attacks. The proposed methodology provides an additional level of protection, which can rapidly identify and localize more than 99% of affected weights in ML models, even when thousands of model parameters are affected simultaneously, with low power, performance, and area (PPA) overheads. In addition, we also propose a method to ensure that the CPS remains functional, even when undergoing attack diagnosis.

Graph Neural Networks (GNNs) are made up of multiple layers, with each layer comprising of different compute kernels involving weight vectors and adjacency matrices of input graph dataset. These layers exhibit varying features such as sparsity, storage requirement, and impact on predictive accuracy. Non-volatile memory (NVM)-based 3D Processing-In-Memory (PIM) architectures offer a promising approach to accelerate GNN inferencing. However, NVM device-based crossbars suffer from various non-idealities that affect the overall predictive accuracy. In this work, we consider the problem of finding a suitable mapping of GNN layers to PIM-based processing elements (PEs) in a 3D manycore architecture such that the impact of crossbar non-idealities on predictive accuracy is minimized. We develop a framework called GINA, which leverages low-cost, approximate Hessian-based methodology to automatically determine the GNN layers that are critical for accuracy and find a suitable GNN layer to PE mapping. To tackle non-idealities and to exploit sparsity at the crossbar level, a subset of the full crossbar is activated in a cycle, referred to as Operation Unit (OU). However, OU configurations vary with the above-mentioned GNN layer features, time-dependent conductance drift, and input graph dataset. GINA learns to optimize the OU configuration for unseen datasets as a function of GNN layer features and time-dependent conductance drift. Our experimental results demonstrate that GINA-enabled 3D PIM architecture reduces the latency and energy by 7.4 imes and 13 imes on an average, respectively, compared to state-of-the-art PIM architectures without compromising the predictive accuracy. Finally, we demonstrate the applicability of GINA to Convolutional Neural Networks (CNNs) and Vision Transformers.

Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architecture is an attractive solution for training Graph Neural Networks (GNNs) on edge platforms. However, the immature fabrication process and limited write endurance of ReRAMs make them prone to hardware faults, thereby limiting their widespread adoption for GNN training. Further, the existing fault-tolerant solutions prove inadequate for effectively training GNNs in the presence of faults. In this paper, we propose a fault-aware framework referred to as FARe that mitigates the effect of faults during GNN training. FARe outperforms existing approaches in terms of both accuracy and timing overhead. Experimental results demonstrate that FARe framework can restore GNN test accuracy by 47.6% on faulty ReRAM hardware with a -1 % timing overhead compared to the fault-free counterpart.

The growing use of deep learning has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. In-memory computing (IMC) architectures using non-volatile memory, such as resistive random-access memory (ReRAM), present a promising alternative. In addition to ReRAM, there are a plethora of IMC devices. Each device offers different advantages and drawbacks in terms of power, latency, area, and non-idealities. However, IMCs lack general-purpose computing capability. For instance, ReRAM crossbars are not suited for high-throughput division, which is needed for implementing normalization layers. In this paper, we present architectures that combine both (IMC and general-purpose computing) in an optimized manner to derive the best out of both worlds. The heterogeneous architectures combine the high-throughput multiplications of IMCs with the general-purpose computing ability of floating-point devices (such as CPU, GPU, etc.) to implement both training and inferencing of various AI algorithms.

ReRAM-based Compute-in-Memory (CiM) architectures offer an attractive design choice for accelerating Convolutional Neural Network (CNN) inferencing in edge computing environments. However, these architectures are susceptible to stuck-at-faults (SAFs) in ReRAM cells stemming from manufacturing defects and cell wearout over time, significantly degrading CNN inferencing accuracy. To address this challenge, we propose a technique called Selective Error Compensation for CiM (SEC-CiM). This technique strategically mitigates errors by leveraging the insight that compensating for errors in a limited number of selected columns in a crossbar is sufficient to maintain CNN inferencing accuracy. With this strategy, SEC-CiM achieves significantly lower overhead compared to previous work. Notably, it effectively addresses errors resulting from stuck-at intermediate levels, a critical aspect that was previously overlooked. We develop a theoretical framework to determine the minimum number of columns requiring error compensation. Simulation results demonstrate that SEC-CiM limits the drop in inferencing accuracy to 2% for the ResNet18 and VGG16 models, even when up to 30% of the ReRAM cells in the crossbar are faulty. Similarly, for the Densenet121 CNN, comparable accuracy results are obtained when up to 15% of the ReRAM cells are faulty. We achieve this high level of fault tolerance with moderate area and power consumption overhead of 12.2% and 10.2%, respectively.

Graph neural networks (GNNs) are becoming popular in various real-world applications. However, hardware-level security is a concern when GNN models are mapped to emerging neuromorphic computing architectures such as memristor-based crossbars. We identify a vulnerability of memristor-mapped GNNs and propose an attack mechanism based on the identified vulnerability. The proposed attack tampers memristor-mapped graph-structured data of a GNN by injecting adversarial edges to the graph and inducing slow-to-write errors in crossbars. We present a defense mechanism based on the write-verify (WV) scheme. We analyze the effectiveness of the WV-based defense and provide theoretical security guarantees. This analysis also provides guidance for selecting appropriate design parameters for the WV scheme to ensure its effectiveness in countering slow-to-write errors induced by attacks. Experimental results for the proposed attack show that there is a 5.72× increase in the success rate compared to a software-based baseline. We also demonstrate the efficacy of the WV-based defense in mitigating all slow-to-write errors induced by the proposed attack.

Processing-in-memory (PIM) architectures have emerged as an attractive computing paradigm for accelerating deep neural network (DNN) training and inferencing. However, a plethora of PIM devices, e.g., resistive random-access memory, ferroelectric field-effect transistor, phase change memory, MRAM, static random-access memory, exists and each of these devices offers advantages and drawbacks in terms of power, latency, area, and nonidealities. A heterogeneous architecture that combines the benefits of multiple devices in a single platform can enable energy-efficient and high-performance DNN training and inference. 3-D integration enables the design of such a heterogeneous architecture where multiple planar tiers consisting of different PIM devices can be integrated into a single platform. In this work, we propose the HuNT framework, which hunts for (finds) an optimal DNN neural layer mapping, and planar tier configurations for a 3-D heterogeneous architecture. Overall, our experimental results demonstrate that the HuNT-enabled 3-D heterogeneous architecture achieves up to$10 {\times }$and$3.5 {\times }$improvement with respect to the homogeneous and existing heterogeneous PIM-based architectures, respectively, in terms of energy-efficiency (TOPS/W). Similarly, the proposed HuNT-enabled architecture outperforms existing homogeneous and heterogeneous architectures by up to$8 {\times }$and$2.4\times $, respectively, in terms of compute-efficiency (TOPS/mm2) without compromising the final DNN accuracy.

Graph Neural Networks (GNNs) have achieved remarkable accuracy in cognitive tasks such as predictive analytics on graph-structured data. Hence, they have become very popular in diverse real-world applications. However, GNN training with large real-world graph datasets in edge-computing scenarios is both memory- and compute-intensive. Traditional computing platforms such as CPUs and GPUs do not provide the energy efficiency and low latency required in edge intelligence applications due to their limited memory bandwidth. Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures have been proposed as suitable candidates for accelerating AI applications at the edge, including GNN training. However, ReRAM-based PIM architectures suffer from low reliability due to their limited endurance, and low performance when they are used for GNN training in real-world scenarios with large graphs. In this work, we propose a learning-for-data-pruning framework, which leverages a trained Binary Graph Classifier (BGC) to reduce the size of the input data graph by pruning subgraphs early in the training process to accelerate the GNN training process on ReRAM-based architectures. The proposed light-weight BGC model reduces the amount of redundant information in input graph(s) to speed up the overall training process, improves the reliability of the ReRAM-based PIM accelerator, and reduces the overall training cost. This enables fast, energy-efficient, and reliable GNN training on ReRAM-based architectures. Our experimental results demonstrate that using this learning for data pruning framework, we can accelerate GNN training and improve the reliability of ReRAM-based PIM architectures by up to 1.6×, and reduce the overall training cost by 100× compared to state-of-the-art data pruning techniques.

Resistive random-access memory (ReRAM) is a promising technology for both memory and for in-memory computing. However, these devices have security vulnerabilities that are yet to be adequately investigated. In this work, we identify one such vulnerability that arises from the write mechanism in ReRAMs. Whenever a cell/row is written, a constant bias is automatically applied to the remaining cells/rows to reduce sneak current. We develop a new attack (referred as WriteHammer) that exploits this process. By repeatedly exposing a subset of cells to this bias, WriteHammer can cause noticeable resistance drift in the victim ReRAM cells. Experimental results indicate that WriteHammer can cause up to 3.5X change in cell resistance by repeatedly writing to the ReRAM cells for a duration of 4 ms.

A ReRAM crossbar-based computing system (RCS) can accelerate CNN training. However, hardware faults due to manufacturing defects and limited endurance impede the widespread adoption of RCS. We propose a dynamic task remapping-based technique for reliable CNN training on faulty RCS. Experimental results demonstrate that the proposed low-overhead method incurs only 0.85% accuracy loss on average while training popular CNNs such as VGGs, ResNets, and SqueezeNet with the CIFAR-IO, CIFAR-100, and SVHN datasets in the presence of faults.

Graph neural networks (GNNs) are becoming popular in various real-world applications. However, hardware-level security is a concern when GNN models are mapped to emerging neuromorphic technologies such as memristor-based crossbars. These security issues can lead to malfunction of memristor-mapped GNNs. We identify a vulnerability of memristor-mapped GNNs and propose an attack mechanism based on the identified vulnerability. The proposed attack tampers memristor-mapped graph-structured data of a GNN by injecting adversarial edges to the graph and inducing slow-to-write errors in crossbars. We show that 10% adversarial edge injection induces 1.11× longer write latency, eventually leading to a 44.33% error in node classification. Experimental results for the proposed attack also show that there is a 5.72× increase in the success rate compared to a software-based baseline.

Machine learning (ML) models have gained prominence in solving real-world tasks. However, implementing ML models is both compute- and memory-intensive. Domain-specific architectures such as Resistive Random Access Memory (ReRAM)-based Processing-in-Memory (PIM) platforms have been proposed to efficiently accelerate ML training and inference. However, existing ML workloads require a high amount of area and power for training. A major contributor to the area and power overheads is the Analog-to-Digital Converter (ADC). In this work, we propose a mixed pruning technique along with a novel reconfigurable ADC design to improve the power consumption profile. Overall, the pruned model with the reconfigurable ADC achieves ~50% reduction in power for training compared to existing state-of-the-art ReRAM-based architectures.

Rowhammer is a memory vulnerability that can compromise system-level security. Rowhammer occurs when a DRAM row is accessed repeatedly, potentially causing bit-flips for neighboring rows. The threshold for Rowhammer has decreased from 139K accesses in 2014 to 3.2K in 2022. This threshold is projected to decrease further. Many existing solutions are not scalable, incur high overhead, or fail to offer protection in realistic scenarios. We propose Simply-Track-And-Refresh (STAR) as an effective and scalable Rowhammer mitigation. We compare STAR's performance overhead to recent solutions, HYDRA and AQUA. At ultra-low thresholds (500), STAR introduces 9.5x/31.7x lower average execution time overhead than HYDRA/AQUA. In addition, STAR introduces up to 4.3x lower area overhead and up to 3.3x lower power consumption compared to HYDRA and AQUA. We present proof of correctness, area and power consumption results derived using CACTI, and evaluation results from the PARSEC, SPLASH-2, SPEC2006, SPEC2017, and PAMPAR benchmark suites.

Rowhammer is a security vulnerability that arises due to the undesirable electrical interaction between physically adjacent rows in DRAMs. Bit flips caused by Rowhammer can be exploited to craft many types of attacks in platforms ranging from edge devices to datacenter servers. Existing DRAM protections using error-correction codes and targeted row refresh are not adequate for defending against Rowhammer attacks. In this work, we propose a Rowhammer mitigation solution using machine learning (ML). We show that the ML-based technique can reliably detect and prevent bit flips for all the different types of Rowhammer attacks (including the recently proposed Half-double and Blacksmith attacks) considered in this work. Moreover, the ML model is associated with lower power and area overhead compared to recently proposed Rowhammer mitigation techniques, namely, Graphene and Blockhammer, for 40 different applications from the Parsec, Pampar, Splash-2, SPEC2006, and SPEC 2017 benchmark suites.

Graph neural networks (GNNs) are used for predictive analytics on graph-structured data, and they have become very popular in diverse real-world applications. Resistive random-access memory (ReRAM)-based PIM architectures can accelerate GNN training. However, GNN training on ReRAM-based architectures is both compute- and data intensive in nature. In this work, we propose a framework calledSlimGNNthat synergistically combines both graph and model pruning to accelerate GNN training on ReRAM-based architectures. The proposed framework reduces the amount of redundant information in both the GNN model and input graph(s) to streamline the overall training process. This enables fast and energy-efficient GNN training on ReRAM-based architectures. Experimental results demonstrate that using this framework, we can accelerate GNN training by up to$ {4}. {5} {\times }$while using$ {6}. {6} {\times }$less energy compared to the unpruned counterparts.

Rowhammer is a vulnerability that arises due to the undesirable interaction between physically adjacent rows in DRAMs. Existing DRAM protections are not adequate to defend against Rowhammer attacks. We propose a Rowhammer mitigation solution using machine learning (ML). We show that the ML-based technique can reliably detect and prevent bit flips for all the different types of Rowhammer attacks considered here. Moreover, the ML model is associated with lower power and area overhead compared to recently proposed Rowhammer mitigation techniques for 26 different applications from the Parsec, Pampar, and Splash-2 benchmark suites.

Resistive random-access memory has become one of the most popular choices of hardware implementation for machine learning application workloads. However, these devices exhibit non-ideal behavior, which presents a challenge towards widespread adoption. Training/inferencing on these faulty devices can lead to poor prediction accuracy. However, existing fault tolerant methods are associated with high implementation overheads. In this paper, we present some new directions for solving reliability issues using software solutions. These software-based methods are inherent in deep learning training/inferencing, and they can also be used to address hardware reliability issues as well. These methods prevent accuracy drop during training/inferencing due to unreliable ReRAMs and are associated with lower area and power overheads.

Resistive random-access memory (ReRAM)-based architectures can be used to accelerate convolutional neural network (CNN) training. However, existing architectures either do not support normalization at all or they support only a limited version of it. Moreover, it is common practice for CNNs to add normalization layers after every convolution layer. In this work, we show that while normalization layers are necessary to train deep CNNs, only a few such layers are sufficient for effective training. A large number of normalization layers do not improve prediction accuracy; it necessitates additional hardware and gives rise to performance bottlenecks. To address this problem, we proposeDeepTrain, a heterogeneous architecture enabled by a Bayesian optimization (BO) methodology; together, they provide adequate hardware and software support for normalization operations. The proposed BO methodology determines the minimum number of normalization operations necessary for a given CNN. Experimental evaluation indicates that the BO-enabledDeepTrainarchitecture achieves up to$15\times $speedup compared to a conventional GPU for training CNNs with no accuracy loss while utilizing only a few normalization layers.

Resistive random-access memory (ReRAM)-based manycore architectures enable acceleration of graph neural network (GNN) inference and training. GNNs exhibit characteristics of both DNNs and graph analytics. Hence, GNN training/inferencing on ReRAM-based manycore architectures give rise to both computation and on-chip communication challenges. In this work, we leverage model pruning and efficient graph storage to reduce the computation and communication bottlenecks associated with GNN training on ReRAM-based manycore accelerators. However, traditional pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. In this work, we propose a GNN pruning technique called DietGNN. DietGNN is a crossbar-aware pruning technique that achieves high accuracy training and enables energy, area, and storage efficient computing on ReRAM-based manycore platforms. The DietGNN pruned model can be trained from scratch without any noticeable accuracy loss. Our experimental results show that when mapped on to a ReRAM-based manycore architecture, DietGNN can reduce the number of crossbars by over 90% and accelerate GNN training by${\sim }{2}.{7}{\times }$compared to its unpruned counterpart. In addition, DietGNN reduces energy consumption by more than${\sim }{3}.{5}{\times }$compared to the unpruned counterpart.

Graph Neural Network (GNN) is a variant of Deep Neural Networks (DNNs) operating on graphs. However, GNNs are more complex compared to traditional DNNs as they simultaneously exhibit features of both DNN and graph applications. As a result, architectures specifically optimized for either DNNs or graph applications are not suited for GNN training. In this work, we propose a 3D heterogeneous manycore architecture for on-chip GNN training to address this problem. The proposed architecture, ReGraphX, involves heterogeneous ReRAM crossbars to fulfill the disparate requirements of both DNN and graph computations simultaneously. The ReRAM-based architecture is complemented with a multicast-enabled 3D NoC to improve the overall achievable performance. We demonstrate that ReGraphX outperforms conventional GPUs by up to 3.5X (on an average 3X) in terms of execution time, while reducing energy consumption by as much as 11X.

Three-dimensional (3D) integration has frequently been described as a means to overcome scaling bottlenecks, and advance both “More Moore” and “More Than Moore” through the use of vertical interconnects and die/wafer stacking. Recent industry trends show the viability of 3D integration in real products. Flash memory producers have also demonstrated multiple layers of memory on top of each other. However, conventional TSV-based 3D designs cannot achieve the full-potential of vertical integration and perform suboptimally. Monolithic 3D (M3D) is an emerging vertical integration technology that promises significant power-performance-area benefits compared to TSVs. Hence, it is important to understand the necessary design trade-offs and challenges associated with this new paradigm. In this paper, we present both the advantages and the various design challenges in M3D-enabled system design considering Processing-in-Memory (PIM) and manycore systems as suitable case-studies.

Graph Neural Networks (GNNs) are a variant of Deep Neural Networks (DNNs) operating on graphs. GNNs have attributes of both DNNs and graph computation. However, training GNNs on manycore architectures is a challenging task because it involves heavy communication that bottlenecks performance. DropEdge and Dropout, which we collectively refer to as DropLayer, are regularization techniques that can improve the predictive accuracy of GNNs. Moreover, when implemented on a manycore architecture, DropEdge and Dropout are capable of reducing the on-chip traffic. In this paper, we present a ReRAM-based 3D manycore architecture called DARe, tailored for accelerating on-chip training of GNNs. The key component of the DARe architecture is a Network-on-Chip (NoC) that reduces the amount of communication using DropLayer. The reduced traffic prevents communication hotspots and leads to better performance. We demonstrate that DARe outperforms conventional GPUs by up to 6.7X (5.6X on average) in terms of execution time, while being up to 30X (23X on average) more energy efficient for GNN training.

Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures have recently become a popular architectural choice for deep-learning applications. ReRAM-based architectures can accelerate inferencing and training of deep learning algorithms and are more energy efficient compared to traditional GPUs. However, these architectures have various limitations that affect the model accuracy and performance. Moreover, the choice of the deep-learning application also imposes new design challenges that must be addressed to achieve high performance. In this paper, we present the advantages and challenges associated with ReRAM-based PIM architectures by considering Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs) as important application domains. We also outline methods that can be used to address these challenges.

Resistive random-access memory (ReRAM) is a promising technology for designing hardware accelerators for deep neural network (DNN) inferencing. However, stochastic noise in ReRAM crossbars can degrade the DNN inferencing accuracy. We propose the design and optimization of a high-performance, area-and energy-efficient ReRAM-based hardware accelerator to achieve robust DNN inferencing in the presence of stochastic noise. We make two key technical contributions. First, we propose a stochastic-noise-aware training method, referred to as ReSNA, to improve the accuracy of DNN inferencing on ReRAM crossbars with stochastic noise. Second, we propose an information-theoretic algorithm, referred to as CF-MESMO, to identify the Pareto set of solutions to trade-off multiple objectives, including inferencing accuracy, area overhead, execution time, and energy consumption. The main challenge in this context is that executing the ReSNA method to evaluate each candidate ReRAM design is prohibitive. To address this challenge, we utilize the continuous-fidelity evaluation of ReRAM designs associated with prohibitive high computation cost by varying the number of training epochs to trade-off accuracy and cost. CF-MESMO iteratively selects the candidate ReRAM design and fidelity pair that maximizes the information gained per unit computation cost about the optimal Pareto front. Our experiments on benchmark DNNs show that the proposed algorithms efficiently uncover high-quality Pareto fronts. On average, ReSNA achieves 2.57% inferencing accuracy improvement for ResNet20 on the CIFAR-10 dataset with respect to the baseline configuration. Moreover, CF-MESMO algorithm achieves 90.91% reduction in computation cost compared to the popular multi-objective optimization algorithm NSGA-II to reach the best solution from NSGA-II.

The growing popularity of convolutional neural networks (CNNs) along with their complexity has led to the search for efficient computational platforms suitable for them. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for training CNNs. However, due to their low-precision storage capability, these architectures cannot support all types of CNN layers and suffer from accuracy loss of the learned model. In addition, ReRAM behavior varies with temperature. High temperature reduces noise margin and introduces additional noise. This makes training of CNNs challenging as outputs can be misinterpreted at higher operating temperatures leading to accuracy loss. In this work, we propose an M3D-enabled heterogeneous architecture: AccuReD, that combines ReRAM arrays with GPU cores, to address these challenges and achieve high accuracy CNN training. AccuReD supports all types of CNN layers and achieve near-GPU accuracy even with low-precision and nonideal behavior of ReRAMs. In addition, to reduce temperature, we present a performance-thermal-aware mapping policy that maps CNN layers to the computing elements of AccuReD. Experimental evaluation indicates that AccuReD does not lose accuracy while accelerating CNN training by 12× on an average compared to conventional GPU-only platforms.

The growing popularity of convolutional neural networks (CNNs) has led to the search for efficient computational platforms to accelerate CNN training. Resistive random-access memory (ReRAM)-based manycore architectures offer a promising alternative to commonly used GPU-based platforms for training CNNs. However, due to the immature fabrication process and limited write endurance, ReRAMs suffer from different types of faults. This makes training of CNNs challenging as weights are misrepresented when they are mapped to faulty ReRAM cells. This results in unstable training, leading to unacceptably low accuracy for the trained model. Due to the distributed nature of the mapping of the individual bits of a weight to different ReRAM cells, faulty weights often lead to exploding gradients. This in turn introduces a positive feedback in the training loop, resulting in extremely large and unstable weights. In this paper, we propose a lightweight and reliable CNN training methodology using weight clipping to prevent this phenomenon and enable training even in the presence of many faults. Weight clipping prevents large weights from destabilizing CNN training and provides the backpropagation algorithm with the opportunity to compensate for the weights mapped to faulty cells. The proposed methodology achieves near-GPU accuracy without introducing significant area or performance overheads. Experimental evaluation indicates that weight clipping enables the successful training of CNNs in the presence of faults, while also reducing training time by 4 X on average compared to a conventional GPU platform. Moreover, we also demonstrate that weight clipping outperforms a recently proposed error correction code (ECC)-based method when training is carried out using faulty ReRAMs.

Heterogeneous manycore architectures are the key to efficiently execute compute- and data-intensive applications. Through-silicon-via (TSV)-based 3D manycore system is a promising solution in this direction as it enables the integration of disparate computing cores on a single system. Recent industry trends show the viability of 3D integration in real products (e.g., Intel Lakefield SoC Architecture, the AMD Radeon R9 Fury X graphics card, and Xilinx Virtex-7 2000T/H580T, etc.). However, the achievable performance of conventional TSV-based 3D systems is ultimately bottlenecked by the horizontal wires (wires in each planar die). Moreover, current TSV 3D architectures suffer from thermal limitations. Hence, TSV-based architectures do not realize the full potential of 3D integration. Monolithic 3D (M3D) integration, a breakthrough technology to achieve “More Moore and More Than Moore,” opens up the possibility of designing cores and associated network routers using multiple layers by utilizing monolithic inter-tier vias (MIVs) and hence, reducing the effective wire length. Compared to TSV-based 3D integrated circuits (ICs), M3D offers the “true” benefits of vertical dimension for system integration: the size of an MIV used in M3D is over 100 × smaller than a TSV. This dramatic reduction in via size and the resulting increase in density opens up numerous opportunities for design optimizations in 3D manycore systems: designers can use up to millions of MIVs for ultra-fine-grained 3D optimization, where individual cores and routers can be spread across multiple tiers for extreme power and performance optimization. In this work, we demonstrate how M3D-enabled vertical core and uncore elements offer significant performance and thermal improvements in manycore heterogeneous architectures compared to its TSV-based counterpart. To overcome the difficult optimization challenges due to the large design space and complex interactions among the heterogeneous components (CPU, GPU, Last Level Cache, etc.) in a M3D-based manycore chip, we leverage novel design-space exploration algorithms to trade off different objectives. The proposed M3D-enabled heterogeneous architecture, called HeM3D , outperforms its state-of-the-art TSV-equivalent counterpart by up to 18.3% in execution time while being up to 19°C cooler.

Graph neural network (GNN) is a variant of deep neural networks (DNNs) operating on graphs. However, GNNs are more complex compared with DNNs as they simultaneously exhibit attributes of both DNN and graph computations. In this work, we propose a ReRAM-based 3-D manycore processing-in-memory architecture called ReMaGN, tailored for on-chip training of GNNs. ReMaGN implements GNN training using reduced-precision representation to make the computation faster and reduce the load on the communication backbone. However, reduced precision can potentially compromise the accuracy of training. Hence, we undertake a study of performance and accuracy tradeoffs in such architectures. We demonstrate that ReMaGN outperforms conventional GPUs by up to$9.5\times $(on average$7.1\times $) in terms of execution time, while being up to$42\times $(on average$33.5\times $) more energy efficient without sacrificing accuracy.

Deep Neural Networks (DNNs) employed for image segmentation are computationally more expensive and complex compared to the ones used for classification. However, manycore architectures to accelerate the training of these DNNs are relatively unexplored. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for training DNNs. However, due to their low-precision storage capability, these architectures cannot support all DNN layers and suffer from accuracy loss of the learned models. To address these challenges, we propose GRAMARCH, a heterogeneous architecture that combines the benefits of ReRAM and GPUs simultaneously by using a high-throughput 3D Network-on-Chip. Experimental results indicate that by suitably mapping DNN layers to processing elements, it is possible to achieve up to 53X better performance compared to conventional GPUs for image segmentation.

Due to the growing needs of Big Data applications (e.g., deep learning, graph analytics, and scientific computing) and the ending of Moore's law, there is a great need for low-cost, high-performance, energy-efficient, and small form-factor manycore systems. With more stringent design objectives, application specialization, and more cores on a single chip, design-time optimization becomes more complex. Moreover, with the advent of emerging interconnect technologies like 3D integration, the design optimization process has become more challenging. This increases the need for a holistically optimized design process that makes design decisions across multiple layers of the system, e.g., memory, compute, interconnect technology, and network infrastructure. In this paper, we present various challenges of designing heterogeneous manycore architectures using 3D integration and viable optimization techniques to solve them.

The growing popularity of Convolutional Neural Networks (CNNs) has led to the search for efficient computational platforms to enable these algorithms. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for CNN training. However, backpropagation in CNNs is susceptible to the limited precision of ReRAMs. As a result, training CNNs on ReRAMs affects the final accuracy of learned model. In this work, we propose REGENT, a heterogeneous architecture that combines ReRAM arrays with GPU cores, and exploits the benefits provided by 3D integration along with a high-throughput yet energy efficient Network-on-Chip (NoC) for training CNNs. We also propose a bin-packing based framework that maps CNN layers and then optimize the placement of computing elements to meet the targeted design objectives. Experimental evaluations indicate that REGENT improves full-system EDP by 55.7% on average compared to conventional GPU-only platforms for training CNNs.

Counting k-mers (substrings of fixed length k) in DNA and protein sequences generate non-uniform and irregular memory access patterns. Processing-in-Memory (PIM) architectures have the potential to significantly reduce the overheads associated with such frequent and irregular memory accesses. However, existing k-mer counting algorithms are not designed to exploit the advantages of PIM architectures. Furthermore, owing to thermal constraints, the allowable power budget is limited in conventional PIM designs. Moreover, k-mer counting generates unbalanced and long-range traffic patterns that need to be handled by an efficient Network-on-Chip (NoC). In this paper, we present an NoC-enabled software/hardware co-design framework to implement high-performance k-mer counting. The proposed architecture enables more computational power, efficient communication between cores/memory - all without creating a thermal bottleneck; while the software component exposes more in-memory opportunities to exploit the PIM and aids in the NoC design. Experimental results show that the proposed architecture outperforms a state-of-the-art software implementation of k-mer counting utilizing Hybrid Memory Cube (HMC), by up to 7.14X, while allowing significantly higher power budgets.

The rising use of deep learning and other big-data algorithms has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. Due to the amount of data parallelism in these algorithms, high-performance three-dimensional (3D) manycore platforms that incorporate both CPUs and GPUs present a promising direction. However, as systems use heterogeneity (e.g., a combination of CPUs, GPUs, and accelerators) to improve performance and efficiency, it becomes more pertinent to address the distinct and likely conflicting communication requirements (e.g., CPU memory access latency or GPU network throughput) that arise from such heterogeneity. Unfortunately, it is difficult to quickly explore the hardware design space and choose appropriate tradeoffs between these heterogeneous requirements. To address these challenges, we propose the design of a 3D Network-on-Chip (NoC) for heterogeneous manycore platforms that considers the appropriate design objectives for a 3D heterogeneous system and explores various tradeoffs using an efficient machine learning (ML)-based multi-objective optimization (MOO) technique. The proposed design space exploration considers the various requirements of its heterogeneous components and generates a set of 3D NoC architectures that efficiently trades off these design objectives. Our findings show that by jointly considering these requirements (latency, throughput, temperature, and energy), we can achieve 9.6 percent better Energy-Delay Product on average at nearly iso-temperature conditions when compared to a thermally-optimized design for 3D heterogeneous NoCs. More importantly, our results suggest that our 3D NoCs optimized for a few applications can be generalized for unknown applications as well. Our results show that these generalized 3D NoCs only incur a 1.8 percent (36-tile system) and 1.1 percent (64-tile system) average performance loss compared to application-specific NoCs.

The growing needs of emerging applications has posed significant challenges for the design of optimized manycore systems. Network-on-Chip (NoC) enables the integration of a large number of processing elements (PEs) in a single die. To design optimized manycore systems, we need to establish suitable trade-offs among multiple objectives including power, performance, and thermal. Therefore, we consider multi-objective design space exploration (MO-DSE) problems arising in the design of NoC-enabled manycore systems: placement of PEs and communication links to optimize two or more objectives (e.g., latency, energy, and throughput). Existing algorithms to solve MO-DSE problems suffer from scalability and accuracy challenges as size of the design space and the number of objectives grow. In this paper, we propose a novel framework referred as Multi-Objective Optimistic Search (MOOS) that performs adaptive design space exploration using a data-driven model to improve the speed and accuracy of multi-objective design optimization process. We apply MOOS to design both 3D heterogeneous and homogeneous manycore systems using Rodinia, PARSEC, and SPLASH2 benchmark suites. We demonstrate that MOOS improves the speed of finding solutions compared to state-of-the-art methods by up to 13X while uncovering designs that are up to 20% better in terms of NoC. The optimized 3D manycore systems improve the EDP up to 38% when compared to 3D mesh-based designs optimized for the placement of PEs.

3D Network-on-Chip (NoC) architectures are capable of achieving better performance and lower energy consumption compared to their planar counterparts. However, conventional 3D NoCs are not efficient in handling collective communication. Existing works mainly explore Path and Tree multicast distribution schemes for 3D NoCs. However, both these mechanisms involve high network latency and lack scalability. In this work, we propose a SMART (Single-cycle Multi-hop Asynchronous Repeated Traversal) 3D NoC architecture that is capable of achieving high-performance collective communication. The proposed High-Performance SMART (HP-SMART) 3D NoC achieves 65% and 31% latency improvements compared to the existing Path and Tree multicast-based 3D NoCs respectively. HP-SMART 3D NoC also achieves significant improvement in message latency compared to its 2D counterpart.

The widespread adoption of big data has led to the search for highperformance and low-power computational platforms. Emerging heterogeneous manycore processing platforms consisting of CPU and GPU cores along with various types of accelerators offer power and area-efficient trade-offs for running these applications. However, heterogeneous manycore architectures need to satisfy the communication and memory requirements of the diverse computing elements that conventional Network-on-Chip (NoC) architectures are unable to handle effectively. Further, with increasing system sizes and level of heterogeneity, it becomes difficult to quickly explore the large design space and establish the appropriate design trade-offs. To address these challenges, machine learning-inspired heterogeneous manycore system design is a promising research direction to pursue. In this paper, we highlight various salient features of heterogeneous manycore architectures enabled by emerging interconnect technologies and machine learning techniques.

As deep learning technology is increasingly employed in diverse applications domains, the demand for computational power to enable these algorithms also increases. In this respect, high-performance three-dimensional (3D) heterogeneous manycore systems present a promising direction. However, deep learning on these systems pose several design challenges. First, the network-on-chip (NoC) must handle the traffic requirements of both CPU and GPU communications. Second, 3D system designs must address thermal issues resulting from high-power density. In this work, we propose a design methodology for a heterogeneous 3D NoC architecture that not only satisfies the traffic requirements of both CPUs and GPUs, but also reduces thermal hotspots. To this end, we target the training of two widely employed convolutional neural networks (CNN), namely, LeNet and CIFAR. By using our joint performance-thermal optimization methodology to create a 3D NoC for training CNNs, we reduce the maximum temperature by 22% while incurring only 5% full-system energy-delay-product degradation over a solely performance optimized 3D NoC. This demonstrates that, our design methodology achieves considerable temperature reduction with negligible loss in performance.