Effective acceleration of Generative AI workloads is constrained by heterogeneous computational and memory characteristics. Large Language Models (LLMs), for example, are composed of diverse kernels, ranging from memory-bound attention mechanisms to compute-intensive feed-forward networks. Such variability leads to poor resource utilization and renders homogeneous architectures inefficient. This paper outlines a methodology to address this challenge using two synergistic approaches. First, we discuss the design of a three-dimensional (3D) manycore architecture enabled by heterogeneous integration. Specifically, the architecture leverages emerging non-volatile memory alongside CMOS-based computing cores to create an optimized kernel-to-compute mapping. Second, we explore a hardware and software co-design framework to enable fine-tuning of unimodal and multimodal LLMs. This framework ensures thermally robust inference when leveraging emerging non-volatile memories within the heterogeneous architecture. Experimental results on multiple LLMs demonstrate the efficacy of the 3D heterogeneous architecture and the co-design framework.

Traditional digital implementations of neural accelerators are limited by high power consumption and area overheads, while analog and non-CMOS implementations suffer from noise, device mismatch, and reliability issues. This paper introduces a CMOS Look-Up Table (LUT)-based Architecture for Neural Accelerators (LANA) that reduces the power consumption and area overhead of traditional digital implementations through precomputed, faster LUT access while avoiding noise and mismatch challenges of analog circuits. To solve the scalability issues of conventional LUT-based computation, we split high-precision multiply and accumulate (MAC) operations into lower-precision MACs using a divide-and-conquer (D&C) based approach. LANA achieves up to 29.54× lower area with 3.34× lower energy per inference task compared to traditional LUT-based techniques and up to 1.24× lower area with 1.80× lower energy per inference task than conventional digital MAC-based techniques (Wallace Tree/Array Multipliers) without retraining and without affecting the accuracy of pre-trained unpruned models, as well as on Lottery Ticket Pruned (LTP) models that already reduce the number of required MAC operations by up to 98%. Finally, we introduce mixed precision analysis in the LANA framework for all LTP pruned and unpruned models (VGG11, VGG19, Resnet18, Resnet34, GoogleNet) that achieved up to 29.59× (GoogleNet pruned)-62.83× (VGG11 unpruned) lower area across models with 3.34× (GoogleNet pruned)-8.1× (VGG11 unpruned) lower energy per inference than traditional LUT-based techniques, and up to 1.24× (GoogleNet pruned)-2.63× (VGG11 unpruned) lower area requirement with 1.81× (GoogleNet pruned)-4.37× (VGG11 unpruned) lower energy per inference across models as compared to conventional digital MAC-based techniques with 1% accuracy loss relative to the baseline.

Rapidly evolving artificial intelligence and machine learning applications require ever-increasing computational capabilities, while monolithic 2D design technologies approach their limits. 2.5D/3D heterogeneous integration of smaller chiplets using advanced packaging has emerged as a promising paradigm for addressing this limit and meeting performance demands. These approaches offer a significant cost reduction and higher manufacturing yield than monolithic 2D integrated circuits. However, the compact arrangement and high compute density of these systems exacerbate thermal management challenges, potentially compromising performance. Addressing these thermal modeling challenges is critical, especially as system sizes grow and different design stages require varying levels of accuracy and speed. Since no single thermal modeling technique meets all these needs, this article introduces MFIT, a range of multi-fidelity thermal models that effectively balance accuracy and speed. These multi-fidelity models can enable efficient design space exploration and runtime thermal management. Our extensive testing on systems with 16, 36, and 64 2.5D integrated chiplets and 16×3 3D integrated chiplets demonstrates that these models can reduce execution times from days to mere seconds and milliseconds with negligible loss in accuracy.

Processing-In-Memory (PIM) has emerged as a promising technology for accelerating machine learning (ML) workloads. Specifically, non-volatile memory-based PIM architectures have enabled effective ML acceleration due to their ability to perform energy-efficient matrix-vector multiplication operations. However, these devices suffer from non-idealities such as thermal noise. This noise alters the stored values in the memory cells which correspond to actual model weights, compromising the inference accuracy. In this work, we introduce ThRIve, a noise-aware training methodology that leverages low-rank adaptation to enable thermally robust inference on heterogeneous PIM architectures. ThRIve selectively stores these low-rank noise-aware parameters on a hardware that is less susceptible to thermal noise, enabling robustness against temperature-induced noise variations. ThRIve mitigates the effects of thermal-noise and prevent the drop in inference accuracy across the entire operating temperature range. Experimental results demonstrate that ThRIve-enabled architectures maintain consistent inference accuracy, with the mean accuracy staying within 2% of the ideal (i.e., noise-free) accuracy, and the variation in accuracy across the entire operating temperature range remaining within 2% of the mean. The proposed methodology achieves accuracy and robustness comparable to thermally-resilient Static Random-Access Memory (SRAM)-based PIM systems, while delivering up to 5.4 × reduction in energy-delay product (EDP) during CNN model inferencing.

ReRAM-based compute-in-memory (CiM) architectures present an attractive design choice for accelerating deep neural network (DNN) training. However, these architectures are susceptible to stuck-at faults (SAFs) in ReRAM cells, which arise from manufacturing defects and cell wearout over time, particularly due to the continuous weight updates during DNN training. These faults significantly degrade accuracy and compromise dependability. To address this issue, we propose DEAR: dependable 3D architecture for robust DNN training. DEAR introduces a novel online compensation method that employs a digital compensation unit to correct SAF-induced errors dynamically during both forward and backward propagation. This approach mitigates errors induced by SAFs during both the forward and backward phases of DNN training. Additionally, DEAR leverages an HBM-based 3D memory structure to store fault-related error information efficiently. Experimental results show that DEAR limits inferencing accuracy loss to under 2% even when up to 10% of cells are faulty with uniformly distributed faults, and under 2% for up to 5% faulty cells in clustered distributions. This high fault tolerance is achieved with an area overhead of 11.5% and energy overhead of less than 6% for VGG networks and less than 12% for ResNet networks.

ReRAM-based Processing-In-Memory (PIM) architectures enable energy-efficient Deep Neural Network (DNN) inferencing. However, ReRAM crossbars suffer from various non-idealities that affect the overall inferencing accuracy. To address that, the matrix-vector-multiplication (MVM) operations are computed by activating a subset of the full crossbar, referred to as Operation Unit (OU). However, OU configurations vary with the neural layers' features such as sparsity, kernel size, and their impact on predictive accuracy. In this paper, we consider the problem of learning appropriate layer-wise OU configurations in ReRAM crossbars for unseen DNNs at runtime such that the performance is maximized without loss in predictive accuracy. We develop a machine learning (ML) based framework called Odin, which selects the OU sizes for different neural layers as a function of the neural layer features and time-dependent ReRAM conductance drift. Our experimental results demonstrate that the energy-delay-product (EDP) is reduced by up to 8.7× over state-of-the-art homogeneous OU configurations without compromising predictive accuracy.

Transformer architectures have become the standard neural network model for various machine learning (ML) applications, including natural language processing and computer vision. However, the compute and memory requirements introduced by transformer models make them challenging to adopt for edge applications. Furthermore, fine-tuning pretrained transformers (e.g., foundation models) is a common task to enhance the model’s predictive performance on specific tasks/applications. Existing transformer accelerators are oblivious to complexities introduced by fine-tuning. In this article, we propose the design of a three-dimensional (3D) heterogeneous architecture referred to as Atleus that incorporates heterogeneous computing resources specifically optimized to accelerate transformer models for the dual purposes of fine-tuning and inference. Specifically, Atleus utilizes nonvolatile memory and systolic array for accelerating transformer computational kernels using an integrated 3D platform. Moreover, we design a suitable NoC to achieve high performance and energy efficiency. Finally, Atleus adopts an effective quantization scheme to support model compression. Experimental results demonstrate that Atleus outperforms existing state-of-the-art by up to$56\times $and$64.5\times $in terms of performance and energy efficiency, respectively.

Operation unit (OU)-based configurations enable the design of energy-efficient and reliable ReRAM crossbar-based processing-in-memory (PIM) architectures for deep neural network (DNN) inferencing. To exploit sparsity and tackle crossbar nonidealities, matrix-vector-multiplication (MVM) operations are computed at a much smaller level of granularity than a full crossbar, referred to as OUs. However, determining the suitable OU size for a given DNN workload presents a nontrivial challenge as the DNN layers exhibit different levels of sparsity and have varying impact on overall predictive accuracy. In this article, we propose a framework for designing heterogeneous OU-based PIM accelerators. The OU configurations vary based on the characteristics of the neural layers and the time-dependent conductance drift of PIM devices due to repeated inference runs. Overall, our experimental results demonstrate that the sparsity-aware layer-wise heterogeneous OU-based PIM computation reduces latency and energy by 34% and 73% on average, respectively, compared to state-of-the-art homogeneous OU-based architectures without compromising the predictive accuracy.

Processing-in-memory (PIM) has been proposed as an alternative computing paradigm for training Deep Neural Networks, including Graph Neural Networks (GNNs). Despite these advancements, training GNN workloads on PIM devices necessitates off-chip memory access. This off-chip access is expensive in terms of performance and energy, thereby impacting the overall energy efficiency of the training process on PIM platforms. In this article, we propose a novel hybrid training framework called ERGo that automatically switches from a full-parameter phase to a parameter-efficient phase during the training process with negligible loss in predictive accuracy. The parameter-efficient phase employs low-rank representations of GNN weights, effectively reducing the number of trainable parameters. This helps in reducing the off-chip access during the end-to-end training process on PIM architectures, leading to notable improvements in both performance and energy efficiency. ERGo outperforms the conventional full-parameter GNN training on a PIM-based platform by up to 3.15x in speedup and enhances energy efficiency by 10.5x. Furthermore, ERGo offers additional advantages when utilized on heterogeneous architectures incorporating non-volatile memory (NVM) and static random-access memory (SRAM). Training on the heterogeneous NVM-SRAM architecture typically utilizes NVM for the forward-pass and SRAM for the backward-pass. However, NVM cells suffer from low device endurance and repeated write operations due to weight updates can lead to poor lifetime. ERGo improves the lifetime of NVM devices by freezing the weights and preventing weight updates in the parameter-efficient phase. Experimental results demonstrate that ERGo improves the lifetime of heterogeneous NVM-SRAM architecture by up to 33x in comparison to full-parameter implementation.

Chiplet-based integration enables large-scale systems that combine diverse technologies, enabling higher yield, lower costs, and scalability, making them well-suited to AI workloads. Processing-in-Memory (PIM) has emerged as a promising solution for AI inference, leveraging technologies such as ReRAM, SRAM, and FeFET, each offering unique advantages and tradeoffs. A heterogeneous chiplet-based PIM architecture can harness the complementary strengths of these technologies to enable higher performance and energy efficiency. However, scheduling AI workloads across such a heterogeneous system is challenging due to competing performance objectives, dynamic workload characteristics, and power and thermal constraints. To address this need, we propose THERMOS, a thermally-aware, multi-objective scheduling framework for AI workloads on heterogeneous multi-chiplet PIM architectures. THERMOS trains a single multi-objective reinforcement learning (MORL) policy that is capable of achieving Pareto-optimal execution time, energy, or a balanced objective at runtime, depending on the target preferences. Comprehensive evaluations show that THERMOS achieves up to 89% faster average execution time and 57% lower average energy consumption than baseline AI workload scheduling algorithms with only 0.14% runtime and 0.022% energy overhead.

Graph Neural Networks (GNNs) are made up of multiple layers, with each layer comprising of different compute kernels involving weight vectors and adjacency matrices of input graph dataset. These layers exhibit varying features such as sparsity, storage requirement, and impact on predictive accuracy. Non-volatile memory (NVM)-based 3D Processing-In-Memory (PIM) architectures offer a promising approach to accelerate GNN inferencing. However, NVM device-based crossbars suffer from various non-idealities that affect the overall predictive accuracy. In this work, we consider the problem of finding a suitable mapping of GNN layers to PIM-based processing elements (PEs) in a 3D manycore architecture such that the impact of crossbar non-idealities on predictive accuracy is minimized. We develop a framework called GINA, which leverages low-cost, approximate Hessian-based methodology to automatically determine the GNN layers that are critical for accuracy and find a suitable GNN layer to PE mapping. To tackle non-idealities and to exploit sparsity at the crossbar level, a subset of the full crossbar is activated in a cycle, referred to as Operation Unit (OU). However, OU configurations vary with the above-mentioned GNN layer features, time-dependent conductance drift, and input graph dataset. GINA learns to optimize the OU configuration for unseen datasets as a function of GNN layer features and time-dependent conductance drift. Our experimental results demonstrate that GINA-enabled 3D PIM architecture reduces the latency and energy by 7.4 imes and 13 imes on an average, respectively, compared to state-of-the-art PIM architectures without compromising the predictive accuracy. Finally, we demonstrate the applicability of GINA to Convolutional Neural Networks (CNNs) and Vision Transformers.

Multi-chiplet architectures enabled by glass interposer offer superior electrical performance, enable higher bus widths due to reduced crosstalk, and have lower capacitance in the redistribution layer than current silicon interposer-based systems. These advantages result in lower energy per bit, higher communication frequencies, and extended interconnect range. However, deformation of the package (warpage) in glass interposer-based systems becomes a critical challenge as system size increases, leading to severe mechanical stress and reliability concerns. Beyond a certain size, conventional packaging techniques fail to manage warpage effectively, necessitating new approaches to mitigate warpage induced bending with scalable performance for glass interposer based multi-chiplet systems. To address these inter-twined challenges, we propose a thermal-, warpage-, and performance-aware design framework that employs architecture and packaging co-optimization . The proposed framework disintegrates the surface and embedded chiplets to balance conflicting design objectives, ensuring optimal tradeoffs between performance, power, and structural reliability. Our experiments demonstrate that optimized multi-chiplet architectures from our design framework achieve up to 64.7% performance improvement and 40% power reduction compared to traditional 2.5D systems to execute deep neural network workloads with lower fabrication costs.

Transformers have revolutionized deep learning and generative modeling, enabling advancements in natural language processing tasks. However, the size of transformer models is increasing continuously, driven by enhanced capabilities across various deep learning tasks. This trend of ever-increasing model size has given rise to new challenges in terms of memory and compute requirements. Conventional computing platforms, including GPUs, suffer from suboptimal performance due to the memory demands imposed by models with millions/billions of parameters. The emerging chiplet-based platforms provide a new avenue for compute- and data-intensive machine learning applications enabled by a Network-on-Interposer (NoI). However, designing suitable hardware accelerators for executing Transformer inference workloads is challenging due to a wide variety of complex computing kernels in the Transformer architecture. In this article, we leverage chiplet-based heterogeneous integration to design a high-performance and energy-efficient multichiplet platform to accelerate transformer workloads. We demonstrate that the proposed NoI architecture caters to the data access patterns inherent in a transformer model. The optimized placement of the chiplets and the associated NoI links and routers enable superior performance compared to the state-of-the-art hardware accelerators. The proposed NoI-based architecture demonstrates scalability across varying transformer models and improves latency and energy efficiency by up to 11.8× and 2.36×, respectively when compared with the existing state-of-the-art architecture HAIMA.

Transformer models have become widely popular in numerous applications, and especially for building foundation large language models (LLMs). Recently, there has been a surge in the exploration of transformer-based architectures in non-LLM applications. In particular, the self-attention mechanism within the transformer architecture offers a way to exploit any hidden relations within data, making it widely applicable for a variety of spatio-temporal tasks in scientific computing domains (e.g., weather, traffic, agriculture). Most of these efforts have primarily focused on accelerating the inference phase. However, the computational resources required to train these attention-based models for scientific applications remain a significant challenge to address. Emerging non-volatile memory (NVM)-based processing-in-memory (PIM) architectures can achieve higher performance and better energy efficiency than their GPU-based counterparts. However, the frequent weight updates during training would necessitate write operations to NVM cells, posing a significant barrier for considering stand-alone NVM-based PIM architectures. In this paper, we presentHpT, a new hybrid approach to accelerate the training of attention-based models for scientific applications. Our approach is hybrid at two different layers: at the software layer, our approach dynamically switches from a full-parameter training mode to a lower-parameter training mode by incorporating intrinsic dimensionality; and at the hardware layer, our approach harnesses the combined power of GPUs, resistive random-access memory (ReRAM)-based PIM devices, and systolic arrays. This software-hardware co-design approach is aimed at adaptively reducing both runtime and energy costs during the training phase, without compromising on quality. Experiments on four concrete real-world scientific applications demonstrate that our hybrid approach is able to significantly reduce training time (up to$11.9\times$) and energy consumption (up to$12.05\times$), compared to the corresponding full-parameter training executing on only GPUs. Our approach serves as an example for accelerating the training of attention-based models on heterogeneous platforms including ReRAMs.

Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architecture is an attractive solution for training Graph Neural Networks (GNNs) on edge platforms. However, the immature fabrication process and limited write endurance of ReRAMs make them prone to hardware faults, thereby limiting their widespread adoption for GNN training. Further, the existing fault-tolerant solutions prove inadequate for effectively training GNNs in the presence of faults. In this paper, we propose a fault-aware framework referred to as FARe that mitigates the effect of faults during GNN training. FARe outperforms existing approaches in terms of both accuracy and timing overhead. Experimental results demonstrate that FARe framework can restore GNN test accuracy by 47.6% on faulty ReRAM hardware with a -1 % timing overhead compared to the fault-free counterpart.

Processing-in-memory (PIM) has emerged as an enabler for the energy-efficient and high-performance acceleration of deep learning (DL) workloads. Resistive random-access memory (ReRAM) is one of the most promising technologies to implement PIM. However, as the complexity of Deep convolutional neural networks (DNNs) grows, we need to design a manycore architecture with multiple ReRAM-based processing elements (PEs) on a single chip. Existing PIM-based architectures mostly focus on computation while ignoring the role of communication. ReRAM-based tiled manycore architectures often involve many Processing Elements (PEs), which need to be interconnected via an efficient on-chip communication infrastructure. Simply allocating more resources (ReRAMs) to speed up only computation is ineffective if the communication infrastructure cannot keep up with it. In this paper, we highlight the design principles of a dataflow-aware PIM-enabled manycore platform tailor-made for various types of DL workloads. We consider the design challenges with both 2.5D interposer- and 3D integration-enabled architectures.

The growing use of deep learning has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. In-memory computing (IMC) architectures using non-volatile memory, such as resistive random-access memory (ReRAM), present a promising alternative. In addition to ReRAM, there are a plethora of IMC devices. Each device offers different advantages and drawbacks in terms of power, latency, area, and non-idealities. However, IMCs lack general-purpose computing capability. For instance, ReRAM crossbars are not suited for high-throughput division, which is needed for implementing normalization layers. In this paper, we present architectures that combine both (IMC and general-purpose computing) in an optimized manner to derive the best out of both worlds. The heterogeneous architectures combine the high-throughput multiplications of IMCs with the general-purpose computing ability of floating-point devices (such as CPU, GPU, etc.) to implement both training and inferencing of various AI algorithms.

Transformers have revolutionized deep learning and generative modeling to enable unprecedented advancements in natural language processing tasks and beyond. However, designing hardware accelerators for executing transformer models is challenging due to the wide variety of computing kernels involved in the transformer architecture. Existing accelerators are either inadequate to accelerate end-to-end transformer models or suffer notable thermal limitations. In this paper, we propose the design of a three-dimensional heterogeneous architecture referred to as HeTraX specifically optimized to accelerate end-to-end transformer models. HeTraX employs hardware resources aligned with the computational kernels of transformers and optimizes both performance and energy. Experimental results show that HeTraX outperforms existing state-of-the-art by up to 5.6x in speedup and improves EDP by 14.5x while ensuring thermally feasibility.

ReRAM-based Compute-in-Memory (CiM) architectures offer an attractive design choice for accelerating Convolutional Neural Network (CNN) inferencing in edge computing environments. However, these architectures are susceptible to stuck-at-faults (SAFs) in ReRAM cells stemming from manufacturing defects and cell wearout over time, significantly degrading CNN inferencing accuracy. To address this challenge, we propose a technique called Selective Error Compensation for CiM (SEC-CiM). This technique strategically mitigates errors by leveraging the insight that compensating for errors in a limited number of selected columns in a crossbar is sufficient to maintain CNN inferencing accuracy. With this strategy, SEC-CiM achieves significantly lower overhead compared to previous work. Notably, it effectively addresses errors resulting from stuck-at intermediate levels, a critical aspect that was previously overlooked. We develop a theoretical framework to determine the minimum number of columns requiring error compensation. Simulation results demonstrate that SEC-CiM limits the drop in inferencing accuracy to 2% for the ResNet18 and VGG16 models, even when up to 30% of the ReRAM cells in the crossbar are faulty. Similarly, for the Densenet121 CNN, comparable accuracy results are obtained when up to 15% of the ReRAM cells are faulty. We achieve this high level of fault tolerance with moderate area and power consumption overhead of 12.2% and 10.2%, respectively.

Heterogeneous integration using 2.5D chiplet platforms provides a new avenue for compact scale-out implementations of emerging applications, such as deep learning (DL). Integrating multiple small chiplets using a Network-on-Interposer (NoI) offers not only significant cost reductions and higher manufacturing yield compared to 2D ICs but also better thermal efficiency than 3D ICs and easier heterogeneous integration. However, dense integration and substantial compute density exacerbate thermal design problems, threatening to undermine the potential performance and cost benefits. Due to the significant role of temperature in the operation and reliability of integrated systems, it is critical to understand the role of heat in this emerging design area. However, little work has considered the thermal consequences of closely packaging a large number of computational elements. This paper overviews the thermal modeling challenges for chiplet-based 2.5D platforms, overviews existing approaches, and discusses the opportunities enabled by fast and accurate thermal models.

Graph neural networks (GNNs) are becoming popular in various real-world applications. However, hardware-level security is a concern when GNN models are mapped to emerging neuromorphic computing architectures such as memristor-based crossbars. We identify a vulnerability of memristor-mapped GNNs and propose an attack mechanism based on the identified vulnerability. The proposed attack tampers memristor-mapped graph-structured data of a GNN by injecting adversarial edges to the graph and inducing slow-to-write errors in crossbars. We present a defense mechanism based on the write-verify (WV) scheme. We analyze the effectiveness of the WV-based defense and provide theoretical security guarantees. This analysis also provides guidance for selecting appropriate design parameters for the WV scheme to ensure its effectiveness in countering slow-to-write errors induced by attacks. Experimental results for the proposed attack show that there is a 5.72× increase in the success rate compared to a software-based baseline. We also demonstrate the efficacy of the WV-based defense in mitigating all slow-to-write errors induced by the proposed attack.

Processing-in-memory (PIM) architectures have emerged as an attractive computing paradigm for accelerating deep neural network (DNN) training and inferencing. However, a plethora of PIM devices, e.g., resistive random-access memory, ferroelectric field-effect transistor, phase change memory, MRAM, static random-access memory, exists and each of these devices offers advantages and drawbacks in terms of power, latency, area, and nonidealities. A heterogeneous architecture that combines the benefits of multiple devices in a single platform can enable energy-efficient and high-performance DNN training and inference. 3-D integration enables the design of such a heterogeneous architecture where multiple planar tiers consisting of different PIM devices can be integrated into a single platform. In this work, we propose the HuNT framework, which hunts for (finds) an optimal DNN neural layer mapping, and planar tier configurations for a 3-D heterogeneous architecture. Overall, our experimental results demonstrate that the HuNT-enabled 3-D heterogeneous architecture achieves up to$10 {\times }$and$3.5 {\times }$improvement with respect to the homogeneous and existing heterogeneous PIM-based architectures, respectively, in terms of energy-efficiency (TOPS/W). Similarly, the proposed HuNT-enabled architecture outperforms existing homogeneous and heterogeneous architectures by up to$8 {\times }$and$2.4\times $, respectively, in terms of compute-efficiency (TOPS/mm2) without compromising the final DNN accuracy.

The emergence of 2.5D chiplet platforms provides a new avenue for compact scale-out implementations of deep learning (DL) workloads (WLs). Integrating multiple small chiplets using a network-on-interposer (NoI) offers not only significant cost reduction and higher manufacturing yield than 2-D ICs but also better energy efficiency and performance. However, defects in chiplets may compromise performance since they restrict the computing capability. Therefore, carefully designed chiplet and NoI link placement, and task mapping schemes, in presence of defects, are necessary. In this article, we propose a defect-aware NoI design approach using a custom-defined space-filling curve (SFC) for efficient execution of mixed WLs of convolutional neural network (CNN) inference tasks. We demonstrate that the k-ary n-cube-based NoI topologies can be degenerated into SFC-based counterparts, which we refer to as SFCed NoI topologies. They enable high performance and energy efficiency with lower fabrication costs over their parent k-ary n-cube counterparts. The SFCed approach helps us to extract high performance from an inherently defective system. We demonstrate that SFCed design achieves up to$2.3\times $and$3.5\times $reduction in latency and energy, respectively, compared to parent NoI architectures while executing diverse DL WLs.

Resistive random access memory (ReRAM)-based processing-in-memory (PIM) architectures have demonstrated great potential to accelerate Deep Neural Network (DNN) training/ inference. However, the computational accuracy of analog PIM is compromised due to the non-idealities, such as the conductance variation of ReRAM cells. The impact of these non-idealities worsens as the number of concurrently activated wordlines and bitlines increases. To guarantee computational accuracy, only a limited number of wordlines and bitlines of the crossbar array can be turned on concurrently, significantly reducing the achievable parallelism of the architecture. While the constraints on parallelism limit the efficiency of the accelerators, they also provide a new opportunity for finegrained mixed-precision quantization. To enable efficient DNN inference on practical ReRAM-based accelerators, we propose an algorithm-architecture co-design framework called Block-Wise mixed-precision Quantization (BWQ). At the algorithm level, BWQ-A introduces a mixed-precision quantization scheme at the block level, which achieves a high weight and activation compression ratio with negligible accuracy degradation. We also present the hardware architecture design BWQ-H, which leverages the low-bit-width models achieved by BWQ-A to perform high-efficiency DNN inference on ReRAM devices. BWQ-H also adopts a novel precision-aware weight mapping method to increase the ReRAM crossbars throughput. Our evaluation demonstrates the effectiveness of BWQ, which achieves a 6.08× speedup and a 17.47× energy saving on average compared to existing ReRAM-based architectures.

Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures are used extensively to accelerate inferencing/training with convolutional neural networks (CNNs). Three-dimensional (3D) integration is an enabling technology to integrate many PIM cores on a single chip. In this work, we propose the design of a t hermally e fficient data flo w-aware monolithic 3D (M3D) N oC architecture referred to as TEFLON to accelerate CNN inferencing without creating any thermal bottlenecks. TEFLON reduces the Energy-Delay-Product (EDP) by 42%, 46%, and 45% on an average compared to a conventional 3D mesh NoC for systems with 36-, 64-, and 100-PIM cores, respectively. TEFLON reduces the peak chip temperature by 25 K and improves the inference accuracy by up to 11% compared to sole performance-optimized SFC-based counterpart for inferencing with diverse deep CNN models using CIFAR-10/100 datasets on a 3D system with 100-PIM cores.

Graph Neural Networks (GNNs) have achieved remarkable accuracy in cognitive tasks such as predictive analytics on graph-structured data. Hence, they have become very popular in diverse real-world applications. However, GNN training with large real-world graph datasets in edge-computing scenarios is both memory- and compute-intensive. Traditional computing platforms such as CPUs and GPUs do not provide the energy efficiency and low latency required in edge intelligence applications due to their limited memory bandwidth. Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures have been proposed as suitable candidates for accelerating AI applications at the edge, including GNN training. However, ReRAM-based PIM architectures suffer from low reliability due to their limited endurance, and low performance when they are used for GNN training in real-world scenarios with large graphs. In this work, we propose a learning-for-data-pruning framework, which leverages a trained Binary Graph Classifier (BGC) to reduce the size of the input data graph by pruning subgraphs early in the training process to accelerate the GNN training process on ReRAM-based architectures. The proposed light-weight BGC model reduces the amount of redundant information in input graph(s) to speed up the overall training process, improves the reliability of the ReRAM-based PIM accelerator, and reduces the overall training cost. This enables fast, energy-efficient, and reliable GNN training on ReRAM-based architectures. Our experimental results demonstrate that using this learning for data pruning framework, we can accelerate GNN training and improve the reliability of ReRAM-based PIM architectures by up to 1.6×, and reduce the overall training cost by 100× compared to state-of-the-art data pruning techniques.

ReRAM-based Processing-in-Memory (PIM) offers a promising paradigm for computing near data, making it an attractive platform of choice for graph applications that suffer from sparsity and irregular memory access. However, the performance of ReRAM-based graph accelerators is limited by two key challenges - significant storage requirements (particularly due to wasted zero cell storage of a graph's adjacency matrix), and significant amount of on-chip traffic between ReRAM-based processing elements. In this paper we present, GraphIte, an approximate computing-based framework for accelerating iterative graph applications on ReRAM-based architectures. GraphIte uses sparsification and approximate updates to achieve significant reductions in ReRAM storage and data movement. Our experiments on PageRank and community detection show that our proposed architecture outperforms a state-of-the-art ReRAM-based graph accelerator by up to 83.4%reduction in execution time while consuming up to 87.9%less energy for a range of graph inputs and workloads.

Chiplet-based 2.5D systems that integrate multiple smaller chips on a single die are gaining popularity for executing both compute-and data-intensive applications. While smaller chips (chiplets) reduce fabrication costs, they also provide less functionality. Hence, manufacturing several smaller chiplets and combining them into a single system enables the functionality of a larger monolithic chip without prohibitive fabrication costs. The chiplets are connected through the network-on-interposer (NoP). Designing a high-performance and energy-efficient NoP architecture is essential as it enables large-scale chiplet integration. This paper highlights the challenges and existing solutions for designing suitable NoP architectures targeted for 2.5D systems catered to datacenter-scale applications. We also highlight the future research challenges stemming from the current state-of-the-art to make the NoP-based 2.5D systems widely applicable.

A ReRAM crossbar-based computing system (RCS) can accelerate CNN training. However, hardware faults due to manufacturing defects and limited endurance impede the widespread adoption of RCS. We propose a dynamic task remapping-based technique for reliable CNN training on faulty RCS. Experimental results demonstrate that the proposed low-overhead method incurs only 0.85% accuracy loss on average while training popular CNNs such as VGGs, ResNets, and SqueezeNet with the CIFAR-IO, CIFAR-100, and SVHN datasets in the presence of faults.

Graph neural networks (GNNs) are becoming popular in various real-world applications. However, hardware-level security is a concern when GNN models are mapped to emerging neuromorphic technologies such as memristor-based crossbars. These security issues can lead to malfunction of memristor-mapped GNNs. We identify a vulnerability of memristor-mapped GNNs and propose an attack mechanism based on the identified vulnerability. The proposed attack tampers memristor-mapped graph-structured data of a GNN by injecting adversarial edges to the graph and inducing slow-to-write errors in crossbars. We show that 10% adversarial edge injection induces 1.11× longer write latency, eventually leading to a 44.33% error in node classification. Experimental results for the proposed attack also show that there is a 5.72× increase in the success rate compared to a software-based baseline.

Large-scale manycore System-on-Chips (SoCs) need to satisfy the conflicting objectives of maximizing performance and minimizing energy consumption for dynamically changing applications. In this paper, we consider the problem of dynamic power management (DPM) in large manycore SoCs for unseen applications at runtime. We employ a machine learning (ML) based DPM policy, which selects the voltage/frequency (V/F) levels for different cluster of cores as a function of the application features such as core computation, inter-core traffic etc. We propose a novel uncertainty-aware online learning framework to learn the DPM policy, which can adapt to unseen applications at runtime. It relies on two key ideas. First, an entropy-based uncertainty measure is used to distinguish between seen and unseen system states. Second, we employ conformal prediction to compute uncertain V/F sets for unseen system states. We perform bounded-search over the uncertain V/F configurations using power/performance models to identify the best V/F configurations to minimize the energy-delay product (EDP) and create supervised examples for online learning. Our experiments on 64-core system show that the EDP is reduced by up to 50 % and 60 % when compared to existing online-imitation learning and reinforcement learning methods, respectively.

Advanced computing systems have long been enablers for breakthroughs in Machine Learning (ML) algorithms either through sheer computational power or form-factor miniaturization. However, as ML algorithms become more complex and the size of datasets increase, existing computing platforms are no longer sufficient to bridge the gap between algorithmic innovation and hardware design. With the rising needs of advanced algorithms for large-scale data analysis and data-driven discovery, and significant growth in emerging applications from the edge to the cloud, we need energy-efficient, low-cost, high- performance, and reliable computing systems targeted for these applications. This paper presents the latest developments in oscillatory neural networks, optical computing, and memristive processing-in-memory (PIM) to address the various challenges in designing efficient computing systems specifically targeting ML applications.

Machine learning (ML) models have gained prominence in solving real-world tasks. However, implementing ML models is both compute- and memory-intensive. Domain-specific architectures such as Resistive Random Access Memory (ReRAM)-based Processing-in-Memory (PIM) platforms have been proposed to efficiently accelerate ML training and inference. However, existing ML workloads require a high amount of area and power for training. A major contributor to the area and power overheads is the Analog-to-Digital Converter (ADC). In this work, we propose a mixed pruning technique along with a novel reconfigurable ADC design to improve the power consumption profile. Overall, the pruned model with the reconfigurable ADC achieves ~50% reduction in power for training compared to existing state-of-the-art ReRAM-based architectures.

Graph neural networks (GNNs) are used for predictive analytics on graph-structured data, and they have become very popular in diverse real-world applications. Resistive random-access memory (ReRAM)-based PIM architectures can accelerate GNN training. However, GNN training on ReRAM-based architectures is both compute- and data intensive in nature. In this work, we propose a framework calledSlimGNNthat synergistically combines both graph and model pruning to accelerate GNN training on ReRAM-based architectures. The proposed framework reduces the amount of redundant information in both the GNN model and input graph(s) to streamline the overall training process. This enables fast and energy-efficient GNN training on ReRAM-based architectures. Experimental results demonstrate that using this framework, we can accelerate GNN training by up to$ {4}. {5} {\times }$while using$ {6}. {6} {\times }$less energy compared to the unpruned counterparts.

Processing-in-memory (PIM) enables energy-efficient deployment of convolutional neural networks (CNNs) from edge to cloud. Resistive random-access memory (ReRAM) is one of the most commonly used technologies for PIM architectures. One of the primary limitations of ReRAM-based PIM in neural network training arises from the limited write endurance due to the frequent weight updates. To make ReRAM-based architectures viable for CNN training, the write endurance issue needs to be addressed. This work aims to reduce the number of weight reprogrammings without compromising the final model accuracy. We propose the ESSENCE framework with an endurance-aware structured stochastic gradient pruning method, which dynamically adjusts the probability of gradient update based on the current update counts. Experimental results with multiple CNNs and datasets demonstrate that the proposed method can extend ReRAM’s life time for training. For instance, with the ResNet20 network and CIFAR-10 dataset, ESSENCE can save the mean update counts of up to$10.29\times $compared to the stochastic gradient descent method and effectively reduce the maximum update counts compared with the No Endurance method. Furthermore, an aggressive tuning method based on ESSENCE can boost the mean update count savings by up to$14.41\times $.

Recent advances in 2.5D chiplet platforms provide a new avenue for compact scale-out implementations of emerging compute- and data-intensive applications including machine learning. Network-on-Interposer (NoI) enables integration of multiple chiplets on a 2.5D system. While these manycore platforms can deliver high computational throughput and energy efficiency by running multiple specialized tasks concurrently, conventional NoI architectures have a limited computational throughput due to their inherent multi-hop topologies. In this paper, we propose Floret, a novel NoI architecture based on space-filling curves (SFCs). The Floret architecture leverages suitable task mapping, exploits the data flow pattern, and optimizes the inter-chiplet data exchange to extract high performance for multiple types of convolutional neural network (CNN) inference tasks running concurrently. We demonstrate that the Floret architecture reduces the latency and energy up to 58% and 64%, respectively, compared to state-of-the-art NoI architectures while executing datacenter-scale workloads involving multiple CNN tasks simultaneously. Floret achieves high performance and significant energy savings with much lower fabrication cost by exploiting the data-flow awareness of the CNN inference tasks.

Graph application workloads are dominated by random memory accesses with the poor locality. To tackle the irregular and sparse nature of computation, ReRAM-based Processing-in-Memory (PIM) architectures have been proposed recently. Most of these ReRAM architecture designs have focused on mapping graph computations into a set of multiply-and-accumulate (MAC) operations. ReRAMs also offer a key advantage in reducing memory latency between cores and memory by allowing for PIM. However, when implemented on a ReRAM-based manycore architecture, graph applications still pose two key challenges—significant storage requirements (particularly due to wasted zero cell storage), and significant amount of on-chip traffic. To tackle these two challenges, in this article, we propose the design of a 3D NoC-enabled ReRAM-based manycore architecture. Our proposed architecture incorporates a novel crossbar-aware node reordering to reduce ReRAM storage requirements. Secondly, its 3D NoC-enabled design reduces on-chip communication latency. Our architecture outperforms the state-of-the-art in ReRAM-based graph acceleration by up to 5× in performance while consuming up to 10.3× less energy for a range of graph inputs and workloads.

The complexity of manycore System-on-chips (SoCs) is growing faster than our ability to manage them to reduce the overall energy consumption. Further, as SoC design moves toward three-dimensional (3D) architectures, the core's power density increases leading to unacceptable high peak chip temperatures. In this article, we consider the optimization problem of dynamic power management (DPM) in manycore SoCs for an allowable performance penalty (say, 5%) and admissible peak chip temperature. We employ a machine learning– (ML) based DPM policy, which selects the voltage/frequency levels for different cluster of cores as a function of the application workload features such as core computation and inter-core traffic, and so on. We propose a novel learning-to-search (L2S) framework to automatically identify an optimized sequence of DPM decisions from a large combinatorial space for joint energy-thermal optimization for one or more given applications. The optimized DPM decisions are given to a supervised learning algorithm to train a DPM policy, which mimics the corresponding decision-making behavior. Our experiments on two different manycore architectures designed using wireless interconnect and monolithic 3D demonstrate that principles behind the L2S framework are applicable for more than one configuration. Moreover, L2S-based DPM policies achieve up to 30% energy-delay product savings and reduce the peak chip temperature by up to 17 °C compared to the state-of-the-art ML methods for an allowable performance overhead of only 5%.

Resistive random-access memory has become one of the most popular choices of hardware implementation for machine learning application workloads. However, these devices exhibit non-ideal behavior, which presents a challenge towards widespread adoption. Training/inferencing on these faulty devices can lead to poor prediction accuracy. However, existing fault tolerant methods are associated with high implementation overheads. In this paper, we present some new directions for solving reliability issues using software solutions. These software-based methods are inherent in deep learning training/inferencing, and they can also be used to address hardware reliability issues as well. These methods prevent accuracy drop during training/inferencing due to unreliable ReRAMs and are associated with lower area and power overheads.

Recent advances in GPU-based manycore accelerators provide the opportunity to efficiently process large-scale graphs on chip. However, real world graphs have a diverse range of topology and connectivity patterns (e.g., degree distributions) that make the design of input-agnostic hardware architectures a challenge. Network-on-Chip (NoC)- based architectures provide a way to overcome this challenge as the architectural topology can be used to approximately model the expected traffic patterns that emerge from graph application workloads. In this paper, we first study the mix of long- and short-range traffic patterns generated on-chip using graph workloads, and subsequently use the findings to adapt the design of an optimal NoC-based architecture. In particular, by leveraging emerging three-dimensional (3D) integration technology, we propose design of a small-world NoC (SWNoC)- enabled manycore GPU architecture, where the placement of the links connecting the streaming multiprocessors (SM) and the memory controllers (MC) follow a power-law distribution. The proposed 3D manycore GPU architecture outperforms the traditional planar (2D) counterparts in both performance and energy consumption. Moreover, by adopting a joint performance-thermal optimization strategy, we address the thermal concerns in a 3D design without noticeably compromising the achievable performance. The 3D integration technology is also leveraged to incorporate Near Data Processing (NDP) to complement the performance benefits introduced by the SWNoC architecture. As graph applications are inherently memory intensive, off-chip data movement gives rise to latency and energy overheads in the presence of external DRAM. In conventional GPU architectures, as the main memory layer is not integrated with the logic, off-chip data movement negatively impacts overall performance and energy consumption. We demonstrate that NDP significantly reduces the overheads associated with such frequent and irregular memory accesses in graph-based applications. The proposed SWNoC-enabled NDP framework that integrates 3D memory (like Micron's HMC) with a massive number of GPU cores achieves 29.5% performance improvement and 30.03% less energy consumption on average compared to a conventional planar Mesh-based design with external DRAM.

Resistive random-access memory (ReRAM)-based architectures can be used to accelerate convolutional neural network (CNN) training. However, existing architectures either do not support normalization at all or they support only a limited version of it. Moreover, it is common practice for CNNs to add normalization layers after every convolution layer. In this work, we show that while normalization layers are necessary to train deep CNNs, only a few such layers are sufficient for effective training. A large number of normalization layers do not improve prediction accuracy; it necessitates additional hardware and gives rise to performance bottlenecks. To address this problem, we proposeDeepTrain, a heterogeneous architecture enabled by a Bayesian optimization (BO) methodology; together, they provide adequate hardware and software support for normalization operations. The proposed BO methodology determines the minimum number of normalization operations necessary for a given CNN. Experimental evaluation indicates that the BO-enabledDeepTrainarchitecture achieves up to$15\times $speedup compared to a conventional GPU for training CNNs with no accuracy loss while utilizing only a few normalization layers.

Resistive random-access memory (ReRAM)-based manycore architectures enable acceleration of graph neural network (GNN) inference and training. GNNs exhibit characteristics of both DNNs and graph analytics. Hence, GNN training/inferencing on ReRAM-based manycore architectures give rise to both computation and on-chip communication challenges. In this work, we leverage model pruning and efficient graph storage to reduce the computation and communication bottlenecks associated with GNN training on ReRAM-based manycore accelerators. However, traditional pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. In this work, we propose a GNN pruning technique called DietGNN. DietGNN is a crossbar-aware pruning technique that achieves high accuracy training and enables energy, area, and storage efficient computing on ReRAM-based manycore platforms. The DietGNN pruned model can be trained from scratch without any noticeable accuracy loss. Our experimental results show that when mapped on to a ReRAM-based manycore architecture, DietGNN can reduce the number of crossbars by over 90% and accelerate GNN training by${\sim }{2}.{7}{\times }$compared to its unpruned counterpart. In addition, DietGNN reduces energy consumption by more than${\sim }{3}.{5}{\times }$compared to the unpruned counterpart.

Processing-in-memory (PIM) is a promising technique to accelerate deep learning (DL) workloads. Emerging DL workloads (e.g., ResNet with 152 layers) consist of millions of parameters, which increase the area and fabrication cost of monolithic PIM accelerators. The fabrication cost challenge can be addressed by 2.5-D systems integrating multiple PIM chiplets connected through a network-on-package (NoP). However, server-scale scenarios simultaneously execute multiple compute-heavy DL workloads, leading to significant interchiplet data volume. State-of-the-art NoP architectures proposed in the literature do not consider the nature of DL workloads. In this article, we propose a novel server scale 2.5-D manycore architecture called SWAP that accounts for the traffic characteristics of DL applications. Comprehensive experimental evaluations with different system sizes as well as diverse emerging DL workloads demonstrate that SWAP achieves significant performance and energy consumption improvements with much lower fabrication cost than state-of-the-art NoP topologies.

With the increasing complexity of highly integrated system on chips (SoCs), the power management system (PMS) is required to provide several power supplies efficiently for individual blocks. This paper presents a single-inductor multiple-output (SIMO) inductor-first hybrid converter that generates three outputs between 0.4V and 1.6V from a 1.8V input. The proposed multiple-output hybrid power stage can improve the conversion efficiency by reducing inductor current while extending the output voltage range compared with the existing hybrid topologies. In addition, the proposed converter employs an on-chip switched-capacitor power stage (SCPS) with a dual-switching frequency technique, resulting in a fast response time, low cross-regulation, and reduced number of on-chip pads. Measurement results show that the converter achieves a peak efficiency of 87.5% with the maximum output current of 450mA. The converter is integrated with a fast voltage regulation loop with 500MHz system clock to achieve a less than 0.01mA/mV cross-regulation and a maximum 20mV overshoot at full-load transient response. The design is fabricated in the standard 180nm CMOS technology.

Manycore GPU architectures have become the mainstay for accelerating graph computations. One of the primary bottlenecks to performance of graph computations on manycore architectures is the data movement. Since most of the accesses in graph processing are due to vertex neighborhood lookups, locality in graph data structures plays a key role in dictating the degree of data movement. Vertex reordering is a widely used technique to improve data locality within graph data structures. However, these reordering schemes alone are not sufficient as they need to be complemented with efficient task allocation on manycore GPU architectures to reduce latency due to local cache misses. Consequently, in this article, we introduce a software/hardware co-design framework for accelerating graph computations. Our approach couples an architecture-aware vertex reordering with a priority-based task allocation technique. As the task allocation aims to reduce on-chip latency and associated energy, the choice of Network-on-Chip (NoC) as the communication backbone in the manycore platform is an important parameter. By leveraging emerging three-dimensional (3D) integration technology, we propose design of a small-world NoC (SWNoC)-enabled manycore GPU architecture, where the placement of the links connecting the streaming multiprocessors (SMs) and the memory controllers (MCs) follow a power-law distribution. The proposed 3D SWNoC-enabled software/hardware co-design framework achieves 11.1% to 22.9% performance improvement and 16.4% to 32.6% less energy consumption depending on the dataset and the graph application, when compared to the default order of dataset running on a conventional planar mesh architecture.

Mobile system-on-chips (SoCs) are growing in their complexity and heterogeneity (e.g., Arm’s Big-Little architecture) to meet the needs of emerging applications, including games and artificial intelligence. This makes it very challenging to optimally manage the resources (e.g., controlling the number and frequency of different types of cores) at runtime to meet the desired trade-offs among multiple objectives such as performance and energy. This paper proposes a novel information-theoretic framework referred to as PaRMIS to create Pareto-optimal resource management policies for given target applications and design objectives. PaRMIS specifies parametric policies to manage resources and learns statistical models from candidate policy evaluation data in the form of target design objective values. The key idea is to select a candidate policy for evaluation in each iteration guided by statistical models that maximize the information gain about the true Pareto front. Experiments on a commercial heterogeneous SoC show that PaRMIS achieves better Pareto fronts and is easily usable to optimize complex objectives (e.g., performance per Watt) when compared to prior methods

Graph Neural Network (GNN) is a variant of Deep Neural Networks (DNNs) operating on graphs. However, GNNs are more complex compared to traditional DNNs as they simultaneously exhibit features of both DNN and graph applications. As a result, architectures specifically optimized for either DNNs or graph applications are not suited for GNN training. In this work, we propose a 3D heterogeneous manycore architecture for on-chip GNN training to address this problem. The proposed architecture, ReGraphX, involves heterogeneous ReRAM crossbars to fulfill the disparate requirements of both DNN and graph computations simultaneously. The ReRAM-based architecture is complemented with a multicast-enabled 3D NoC to improve the overall achievable performance. We demonstrate that ReGraphX outperforms conventional GPUs by up to 3.5X (on an average 3X) in terms of execution time, while reducing energy consumption by as much as 11X.

Three-dimensional (3D) integration has frequently been described as a means to overcome scaling bottlenecks, and advance both “More Moore” and “More Than Moore” through the use of vertical interconnects and die/wafer stacking. Recent industry trends show the viability of 3D integration in real products. Flash memory producers have also demonstrated multiple layers of memory on top of each other. However, conventional TSV-based 3D designs cannot achieve the full-potential of vertical integration and perform suboptimally. Monolithic 3D (M3D) is an emerging vertical integration technology that promises significant power-performance-area benefits compared to TSVs. Hence, it is important to understand the necessary design trade-offs and challenges associated with this new paradigm. In this paper, we present both the advantages and the various design challenges in M3D-enabled system design considering Processing-in-Memory (PIM) and manycore systems as suitable case-studies.

Graph Neural Networks (GNNs) are a variant of Deep Neural Networks (DNNs) operating on graphs. GNNs have attributes of both DNNs and graph computation. However, training GNNs on manycore architectures is a challenging task because it involves heavy communication that bottlenecks performance. DropEdge and Dropout, which we collectively refer to as DropLayer, are regularization techniques that can improve the predictive accuracy of GNNs. Moreover, when implemented on a manycore architecture, DropEdge and Dropout are capable of reducing the on-chip traffic. In this paper, we present a ReRAM-based 3D manycore architecture called DARe, tailored for accelerating on-chip training of GNNs. The key component of the DARe architecture is a Network-on-Chip (NoC) that reduces the amount of communication using DropLayer. The reduced traffic prevents communication hotspots and leads to better performance. We demonstrate that DARe outperforms conventional GPUs by up to 6.7X (5.6X on average) in terms of execution time, while being up to 30X (23X on average) more energy efficient for GNN training.

A huge number of edge applications including self-driving cars, mobile health, robotics, and augmented reality / virtual reality are enabled by deep neural networks (DNNs). Currently, much of this computation for these applications happens in the cloud, but there are several good reasons to perform the processing on local edge platforms such as smartphones: improved accessibility to different parts of the world, low latency, and data privacy. In this paper, we present a general hardware and software co-design framework for energy-efficient edge AI for both simple classification and structured output prediction tasks (e.g., 3D shapes from images). This framework relies on two key ideas. First, we design a space of DNNs of increasing complexity (coarse to fine) and perform input-specific adaptive inference by selecting a DNN of appropriate complexity depending on the hardness of input examples. Second, we execute the selected DNN on the target edge platform using a resource management policy to save energy. We also provide instantiations of our co-design framework for three qualitatively different problem settings: convolutional neural networks for image classification, graph convolutional networks for predicting 3D shapes from images, and generative adversarial networks on photo-realistic unconditional image generation. Our experiments on real-world benchmarks and mobile platforms show the effectiveness of our co-design framework in achieving significant gain in energy with little to no loss in accuracy of predictions.

Resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architectures have recently become a popular architectural choice for deep-learning applications. ReRAM-based architectures can accelerate inferencing and training of deep learning algorithms and are more energy efficient compared to traditional GPUs. However, these architectures have various limitations that affect the model accuracy and performance. Moreover, the choice of the deep-learning application also imposes new design challenges that must be addressed to achieve high performance. In this paper, we present the advantages and challenges associated with ReRAM-based PIM architectures by considering Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs) as important application domains. We also outline methods that can be used to address these challenges.

Resistive random-access memory (ReRAM) is a promising technology for designing hardware accelerators for deep neural network (DNN) inferencing. However, stochastic noise in ReRAM crossbars can degrade the DNN inferencing accuracy. We propose the design and optimization of a high-performance, area-and energy-efficient ReRAM-based hardware accelerator to achieve robust DNN inferencing in the presence of stochastic noise. We make two key technical contributions. First, we propose a stochastic-noise-aware training method, referred to as ReSNA, to improve the accuracy of DNN inferencing on ReRAM crossbars with stochastic noise. Second, we propose an information-theoretic algorithm, referred to as CF-MESMO, to identify the Pareto set of solutions to trade-off multiple objectives, including inferencing accuracy, area overhead, execution time, and energy consumption. The main challenge in this context is that executing the ReSNA method to evaluate each candidate ReRAM design is prohibitive. To address this challenge, we utilize the continuous-fidelity evaluation of ReRAM designs associated with prohibitive high computation cost by varying the number of training epochs to trade-off accuracy and cost. CF-MESMO iteratively selects the candidate ReRAM design and fidelity pair that maximizes the information gained per unit computation cost about the optimal Pareto front. Our experiments on benchmark DNNs show that the proposed algorithms efficiently uncover high-quality Pareto fronts. On average, ReSNA achieves 2.57% inferencing accuracy improvement for ResNet20 on the CIFAR-10 dataset with respect to the baseline configuration. Moreover, CF-MESMO algorithm achieves 90.91% reduction in computation cost compared to the popular multi-objective optimization algorithm NSGA-II to reach the best solution from NSGA-II.

Voltage/frequency island (VFI)-based power management is a popular methodology for designing energy-efficient manycore architectures without incurring significant performance overhead. However, monolithic 3D (M3D) integration has emerged as an enabling technology to design high-performance and energy-efficient circuits and systems. The smaller dimension of vertical monolithic inter-tier vias (MIVs) lowers effective wirelength and allows high integration density. However, sequential fabrication of M3D layers introduces inter-tier process variations that affect the performance of transistors and interconnects in different layers. Therefore, VFI-based power management in M3D manycore systems requires the consideration of inter-tier process variation effects. In this work, we present the design of an imitation learning (IL)-enabled VFI-based power-management strategy that considers the inter-tier process-variation effects in M3D manycore chips. We demonstrate that the IL-based power-management strategy can be fine-tuned based on the M3D characteristics. Our policy generates suitable V/F levels based on the computation and communication characteristics of the system for both process-oblivious and process-aware configurations. We show that the proposed process-variation-aware IL-based VFI implementation for M3D manycore chips lowers the overall energy-delay-product (EDP) by up to 16.2% on average compared to an ideal M3D system with no M3D process variations.

The growing popularity of convolutional neural networks (CNNs) along with their complexity has led to the search for efficient computational platforms suitable for them. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for training CNNs. However, due to their low-precision storage capability, these architectures cannot support all types of CNN layers and suffer from accuracy loss of the learned model. In addition, ReRAM behavior varies with temperature. High temperature reduces noise margin and introduces additional noise. This makes training of CNNs challenging as outputs can be misinterpreted at higher operating temperatures leading to accuracy loss. In this work, we propose an M3D-enabled heterogeneous architecture: AccuReD, that combines ReRAM arrays with GPU cores, to address these challenges and achieve high accuracy CNN training. AccuReD supports all types of CNN layers and achieve near-GPU accuracy even with low-precision and nonideal behavior of ReRAMs. In addition, to reduce temperature, we present a performance-thermal-aware mapping policy that maps CNN layers to the computing elements of AccuReD. Experimental evaluation indicates that AccuReD does not lose accuracy while accelerating CNN training by 12× on an average compared to conventional GPU-only platforms.

The growing popularity of convolutional neural networks (CNNs) has led to the search for efficient computational platforms to accelerate CNN training. Resistive random-access memory (ReRAM)-based manycore architectures offer a promising alternative to commonly used GPU-based platforms for training CNNs. However, due to the immature fabrication process and limited write endurance, ReRAMs suffer from different types of faults. This makes training of CNNs challenging as weights are misrepresented when they are mapped to faulty ReRAM cells. This results in unstable training, leading to unacceptably low accuracy for the trained model. Due to the distributed nature of the mapping of the individual bits of a weight to different ReRAM cells, faulty weights often lead to exploding gradients. This in turn introduces a positive feedback in the training loop, resulting in extremely large and unstable weights. In this paper, we propose a lightweight and reliable CNN training methodology using weight clipping to prevent this phenomenon and enable training even in the presence of many faults. Weight clipping prevents large weights from destabilizing CNN training and provides the backpropagation algorithm with the opportunity to compensate for the weights mapped to faulty cells. The proposed methodology achieves near-GPU accuracy without introducing significant area or performance overheads. Experimental evaluation indicates that weight clipping enables the successful training of CNNs in the presence of faults, while also reducing training time by 4 X on average compared to a conventional GPU platform. Moreover, we also demonstrate that weight clipping outperforms a recently proposed error correction code (ECC)-based method when training is carried out using faulty ReRAMs.

Heterogeneous manycore architectures are the key to efficiently execute compute- and data-intensive applications. Through-silicon-via (TSV)-based 3D manycore system is a promising solution in this direction as it enables the integration of disparate computing cores on a single system. Recent industry trends show the viability of 3D integration in real products (e.g., Intel Lakefield SoC Architecture, the AMD Radeon R9 Fury X graphics card, and Xilinx Virtex-7 2000T/H580T, etc.). However, the achievable performance of conventional TSV-based 3D systems is ultimately bottlenecked by the horizontal wires (wires in each planar die). Moreover, current TSV 3D architectures suffer from thermal limitations. Hence, TSV-based architectures do not realize the full potential of 3D integration. Monolithic 3D (M3D) integration, a breakthrough technology to achieve “More Moore and More Than Moore,” opens up the possibility of designing cores and associated network routers using multiple layers by utilizing monolithic inter-tier vias (MIVs) and hence, reducing the effective wire length. Compared to TSV-based 3D integrated circuits (ICs), M3D offers the “true” benefits of vertical dimension for system integration: the size of an MIV used in M3D is over 100 × smaller than a TSV. This dramatic reduction in via size and the resulting increase in density opens up numerous opportunities for design optimizations in 3D manycore systems: designers can use up to millions of MIVs for ultra-fine-grained 3D optimization, where individual cores and routers can be spread across multiple tiers for extreme power and performance optimization. In this work, we demonstrate how M3D-enabled vertical core and uncore elements offer significant performance and thermal improvements in manycore heterogeneous architectures compared to its TSV-based counterpart. To overcome the difficult optimization challenges due to the large design space and complex interactions among the heterogeneous components (CPU, GPU, Last Level Cache, etc.) in a M3D-based manycore chip, we leverage novel design-space exploration algorithms to trade off different objectives. The proposed M3D-enabled heterogeneous architecture, called HeM3D , outperforms its state-of-the-art TSV-equivalent counterpart by up to 18.3% in execution time while being up to 19°C cooler.

Graph neural network (GNN) is a variant of deep neural networks (DNNs) operating on graphs. However, GNNs are more complex compared with DNNs as they simultaneously exhibit attributes of both DNN and graph computations. In this work, we propose a ReRAM-based 3-D manycore processing-in-memory architecture called ReMaGN, tailored for on-chip training of GNNs. ReMaGN implements GNN training using reduced-precision representation to make the computation faster and reduce the load on the communication backbone. However, reduced precision can potentially compromise the accuracy of training. Hence, we undertake a study of performance and accuracy tradeoffs in such architectures. We demonstrate that ReMaGN outperforms conventional GPUs by up to$9.5\times $(on average$7.1\times $) in terms of execution time, while being up to$42\times $(on average$33.5\times $) more energy efficient without sacrificing accuracy.

We consider the task of predicting 3D object shapes from color images on mobile platforms, which has many real-world applications including augmented reality (AR), virtual reality (VR), and robotics. Recent work has developed a Graph Convolution Network (GCN) based approach to produce 3D object shapes in the form of a triangular mesh of increasing polycount (no. of triangles in the mesh). In this paper, we propose a novel approach to trade-off polycount of a 3D object shape for the energy consumed at run-time called Polycount-Energy Trade-off networks (PETNet). The key idea behind PETNets is to design an architecture of increasing complexity with a comparator module and leveraging the pre-trained GCN to perform input-specific adaptive predictions. We perform experiments using pre-trained GCN on the ShapeNet dataset. Results show that with the optimized PETNets, we can get up to 20%-37% gain in energy for negligible loss (0.01 to 0.02) in accuracy, and provides a fine-grained control on performance when compared to a fixed level performance with the state-of-the-art Pixel2Mesh network.

Dynamic resource management has become one of the major areas of research in modern computer and communication system design due to lower power consumption and higher performance demands. The number of integrated cores, level of heterogeneity and amount of control knobs increase steadily. As a result, the system complexity is increasing faster than our ability to optimize and dynamically manage the resources. Moreover, offline approaches are sub-optimal due to workload variations and large volume of new applications unknown at design time. This paper first reviews recent online learning techniques for predicting system performance, power, and temperature. Then, we describe the use of predictive models for online control using two modern approaches: imitation learning (IL) and an explicit nonlinear model predictive control (NMPC). Evaluations on a commercial mobile platform with 16 benchmarks show that the IL approach successfully adapts the control policy to unknown applications. The explicit NMPC provides 25% energy savings compared to a state-of-the-art algorithm for multi-variable power management of modern GPU sub-systems.

Deep Neural Networks (DNNs) employed for image segmentation are computationally more expensive and complex compared to the ones used for classification. However, manycore architectures to accelerate the training of these DNNs are relatively unexplored. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for training DNNs. However, due to their low-precision storage capability, these architectures cannot support all DNN layers and suffer from accuracy loss of the learned models. To address these challenges, we propose GRAMARCH, a heterogeneous architecture that combines the benefits of ReRAM and GPUs simultaneously by using a high-throughput 3D Network-on-Chip. Experimental results indicate that by suitably mapping DNN layers to processing elements, it is possible to achieve up to 53X better performance compared to conventional GPUs for image segmentation.

Monolithic 3D (M3D) technology enables unprecedented degrees of integration on a single chip. The miniscule monolithic inter-tier vias (MIVs) in M3D are the key behind higher transistor density and more flexibility in designing circuits compared to conventional through silicon via (TSV)-based architectures. This results in significant performance and energyefficiency improvements in M3D-based systems. Moreover, the thin inter-layer dielectric (ILD) used in M3D provides better thermal conductivity compared to TSV-based solutions and eliminates the possibility of thermal hotspots. However, the fabrication of M3D circuits still suffers from several non-ideal effects. The thin ILD layer may cause electrostatic coupling between tiers. Furthermore, the low-temperature annealing degrades the top-tier transistors and bottom-tier interconnects. An NoC-based manycore design needs to consider all these M3D- process related non-idealities. In this paper, we discuss various design challenges for an M3D-enabled manycore chip. We present the power-performance-thermal trade-offs associated with these emerging manycore architectures.

Efficiency of power management system (PMS) is one of the key performance metrics for highly integrated system on chips (SoCs). Towards the goal of improving power efficiency of SoCs, we make two key technical contributions in this paper. First, we develop a multi-output switched-capacitor voltage regulator (SCVR) with a new flying capacitor crossing technique (FCCT) and cloud-capacitor method. Second, to optimize the design parameters of SCVR, we introduce a novel machine¬learning (ML)-inspired optimization framework to reduce the number of expensive design simulations. Simulation shows that power loss of the multi-output SCVR with FCCT is reduced by more than 40% compared to conventional multiple single-output SCVRs. Our ML-based design optimization framework is able to achieve more than 90% reduction in the number of simulations needed to uncover optimized circuit parameters of the proposed SCVR.

We consider the task of photo-realistic unconditional image generation (generate high quality, diverse samples that carry the same visual content as the image) on mobile platforms using Generative Adversarial Networks (GANs). In this paper, we propose a novel approach to trade-off image generation accuracy of a GAN for the energy consumed (compute) at run-time called Scale-Energy Trade-off GAN (SETGAN). GANs usually take a long time to train and consume a huge memory hence making it difficult to run on edge devices. The key idea behind SETGAN for an image generation task is for a given input image, we train a GAN on a remote server and use the trained model on edge devices. We use SinGAN, a single image unconditional generative model, that contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. During the training process, we determine the optimal number of scales for a given input image and the energy constraint from target edge device. Results show that with the SETGAN's unique client-server based architecture, we were able to achieve 56% gain in energy for a loss of 3% to 12% SSIM accuracy. Also, with the parallel multi-scale training, we obtain around 4x gain in training time on the server.

3D Network-on-Chip (3D NoC) enables design of high-performance and energy-efficient manycore computing platforms. Two of the commonly used vertical interconnection technologies are: through silicon via (TSV) and near-field inductive coupling (NFIC). Both TSV- and NFIC-based links introduce additional area overhead. One of the possible ways to reduce the area overhead is to design partially connected 3D NoC with minimal effect on overall performance. The achievable performance of the partially connected 3D NoCs depends on the area, energy, and bandwidth of the vertical links. Moreover, the electromigration-induced failure of TSV is more severe than the misalignment-induced error in NFIC. By considering all these pertinent factors, we demonstrate that it is indeed possible to design a partially connected NFIC-based 3D NoC that performs as good as a fully connected TSV-based counterpart when the data rate is below a certain limit. For higher data rates, the partially connected TSV-based 3D NoC is a viable solution. However, NFIC-based 3D NoC is always more robust than the TSV-based counterpart.

Deep neural network (DNN) models are being expanded to a broader range of applications. The computational capability of traditional hardware platforms cannot accommodate the growth of model complexity. Among recent technologies to accelerate DNN, resistive memory (ReRAM)-based processing-in-memory (PIM) emerged as a promising solution for DNN inference due to its high efficiency for matrix-based computation. We face two major technical challenges in extending the use of ReRAM-based accelerators for training: (1) full-precision data is essential in back-propagation; (2) the need to support both feed-forward and back-propagation aggravates the data-movement burden. We propose a heterogeneous architecture named as 3D-ReG, which leverages full-precision GPU to ensure training accuracy and low-overhead 3D integration to provide low-cost data movements. Moreover, we introduce conservative and aggressive task-mapping schemes, which partition the computation phases in different ways to balance execution efficiency and training accuracy. We evaluate 3D-ReG implemented with two 3D integration technologies, through-silicon vias (TSVs) and monolithic inter-tier vias (MIVs), and compare them with GPU-only and PIM-only counterparts. Various GPU-only platforms using two main-memory technologies (DRAM, ReRAM) and three interconnect technologies (2D, TSV, MIV) are evaluated as well. Experimental results show that 3D-ReG can achieve on average 5.64× training speedup and 3.56× higher energy efficiency compared with the GPU with DRAM as main memory, at the cost of 0.05%–3.39% accuracy drop. We define a new metric, gain-loss ratio (GLR), which quantitatively evaluates the capability of a DNN training hardware in terms of the model accuracy and hardware efficiency. The results of our comparison show that the aggressive task-mapping scheme on MIV-based 3D-ReG outperforms the other methods.

Mobile platforms must satisfy the contradictory requirements of fast response time and minimum energy consumption as a function of dynamically changing applications. To address this need, systems-on-chip (SoC) that are at the heart of these devices provide a variety of control knobs, such as the number of active cores and their voltage/frequency levels. Controlling these knobs optimally at runtime is challenging for two reasons. First, the large configuration space prohibits exhaustive solutions. Second, control policies designed offline are at best sub-optimal, since many potential new applications are unknown at design-time. We address these challenges by proposing an online imitation learning approach. Our key idea is to construct an offline policy and adapt it online to new applications to optimize a given metric (e.g., energy). The proposed methodology leverages the supervision enabled by power-performance models learned at runtime. We demonstrate its effectiveness on a commercial mobile platform with 16 diverse benchmarks. Our approach successfully adapts the control policy to an unknown application after executing less than 25% of its instructions.

Monolithic 3-D (M3D) technology enables high density integration, performance, and energy efficiency by sequentially stacking tiers on top of each other. M3D-based network-on-chip (NoC) architectures can exploit these benefits by adopting tier partitioning for intra-router stages. However, conventional fabrication methods are infeasible for M3D-enabled designs due to temperature-related issues. This has necessitated lower temperature and temperature-resilient techniques for M3D fabrication, leading to inferior performance of transistors in the top tier and interconnects in the bottom tier. The resulting inter-tier process variation leads to the performance degradation of M3D-enabled NoCs. In this article, we demonstrate that without considering inter-tier process variation, an M3D-enabled NoC architecture overestimates the energy-delay-product (EDP) on average by 50.8% for a set of SPLASH-2 and PARSEC benchmarks. As a countermeasure, we adopt a process variation-aware design approach. The proposed design and optimization method distributes the intra-router stages and inter-router links among the tiers to mitigate the adverse effects of process variation. Experimental results show that the NoC architecture under consideration improves the EDP by 27.4% on average across all benchmarks compared to the process-oblivious design.

Due to the growing needs of Big Data applications (e.g., deep learning, graph analytics, and scientific computing) and the ending of Moore's law, there is a great need for low-cost, high-performance, energy-efficient, and small form-factor manycore systems. With more stringent design objectives, application specialization, and more cores on a single chip, design-time optimization becomes more complex. Moreover, with the advent of emerging interconnect technologies like 3D integration, the design optimization process has become more challenging. This increases the need for a holistically optimized design process that makes design decisions across multiple layers of the system, e.g., memory, compute, interconnect technology, and network infrastructure. In this paper, we present various challenges of designing heterogeneous manycore architectures using 3D integration and viable optimization techniques to solve them.

The growing popularity of Convolutional Neural Networks (CNNs) has led to the search for efficient computational platforms to enable these algorithms. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for CNN training. However, backpropagation in CNNs is susceptible to the limited precision of ReRAMs. As a result, training CNNs on ReRAMs affects the final accuracy of learned model. In this work, we propose REGENT, a heterogeneous architecture that combines ReRAM arrays with GPU cores, and exploits the benefits provided by 3D integration along with a high-throughput yet energy efficient Network-on-Chip (NoC) for training CNNs. We also propose a bin-packing based framework that maps CNN layers and then optimize the placement of computing elements to meet the targeted design objectives. Experimental evaluations indicate that REGENT improves full-system EDP by 55.7% on average compared to conventional GPU-only platforms for training CNNs.

The notion of networks is inherent in the structure, function and behavior of the natural and engineered world that surround us. Consequently, graph models and methods have assumed a prominent role to play in this modern era of Big Data, and are taking a center stage in the discovery pipelines of various data-driven scientific domains. In this paper, we present a brief review of the state-of-the-art in parallel graph analytics, particularly focusing on iterative graph algorithms and their implementation on modern day multicore/manycore architectures. The class of iterative graph algorithms covers a broad class of graph operations of varying complexities, from simpler routines such as Breadth-First Search (BFS), to polynomially-solvable problems such as shortest path computations, to NP-Hard problems such as community detection and graph coloring. We cover a set of common algorithmic abstractions used in implementing such iterative graph algorithms, state the challenges around parallelization on contemporary parallel platforms (including commodity multicores and emerging manycore platforms), and describe a set of approaches that have led to efficient implementations. We also report on advances in manycore architectural frameworks that have found application in parallel graph analytics. We conclude the paper identifying potential research directions, opportunities, and challenges that lay ahead in the path toward enabling graph analytics at exascale.

Counting k-mers (substrings of fixed length k) in DNA and protein sequences generate non-uniform and irregular memory access patterns. Processing-in-Memory (PIM) architectures have the potential to significantly reduce the overheads associated with such frequent and irregular memory accesses. However, existing k-mer counting algorithms are not designed to exploit the advantages of PIM architectures. Furthermore, owing to thermal constraints, the allowable power budget is limited in conventional PIM designs. Moreover, k-mer counting generates unbalanced and long-range traffic patterns that need to be handled by an efficient Network-on-Chip (NoC). In this paper, we present an NoC-enabled software/hardware co-design framework to implement high-performance k-mer counting. The proposed architecture enables more computational power, efficient communication between cores/memory - all without creating a thermal bottleneck; while the software component exposes more in-memory opportunities to exploit the PIM and aids in the NoC design. Experimental results show that the proposed architecture outperforms a state-of-the-art software implementation of k-mer counting utilizing Hybrid Memory Cube (HMC), by up to 7.14X, while allowing significantly higher power budgets.

The rising use of deep learning and other big-data algorithms has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. Due to the amount of data parallelism in these algorithms, high-performance three-dimensional (3D) manycore platforms that incorporate both CPUs and GPUs present a promising direction. However, as systems use heterogeneity (e.g., a combination of CPUs, GPUs, and accelerators) to improve performance and efficiency, it becomes more pertinent to address the distinct and likely conflicting communication requirements (e.g., CPU memory access latency or GPU network throughput) that arise from such heterogeneity. Unfortunately, it is difficult to quickly explore the hardware design space and choose appropriate tradeoffs between these heterogeneous requirements. To address these challenges, we propose the design of a 3D Network-on-Chip (NoC) for heterogeneous manycore platforms that considers the appropriate design objectives for a 3D heterogeneous system and explores various tradeoffs using an efficient machine learning (ML)-based multi-objective optimization (MOO) technique. The proposed design space exploration considers the various requirements of its heterogeneous components and generates a set of 3D NoC architectures that efficiently trades off these design objectives. Our findings show that by jointly considering these requirements (latency, throughput, temperature, and energy), we can achieve 9.6 percent better Energy-Delay Product on average at nearly iso-temperature conditions when compared to a thermally-optimized design for 3D heterogeneous NoCs. More importantly, our results suggest that our 3D NoCs optimized for a few applications can be generalized for unknown applications as well. Our results show that these generalized 3D NoCs only incur a 1.8 percent (36-tile system) and 1.1 percent (64-tile system) average performance loss compared to application-specific NoCs.

The growing needs of emerging applications has posed significant challenges for the design of optimized manycore systems. Network-on-Chip (NoC) enables the integration of a large number of processing elements (PEs) in a single die. To design optimized manycore systems, we need to establish suitable trade-offs among multiple objectives including power, performance, and thermal. Therefore, we consider multi-objective design space exploration (MO-DSE) problems arising in the design of NoC-enabled manycore systems: placement of PEs and communication links to optimize two or more objectives (e.g., latency, energy, and throughput). Existing algorithms to solve MO-DSE problems suffer from scalability and accuracy challenges as size of the design space and the number of objectives grow. In this paper, we propose a novel framework referred as Multi-Objective Optimistic Search (MOOS) that performs adaptive design space exploration using a data-driven model to improve the speed and accuracy of multi-objective design optimization process. We apply MOOS to design both 3D heterogeneous and homogeneous manycore systems using Rodinia, PARSEC, and SPLASH2 benchmark suites. We demonstrate that MOOS improves the speed of finding solutions compared to state-of-the-art methods by up to 13X while uncovering designs that are up to 20% better in terms of NoC. The optimized 3D manycore systems improve the EDP up to 38% when compared to 3D mesh-based designs optimized for the placement of PEs.

Monolithic-3D-integration (M3D) improves the performance and energy efficiency of 3D ICs over conventional through-silicon-vias-based counterparts. The smaller dimensions of monolithic inter-tier vias offer high-density integration, the flexibility of partitioning logic blocks across multiple tiers, and significantly reduced total wire-length enable high-performance and energy-efficiency. However, the performance of M3D ICs degrades due to the presence of electrostatic coupling when the inter-layer-dielectric thickness between two adjacent tiers is less than 50nm. In this work, we evaluate the performance of an M3D-enabled Network-on-chip (NoC) architecture in the presence of electrostatic coupling. Electrostatic coupling induces significant delay and energy overheads for the multi-tier NoC routers. This in turn results in considerable performance degradation if the NoC design methodology does not incorporate the effects of electrostatic coupling. We demonstrate that electrostatic coupling degrades the energy-delay-product of an M3D NoC by 18.1% averaged over eight different applications from SPLASH-2 and PARSEC benchmark suites. As a countermeasure, we advocate the adoption of electrostatic coupling-aware M3D NoC design methodology. Experimental results show that the coupling-aware M3D NoC reduces performance penalty by lowering the number of multi-tier routers significantly.

I. VLSI Systems: A Glance Into The Last Decades Since their inception in 1970s, VLSI systems have enabled several new technological capabilities and made them accessible to an unceasingly wider range of users, reaching a scale that has been exponentially increasing over the decades[1](seeFig. 1). Relentless integration of more complex systems has driven such remarkable evolution, as made possible by the inexorable miniaturization. As shown inFig. 1, more functionality has been crammed in a consistently smaller form factor, as exemplified by the physical volume shrinking of computers by 100 X/decade[2],[3]. At the same time, the energy per task has been decreasing at 10–100 X/decade, as shown inFig. 2, for several systems and system-on-chip subsystems[4]. This allowed packing more capabilities into the same power envelope, as generally observed in the electronic systems, even before the advent of the integrated circuit[5].

This paper presents a dual-output switchedcapacitor voltage regulator (DOSCVR) with a new flying capacitor crossing technique (FCCT) to improve power efficiency. With the FCCT, the power loss of the DOSCVR is reduced by 28%, increasing the power efficiency by 7% compared with the power efficiency of the conventional method that generates two voltages using independent single-output SCVRs. This enhanced efficiency results in no area penalty since the total flying capacitance remains unchanged. A prototype is then implemented in a 0.13-μm CMOS process to validate the proposed scheme. The prototype simultaneously produces two regulated voltages of 0.44 and 0.88 V from an input voltage of 1.5 V. The output voltages are regulated by low-bound regulation with frequency modulation. The maximum output current is 12 mA with a total flying capacitance of 0.98 nF, and the measured peak efficiency is 78%.

The complexity of heterogeneous mobile platforms is growing at a rate faster than our ability to manage them optimally at runtime. For example, state-of-the-art systems-on-chip (SoCs) enable controlling the type (Big/Little), number, and frequency of active cores. Managing these platforms becomes challenging with the increase in the type, number, and supported frequency levels of the cores. However, existing solutions used in mobile platforms still rely on simple heuristics based on the utilization of cores. This paper presents a novel and practical imitation learning (IL) framework for dynamically controlling the type (Big/Little), number, and the frequencies of active cores in heterogeneous mobile processors. We present efficient approaches for constructing an Oracle policy to optimize different objective functions, such as energy and performance per Watt (PPW). The Oracle policies enable us to design low-overhead power management policies that achieve near-optimal performance matching the Oracle. Experiments on a commercial platform with 19 benchmarks show on an average 101% PPW improvement compared to the default interactive governor.

3D Network-on-Chip (NoC) architectures are capable of achieving better performance and lower energy consumption compared to their planar counterparts. However, conventional 3D NoCs are not efficient in handling collective communication. Existing works mainly explore Path and Tree multicast distribution schemes for 3D NoCs. However, both these mechanisms involve high network latency and lack scalability. In this work, we propose a SMART (Single-cycle Multi-hop Asynchronous Repeated Traversal) 3D NoC architecture that is capable of achieving high-performance collective communication. The proposed High-Performance SMART (HP-SMART) 3D NoC achieves 65% and 31% latency improvements compared to the existing Path and Tree multicast-based 3D NoCs respectively. HP-SMART 3D NoC also achieves significant improvement in message latency compared to its 2D counterpart.

The widespread adoption of big data has led to the search for highperformance and low-power computational platforms. Emerging heterogeneous manycore processing platforms consisting of CPU and GPU cores along with various types of accelerators offer power and area-efficient trade-offs for running these applications. However, heterogeneous manycore architectures need to satisfy the communication and memory requirements of the diverse computing elements that conventional Network-on-Chip (NoC) architectures are unable to handle effectively. Further, with increasing system sizes and level of heterogeneity, it becomes difficult to quickly explore the large design space and establish the appropriate design trade-offs. To address these challenges, machine learning-inspired heterogeneous manycore system design is a promising research direction to pursue. In this paper, we highlight various salient features of heterogeneous manycore architectures enabled by emerging interconnect technologies and machine learning techniques.

In the emerging data-driven science paradigm, computing syStems ranging from IoT and mobile to manycores and datacenters play distinct roles. These systems need to be optimized for the objectives and constraints dictated by the needs of the application. In this paper, we describe how machine learning techniques can be leveraged to improve the computational-efficiency of hardware design optimization. This includes generic methodologies that are applicable for any hardware design space. As an example, we discuss a guided design space exploration framework to accelerate application-specific manycore systems design and advanced imitation learning techniques to improve on-chip resource management. We present some experimental results for application-specific manycore system design optimization and dynamic power management to demonstrate the efficacy of these methods over traditional EDA approaches.

We set up a security analysis framework by aging the Network-on-Chip (NoC) to study planned obsolescence by the original equipment manufacturer (OEM). An NoC is the communication backbone in a manycore System-on-Chip (SoC). Planned obsolescence may adopt any vulnerability in the NoC to cause the SoC to fail. We show how an OEM can craft workloads to generate electromigration-induced stress and crosstalk noise in TSV-based vertical links in the NoC to hasten failure. We analyzed three malicious workloads and confirm that a crafted workload that injects 3-10% more traffic on to a few selected critical vertical links can shorten the lifetime of the NoC by 11%-25% averaged over the benchmarks considered in this work.

The emerging field of NoC with Wireless interconnects is actively being pursued by a number of researchers worldwide, from a variety of different perspectives, ranging from very high levels of abstraction (e.g., system architecture) to very low levels (physical layer and transceiver design). Successful solutions will likely adopt and encompass elements from all or at least several levels of abstraction and rely on interdisciplinary concepts from multi-core architectures, integrated circuits, 3D ICs, digital communications, complex networks, and optimization techniques. This special session will provide a timely and insightful journey into various challenges and emerging solutions regarding the design of future NoC architectures. By scope and contents, this special session represents an engaging proposition to attendees belonging to both academia and industry.

High-performance and energy-efficient Network-on-Chip (NoC) architecture is one of the crucial components of the manycore processing platforms. A very promising NoC architecture recently proposed in the literature is the three-dimensional small-world NoC (3D SWNoC). Due to short vertical links in 3D integration and the robustness of small-world networks, the 3D SWNoC architecture outperforms its other 3D counterparts. However, the performance of 3D SWNoC is highly dependent on the placement of the links and associated routers. In this article, we propose a sensitivity-based link placement algorithm (SEN) to optimize the performance of 3D SWNoC. The sensitivity of a link in a NoC measures the importance of the link. The SEN algorithm optimizes the performance of 3D SWNoC by calculating the sensitivities of all the links in the NoC and removing the least important link repeatedly. We compare the performance of SEN algorithm with simulated annealing- (SA) and recently proposed machine-learning-based (ML) optimization algorithm. The optimized 3D SWNoC obtained by the proposed SEN algorithm achieves, on average, 11.5% and 13.6% lower latency and 18.4% and 21.7% lower energy-delay product than those optimized by the SA and ML algorithms respectively. In addition, the SEN algorithm is 26 to 33 times faster than the SA algorithm for the optimization of 64-, 128-, and 256-core 3D SWNoC designs. The performance gain provided by the SEN-, SA-, and ML-based methods also depend on the characteristics of the benchmarks under consideration. If the traffic pattern generated by a benchmark does not have enough variation, then the ML-based method does not have adequate opportunity to optimize the network. However, we find that ML-based methodology has faster convergence time than SEN and SA for bigger systems. The ML-based optimization algorithm is almost 4 and 97 times faster than the SEN- and SA-based algorithm for a system with 256 cores.

Convolutional Neural Networks (CNNs) have shown a great deal of success in diverse application domains including computer vision, speech recognition, and natural language processing. However, as the size of datasets and the depth of neural network architectures continue to grow, it is imperative to design high-performance and energy-efficient computing hardware for training CNNs. In this paper, we consider the problem of designing specialized CPU-GPU based heterogeneous manycore systems for energy-efficient training of CNNs. It has already been shown that the typical on-chip communication infrastructures employed in conventional CPU-GPU based heterogeneous manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. To address this issue, we first analyze the on-chip traffic patterns that arise from the computational processes associated with training two deep CNN architectures, namely, LeNet and CDBNet, to perform image classification. By leveraging this knowledge, we design a hybrid Network-on-Chip (NoC) architecture, which consists of both wireline and wireless links, to improve the performance of CPU-GPU based heterogeneous manycore platforms running the above-mentioned CNN training workloads. The proposed NoC achieves 1.8× reduction in network latency and improves the network throughput by a factor of 2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC. For the considered CNN workloads, these network-level improvements translate into 25 percent savings in full-system energy-delay-product (EDP). This demonstrates that the proposed hybrid NoC for heterogeneous manycore architectures is capable of significantly accelerating training of CNNs while remaining energy-efficient.

Deep neural networks have seen tremendous success for different modalities of data including images, videos, and speech. This success has led to their deployment in mobile and embedded systems for real-time applications. However, making repeated inferences using deep networks on embedded systems poses significant challenges due to constrained resources (e.g., energy and computing power). To address these challenges, we develop a principled co-design approach. Building on prior work, we develop a formalism referred as coarse-to-fine networks (C2F Nets) that allow us to employ classifiers of varying complexity to make predictions. We propose a principled optimization algorithm to automatically configure C2F Nets for a specified tradeoff between accuracy and energy consumption for inference. The key idea is to select a classifier on-the-fly whose complexity is proportional to the hardness of the input example: simple classifiers for easy inputs and complex classifiers for hard inputs. We perform comprehensive experimental evaluation using four different C2F Net architectures on multiple real-world image classification tasks. Our results show that optimized C2F Net can reduce the energy delay product by 27% to 60% with no loss in accuracy when compared to the baseline solution, where all predictions are made using the most complex classifier in C2F Net.

Three-dimensional (3D) integration enables the design of high-performance and energy-efficient network on chip (NoC) architectures as communication backbones for manycore chips. To exploit the benefits of the vertical dimension of 3D integration, through-silicon-via (TSV) has been predominantly used in state-of-the-art manycore chip design. However, for TSV-based systems, high power density and the resultant thermal hotspot remain major concerns from the perspectives of chip functionality and overall reliability. The power consumption and thermal profiles of 3D NoCs can be improved by incorporating a Voltage-Frequency-Island (VFI)-based power management strategy. However, due to inherent thermal constraints of a TSV-based 3D system, we are unable to fully exploit the benefits offered by the power management methodology. In this context, emergence of monolithic 3D (M3D) integration has opened up new possibility of designing ultra-low-power and high-performance circuits and systems. The smaller dimensions of the inter-layer dielectric (ILD) and monolithic inter-tier vias (MIVs) offer high-density integration, flexibility of partitioning logic blocks across multiple tiers, and significant reduction of total wire-length. In this work, we present the first-ever study of the performance-thermal tradeoffs for energy efficient monolithic 3D manycore chips. In particular, we present a comparative performance evaluation of M3D NoCs with respect to their conventional TSV-based counterparts. We demonstrate that the proposed M3D-based NoC architecture incorporating VFI-based power management achieves a maximum of 29.4% lower energy-delay-product (EDP) compared to the TSV-based designs for a large set of benchmarks. We also demonstrate that the M3D-based NoC shows up to 29.1% lower maximum temperature than the TSV-based counterpart for these benchmarks.

A new reference-spur cancelation technique is presented for supply-regulated ring-oscillator-based integer-N phaselocked loops (PLLs). A passive RC filter is used to implement a feed-forward (FF) spur-coupling path to perform spur cancelation at the PLL control signal. The proposed technique achieves a simulated spur cancelation of about 22 dB at the first spur harmonic. The simulated postcancelation spur value is -79 dBc for an oscillator gain of 0.1 GHz/V and -46 dBc for an oscillator gain of 6 GHz/V. Spur cancelation is also robust against large process, voltage, and temperature variations in the gain and bandwidth of the FF path. A 1-GHz integerN PLL prototype in a 65-nm CMOS process has a measured cancelation of 19.5 and 13 dB at the first and the second spur harmonic, respectively, with 320 μW of total power consumption. The PLL prototype has an oscillator gain of 1.5 GHz/V, which results in a postcancelation spur of -53 dBc. The proposed zero-power technique is suitable for low-power PLLs as it achieves a large spur cancelation without requiring any additional power consumption or calibration.

Wireless interconnects using near-field inductive coupling (NFIC) enables contactless vertical communications necessary for the design of energy efficient and robust 3-D manycore systems. However, the achievable performance, energy efficiency, bandwidth, and associated area overhead of NFICs are intertwined imposing significant design challenges and tradeoffs to explore the optimum link configuration. To address these challenges, in this paper, we propose a holistic design approach for exploring energy-efficient NFICs and target to exploit the benefits of the NFICs in the context of efficient and reliable network-on-chip (NoC) design. The proposed design framework employs statistical link analysis to select optimum NFIC-link configuration and is significantly more efficient in terms of energy efficiency and area overhead compared to the state-of-the-art counterparts. We demonstrate that 3-D NoCs incorporating NFIC-enabled links outperform through-silicon-via (TSV) counterparts. In addition, the overall reliability of TSV- and NFIC-enabled hybrid 3-D NoC is significantly better than only TSV-based NoCs in order to counteract the electromigration and workload-induced stress challenges.

Community detection is an advanced graph operation that is used to reveal tightly-knit groups of vertices (aka. communities) in real-world networks. Given the intractability of the problem, efficient heuristics are used in practice. Yet, even the best of these state-of-the-art heuristics can become computationally demanding over large inputs and can generate workloads that exhibit inherent irregularity in data movement on manycore platforms. In this paper, we posit that effective acceleration of the graph community detection operation can be achieved by reducing the cost of data movement through a combined innovation at both software and hardware levels. More specifically, we first propose an efficient software-level parallelization of community detection that uses approximate updates to cleverly exploit a diminishing returns property of the algorithm. Secondly, as a way to augment this innovation at the software layer, we design an efficient Wireless Network on Chip (WiNoC) architecture that is suited to handle the irregular on-chip data movements exhibited by the community detection algorithm under both unicast- and broadcast-heavy cache coherence protocols. Experimental results show that our resulting WiNoC-enabled manycore platform achieves on average 52% savings in execution time, without compromising on the quality of the outputs, when compared to a traditional manycore platform designed with a wireline mesh NoC and running community detection without employing approximate updates.

A 3D network-on-chip (3D NoC) is an enabler for the design of high-performance and energy-efficient manycore chips. Most popular 3D NoCs utilize the Through-Silicon-Via (TSV)-based vertical links (VLs) as the communication pillars between the planar dies. However, the TSVs in a 3D NoC may fail due to both workload-induced stress and crosstalk capacitance. This failure negatively affects the overall achievable performance of the 3D NoC. In this work, we analyze the joint effects of workload-induced stress and crosstalk on the TSV mean-time-to-failure (MTTF) and hence the 3D NoC lifetime. We demonstrate that if we only consider the effects of electromigration on the TSVs due to workload-induced stress then the estimated MTTF and the subsequently lifetime of 3D NoC are too optimistic. Due to the combined effects of workload and crosstalk noise, the lifetime of 3D NoC reduces significantly. Subsequently, we demonstrate that a spare TSV allocation methodology considering the joint effects of workload and crosstalk noise enhances the lifetime of the 3D NoC by a factor of 4.6 compared to when only the workload is considered for a given spare budget of 5%.

SMART (Single-Cycle Multi-hop Asynchronous Repeated Traversal) NoC architectures enable single cycle data transfers, even between the physically far apart nodes. However, enabling single cycle hops over long distance restricts the achievable clock frequency of the system. In other words, increasing the NoC clock frequency lowers the number of hops that can be traversed in a single-cycle in a conventional SMART NoC. In this work, we demonstrate that by integrating wireless links and a novel look-ahead request mechanism in the SMART NoC, it is possible to enable low-latency and energy efficient data transfers, even when the system is designed with high clock frequencies. For the various applications considered in this work, the wireless-enabled SMART (WiSMART) NoC achieves on an average 33% reduction in message latency compared to the wireline SMART NoC. This network level improvement translates into 16% savings in full system energy-delay-product.

3D integration, a breakthrough technology to achieve "More Moore and More Than Moore," provides numerous benefits such as better performance, lower power consumption, and wide bandwidth by vertical interconnects and 3D stacking. These vertical interconnects enable design of high performance 3D Network-on-Chip (NoC) as a communication backbone for massive manycore platforms. However, existing 3D NoCs are still bottlenecked due to simple extension of 2D architectures without fully exploiting the advantages of the 3D integration. Moreover, the anticipated performance gain of 3D NoC-enabled manycore chips will be compromised due to potential failures of through silicon vias (TSVs) that are predominantly used as vertical interconnects. To address these problems, we explore a holistic design methodology starting from the physical layer to the overall interconnection architecture where the vertical data exchange takes place through contactless links using near field inductive coupling (NFIC).

Emergence of monolithic 3D (M3D) integration has opened up the possibility of designing the ultra-low-power and high-performance circuits and systems. The smaller dimensions of monolithic inter-tier vias (MIVs) offer high density integration, the flexibility of partitioning logic blocks across multiple tiers, and significantly reduced total wire-length. In this work, we explore the design space of M3D-enabled energy-efficient NoC architectures and present a comparative performance evaluation with TSV-based counterparts. We describe the optimization of the link and router placements of the M3D-enabled NoC to ensure maximum achievable performance. The placement of M3D-enabled routers and links are explored using a machine-learning-inspired optimization algorithm. The proposed M3D-enabled NoC architecture achieves 32% lower energy-delay-product (EDP) compared to the conventional mesh-based counterpart. We also demonstrate that for the diverse set of benchmarks considered in this work, the M3D-enabled NoC, on an average, achieves 28% lower EDP than the TSV-based counterpart.

Near Field Inductive Coupling (NFIC) enables design of energy efficient and robust three-dimensional (3D) manycore systems. The associated design challenges and the trade-offs of the NFIC-based vertical links depend on achievable data-rates, energy and area overheads. In this work, we propose a holistic design flow that explores optimum energy and area efficient NFIC-link design as a communication backbone in a 3D manycore chip. Moreover, the design framework employs statistical link analysis to select optimum NFIC link configuration. The proposed NFIC-link design is significantly more efficient in terms of energy efficiency and area overhead compared to state-of-the-art counterpart. Energy efficiency and resiliency of NFIC-links are exploited in the context of a 3D NoC design. We demonstrate that overall reliability of the NFIC-enabled 3D NoC is significantly better compared to a conventional stand-alone TSV-based architecture.

As deep learning technology is increasingly employed in diverse applications domains, the demand for computational power to enable these algorithms also increases. In this respect, high-performance three-dimensional (3D) heterogeneous manycore systems present a promising direction. However, deep learning on these systems pose several design challenges. First, the network-on-chip (NoC) must handle the traffic requirements of both CPU and GPU communications. Second, 3D system designs must address thermal issues resulting from high-power density. In this work, we propose a design methodology for a heterogeneous 3D NoC architecture that not only satisfies the traffic requirements of both CPUs and GPUs, but also reduces thermal hotspots. To this end, we target the training of two widely employed convolutional neural networks (CNN), namely, LeNet and CIFAR. By using our joint performance-thermal optimization methodology to create a 3D NoC for training CNNs, we reduce the maximum temperature by 22% while incurring only 5% full-system energy-delay-product degradation over a solely performance optimized 3D NoC. This demonstrates that, our design methodology achieves considerable temperature reduction with negligible loss in performance.

Understanding the role of competition and cooperation among multiple interacting species of microorganisms that constitute the microbiome and decipher how they enforce homeostasis or trigger diseases requires the development of multi-scale computational models capable of capturing both intra-cell processing (i.e., gene-to-protein interactions) and inter-cell interactions. The multi-scale interdependency that governs the interactions from genes to proteins within a cell and from molecular messengers to cells to microbial communities within the environment raises numerous computation and communication challenges. Internal cell processing cannot be simulated without knowledge of the surroundings. Similarly, cell-cell communication cannot be fully abstracted without stated of internal processing and diffusion effects of molecular messengers. To address the computeand communication-intensive nature of modeling microbial communities, in this paper, we propose a novel reconfigurable NoC-based manycore architecture capable of simulating a large scale microbial community. The reconfiguration of the NoC topology is achieved through the fractal analysis of NoC traffic and use of the on-chip wireless interfaces. More precisely, we analyze the computational and communication workloads and exploit the observed fractal characteristics for proposing a mathematical strategy for NoC reconfiguration. Experimental results demonstrate that the proposed NoC architecture achieves 56.6 and 62.8 percent improvement in energy delay product over the conventional wireline mesh and flatten butterfly-based high radix NoC architectures, respectively.

A 3-D network-on-chip (NoC) enables the design of high performance and low power many-core chips. Existing 3-D NoCs are inadequate for meeting the ever-increasing performance requirements of many-core processors since they are simple extensions of regular 2-D architectures and they do not fully exploit the advantages provided by 3-D integration. Moreover, the anticipated performance gain of a 3-D NoC-enabled many-core chip may be compromised due to the potential failures of through-silicon-vias that are predominantly used as vertical interconnects in a 3-D IC. To address these problems, we propose a machine-learning-inspired predictive design methodology for energy-efficient and reliable many-core architectures enabled by 3-D integration. We demonstrate that a small-world network-based 3-D NoC (3-D SWNoC) performs significantly better than its 3-D MESH-based counterparts. On average, the 3-D SWNoC shows 35% energy-delay-product improvement over 3-D MESH for the PARSEC and SPLASH2 benchmarks considered in this paper. To improve the reliability of 3-D NoC, we propose a computationally efficient spare-vertical link (sVL) allocation algorithm based on a state-space search formulation. Our results show that the proposed sVL allocation algorithm can significantly improve the reliability as well as the lifetime of 3-D SWNoC.

The emergence of 3D network-on-chip (NoC) has revolutionized the design of high-performance and energy-efficient manycore chips. However, the anticipated performance gain can be compromised due to the degradation and failure of vertical links (VLs). The Through-Silicon-Via (TSV)-enabled VLs may fail due to workload-induced stress; the failure of a VL can affect the neighboring VLs, thereby causing a cascade of failures and reducing the lifetime of the chip. To enhance the reliability of 3D NoC-enabled manycore chips, we propose to incorporate a voltage-frequency island (VFI)-based power management strategy that helps to reduce the energy consumption and hence, the workload-induced stress of the highly utilized VLs. The adopted power-management strategy relies on control decisions about the voltage/frequency (V/F) levels on VLs. We demonstrate that compared to the well-known spare TSV allocation and adaptive routing strategies, power management is more effective in enhancing the reliability of a 3D NoC. VFI-based power management improves the reliability of the 3D NoC by one order of magnitude compared to both adaptive routing and spare allocation while running popular SPLASH-2 and PARSEC benchmarks. The principal benefit of power management is that it is capable of reducing the operating temperature of the system, which in turn enhances the Mean-Time-To-Failure (MTTF) of the VLs and reliability of the overall 3D NoC.

As I start my second two-year term (2017–2018) as the Editor-in-Chief (EIC) of the IEEE Transactions on Very Large Scale Integration Systems (TVLSI), I wish the TVLSI readership a very happy new year and continued professional success. It gives me great pleasure to report on the state of the journal and our performance metrics. Over the past two years, TVLSI has seen a healthy increase in the number of submissions—from 687 in 2014 to 770 in 2015, and at the time of writing of this editorial, we are at 760 submissions for 2016. We expect the number of submissions for 2016 to cross 800 before the end of the year. TVLSI, therefore, continues to be the premier archival journal for university researchers and industry practitioners in the broad area of VLSI system design.

Traditional network-on-chip (NoC) interconnects follow conventional packet switching architectures that require multiple cycles to traverse each router hop. In addition, commonly used NoCs lack low-latency multicast replication and acknowledgment aggregation mechanisms that are required to efficiently handle the collective communication requirements exhibited by many modern applications. To address these issues, the single-cycle multihop asynchronous repeated traversal (SMART) NoC architecture is proposed. By using an efficient router-bypass mechanism, the SMART NoC enables single-cycle data transfers even between physically distant on-chip routers. However, enabling single-cycle hops over long metal wires restrict the achievable clock frequency of the system. In other words, increasing the NoC clock frequency lowers the number of hops that can be bypassed in a single cycle in a fully wireline SMART NoC. In this paper, we demonstrate that by integrating on-chip wireless links, a novel look-ahead bypass-request mechanism and a congestion-free broadcast routing methodology, it is possible to enable low-latency and energy efficient data transfers in the SMART NoC even when the system is designed with high clock frequencies. For the various applications considered in this paper, the wireless-enabled SMART NoC achieves on an average 41% reduction in message latency compared to the wireline SMART NoC. This network level improvement translates into 20% savings in full system energy-delay product.

Today's multiprocessor platforms employ the network-on-chip (NoC) architecture as the preferable communication backbone. Conventional NoCs are designed predominantly for unicast data exchanges. In such NoCs, the multicast traffic is generally handled by converting each multicast message to multiple unicast transmissions. Hence, applications dominated by multicast traffic experience high queuing latencies and significant performance penalties when running on systems designed with unicast-based NoC architectures. Various multicast mechanisms such as XY-tree multicast and path multicast have already been proposed to enhance the performance of the traditional wireline mesh NoC incorporating multicast traffic. However, even with such added features, the multihop nature of the wireline mesh NoC leads to high network latencies and thus limits the achievable system performance. In this paper, to sustain the high-bandwidth and high-throughput requirements of emerging applications, we propose the design of a wireless NoC (WiNoC) architecture incorporating necessary multicast support. By integrating congestion-aware multicast routing with network coding, the WiNoC is able to efficiently handle heavy multicast injections. For applications running with a broadcast-heavy Hammer cache coherence protocol, the proposed multicast-aware WiNoC achieves an average of 47% reduction in message latency compared with the XY-tree-based multicast-aware mesh NoC. This network level improvement translates into a 26% saving in full-system energy delay product.

Manycore chips are widely employed in high-performance computing and large-scale data analysis. However, the design of high-performance manycore chips is dominated by power and thermal constraints. In this respect, voltage-frequency island (VFI) is a promising design paradigm to create scalable energy-efficient platforms. By dynamically tailoring the voltage and frequency of each island, we can further improve the energy savings within given performance constraints. Inspired by the recent success of imitation learning (IL) in many application domains and its significant advantages over reinforcement learning (RL), we propose the first architecture-independent IL-based methodology for dynamic VFI (DVFI) control in manycore systems. Due to its popularity in the EDA community, we consider an RL-based DVFI control methodology as a strong baseline. Our experimental results demonstrate that IL is able to obtain higher quality policies than RL (on average, 5% less energy with the same level of performance) with significantly less computation time and hardware area overheads (3.1X and 8.8X, respectively).

Continuing progress and integration levels in silicon technologies make possible complete end-user systems consisting of extremely high number of cores on a single chip targeting either embedded or high-performance computing. However, without new paradigms of energy- and thermally-efficient designs, producing information and communication systems capable of meeting the computing, storage and communication demands of the emerging applications will be unlikely. The broad topic of power and thermal management of massive multicore chips is actively being pursued by a number of researchers worldwide, from a variety of different perspectives, ranging from workload modeling to efficient on-chip network infrastructure design to resource allocation. Successful solutions will likely adopt and encompass elements from all or at least several levels of abstraction. Starting from these ideas, we consider a holistic approach in establishing the Power-Thermal-Performance (PTP) trade-offs of massive multicore processors by considering three inter-related but varying angles, viz., on-chip traffic modeling, novel Networks-on-Chip (NoC) architecture and resource allocation/mapping

In recent years, designing specialized manycore heterogeneous architectures for deep learning kernels has become an area of great interest. However, the typical on-chip communication infrastructures employed on conventional manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. Hence, in this paper, our aim is to enhance the performance of heterogeneous manycore architectures through the design of a hybrid NoC consisting of both wireline and wireless links. To this end, we specifically target the resource-intensive backpropagation algorithm commonly used as the training method in deep learning. For backpropagation, the proposed hybrid NoC achieves 1.9X reduction in network latency and improves the network throughput by a factor of 2 with respect to a highly optimized mesh NoC. These network level improvements translate into 25% savings in full system energy-delay-product (EDP). This demonstrates the capability of the proposed hybrid and heterogeneous manycore architecture in accelerating deep learning kernels in an energy-efficient manner.

The Network-on-Chip (NoC) paradigm has emerged as an enabler for integrating a large number of embedded cores in a single die. Three-dimensional (3D) integration, a breakthrough technology to achieve “More Moore and More Than Moore,” provides numerous benefits e.g., better performance, lower power consumption, and higher bandwidth, by utilizing vertical interconnects and 3D stacking. Energy-efficient and high-bandwidth vertical interconnects enable the design of an energy efficient 3D NoC for massive manycore platforms. Existing 3D NoCs are deficient for meeting ever-increasing performance requirements of manycore processors since they are simple extension of regular 2D architectures and they do not fully exploit the advantages provided by 3D integration. Moreover, the anticipated performance gain of a 3D NoC-enabled manycore chip will be compromised due to the potential failures of through-silicon-vias (TSVs) that are predominantly used as vertical interconnects in a 3D IC. In this paper, we present the various challenges and possible solutions for designing energy-efficient and reliable manycore chips enabled by the 3D integration.

Multiple Voltage Frequency Island (VFI)-based designs can reduce the energy dissipation in multicore chips. Indeed, by tailoring the voltages and frequencies of each VFI domain, we can achieve significant energy savings subject to specific performance constraints. The achievable performance of VFI-based multicore platforms depends on the overall communication backbone, which relies predominantly on Networks-on-Chip (NoCs). Traditionally mesh-based NoCs have been used in VFI-based systems. However, the mesh-based NoCs have large latency and energy overheads due to their inherently long multihop paths. Emerging paradigms such as the millimeter (mm)-wave small-world wireless Networks-on-Chip (mSWNoCs) have lately been observed to help reduce the impact of the communication backbone on the performance of the multicore chips. In this work, we demonstrate that not only do mSWNoC-enabled VFI designs mitigate some of the full-system performance degradation inherent in VFI-partitioned multicore designs, but they also help in eliminating it entirely for certain applications. We also demonstrate that the VFI-partitioned designs used in conjunction with a novel NoC architecture like mSWNoC can achieve significant energy savings while minimizing the impact on the performance for each application under consideration.

In a multi/many-core system, the network-on-chip (NoC)-based communication backbone is responsible for a relevant fraction of the overall energy budget. Reducing the voltage swing for signaling in crossbars and links results in significant energy saving. Unfortunately, as voltage swing reduces, the bit error rate increases, that in turn compromises the communication reliability. Starting from the assumption that not all the communications need same level of reliability, in this paper we propose techniques and architectures for run-time tuning of the voltage swing of the crossbars and interrouter links. The proposed technique is compared with the state of the art in link energy reduction through data encoding under both synthetic and real traffic scenarios. We found that the proposed techniques allow to significantly reduce the energy consumption of the NoC fabric without degrading the performance metrics. Energy savings ranging from 20% to 43% have been observed without any relevant impact on the performance metrics.

With its applicability spanning numerous data-driven fields, the implementation of graph analytics on multicore platforms is gaining momentum. One of the most important components of a multicore chip is its communication backbone. Due to inherent irregularities in data movements manifested by graph-based applications, it is essential to design efficient on-chip interconnection architectures for multicore chips performing graph analytics. In this article, we present a detailed analysis of the traffic patterns generated by graph-based applications when mapped to multicore chips. Based on this analysis, we explore the design-space for the Network-on-Chip (NoC) architecture to enable an efficient implementation of graph analytics. We principally consider three types of NoC architectures, viz., traditional mesh, small-world, and high-radix networks. We demonstrate that the small-world-network-enabled wireless NoC (WiNoC) is the most suitable platform for executing the considered graph applications. The WiNoC achieves an average of 38% and 18% full-system Energy Delay Product savings compared to wireline-mesh and high-radix NoCs, respectively.

Multiple voltage frequency island (VFI)-based designs can reduce the energy dissipation in multicore platforms by taking advantage of the varying nature of the application workloads. Indeed, the voltage/frequency (V/F) levels of the VFIs can be dynamically tailored by considering the workload-driven variations in the application. Traditionally, mesh-based networks-on-chip (NoCs) have been used in VFI-based systems; however, they have large latency and energy overheads due to the inherently long multihop paths. Consequently, in this paper, we explore the emerging paradigm of wireless NoC (WiNoC) and demonstrate that by incorporating WiNoC, VFI, and dynamic V/F tuning in a synergistic manner, we can design energy-efficient multicore platforms without introducing noticeable performance penalty. Our experimental results show that for the benchmarks considered, the proposed approach can achieve between 5.7% and 46.6% energy-delay product (EDP) savings over the state-of-the-art system and 26.8% and 60.5% EDP savings over a standard baseline non-VFI mesh-based system. This opens up a new of class of codesign approaches that can make WiNoCs the communication technology of choice for future multicore platforms.

Internet-of-Things architecture aims to provide smart connectivity not only with existing computers, but also with new context-aware computing resources, extending soon beyond von Neumann devices for the purpose of mining, prediction, and control of cyber and physical components. These cyber-physical systems (CPSs) not only lead to the accumulation of large amounts of data that can be used to build comprehensive mathematical models, but also raise the quest for real-time analysis and control in diverse application domains, such as environment, healthcare, avionics, smart interconnected automobiles, and smart buildings. Endowing the CPS with a higher degree of distributed smartness and cognition (adaptation) to process massive amounts of data requires efficient control modules. In addition, the prohibitive nature of power consumption, data movement, and memory bandwidth issues calls for a shift of processing the decision-making strategies from within large supercomputing centers closer to the actual sensing site via many distributed networks-on-chip (NoCs)-based multicore platforms. Toward this end, in this paper, we propose an efficient NoC-based multicore architecture capable of solving large-scale nonlinear model predictive control (NMPC) problems. By carefully analyzing the spatiotemporal workload characteristics of the NMPC problems, we propose the design of an efficient NoC architecture. Our proposed NoC architecture achieves up to 29% improvement in latency and 28% improvement in energy dissipation over the conventional mesh NoC-based counterpart.

With its applicability spanning numerous data-driven fields, the implementation of graph analytics on multicore platforms is gaining momentum. The most important component of a multicore chip is its communication backbone. Due to the inherent irregularities in data movements manifested by graph based applications, it is essential to design an efficient on-chip interconnect for multicore chips performing graph analytics. In this paper we present a detailed analysis of the traffic patterns generated by graph-based applications when mapped to multicore chips. Based on this analysis, we present the design of wireless Network-on-Chip (WiNoC)-enabled multicore platforms for efficient implementation of graph analytics. When compared to traditional wireline mesh architecture, WiNoC enables a faster data exchange among the computing cores, leading to reduced execution times and lower energy dissipation. We demonstrate that depending on the particular graph application, the WiNoC reduces the execution time up to 35% and lowers the energy dissipation up to 40% when compared to traditional wireline mesh.

In an era when power constraints and data movement are proving to be significant barriers for high-end computing, multicore architectures offer a low-power and highly scalable platform suitable for both data- and compute-intensive applications. MapReduce is a popular framework to facilitate the management and development of big-data workloads. In this work, we demonstrate that by using a wireless NoC-enabled Voltage Frequency Island (VFI)-based multicore platform it is possible to enhance the energy efficiency of MapReduce implementations without paying significant execution time penalties. Our experimental results show that for the benchmarks considered, the designed VFI system can achieve an average of 33.7% energy-delay product (EDP) savings over the standard baseline non-VFI mesh-based system while paying a maximum of 3.22% execution time penalty.

Three dimensional (3D) Network-on-Chip (NoC) architectures enable design of low power and high performance communication fabrics for multicore chips. In spite of achievable performance benefits, 3D NoCs are still bottlenecked by the planar interconnects. To exploit the benefits introduced by the vertical dimension, it is imperative to explore novel 3D NoC architectures. In this paper, we propose design of a small-world (SW) network based 3D NoCs. We demonstrate that the proposed 3D SW NoC outperforms its conventional 3D mesh-based counterparts. On average, it provides ~25% reduction in the energy delay product (EDP) compared to 3D MESH without introducing any additional link overhead in presence of conventional SPLASH-2 and PARSEC benchmarks. The proposed 3D SW NoC is more robust in presence of TSV failures and performs better than fault-free 3D MESH even in the presence of 25% TSVs failure.

Three-dimensional (3D) Network-on-Chip (NoC) is an emerging technology that has the potential to achieve high performance with low power consumption for multicore chips. However, to fully realize their potential, we need to consider novel 3D NoC architectures. In this paper, inspired by the inherent advantages of small-world (SW) 2D NoCs, we explore the design space of SW network-based 3D NoC architectures. We leverage machine learning to intelligently explore the design space to optimize the placement of both planar and vertical communication links for energy efficiency. We demonstrate that the optimized 3D SW NoC designs perform significantly better than their 3D MESH counterparts. On an average, the 3D SW NoC shows 35% energy-delay-product (EDP) improvement over 3D MESH for the nine PARSEC and SPLASH2 benchmarks considered in this work. The highest performance improvement of 43% was achieved for RADIX. Interestingly, even after reducing the number of vertical links by 50%, the optimized 3D SW NoC performs 25% better than the fully connected 3D MESH, which is a strong indication of the effectiveness of our optimization methodology.

During the last decade, we have witnessed a major transition from computation- to communication-centric design of integrated circuits and systems. In particular, the network-on-chip (NoC) approach has emerged as the major design paradigm for multicore systems-on-chip (SoC). The major challenges in traditional wire-based NoCs are the high latency and power consumption of the multi-hop links. By inserting single-hop long-range wireless links in place of multi-hop wired links, the overall system performance can be significantly improved. We should adopt novel architectures inspired by the on-chip wireless links to design high-performance multi-core chips. In this regard, the small-world network-inspired wireless NoC (WiNoC) has emerged as an enabling interconnection infrastructure to design high-bandwidth and energy-efficient multicore chips. In this paper we present the various challenges and possible solutions for designing energy-efficient massive multicore chips enabled by the WiNoC paradigm.

In recent years, multiple Voltage Frequency Island (VFI)-based designs have increasingly made their way into both commercial and research multicore platforms. On the other hand, the wireless Network-on-Chip (WiNoC) architecture has emerged as an energy-efficient and high bandwidth communication backbone for massively integrated multicore platforms. It becomes therefore possible to exploit the small-world effects induced by the wireless links of a WiNoC to achieve efficient inter-VFI data exchanges. In this work, we demonstrate that WiNoCs can provide better latency and energy profiles compared to traditional mesh-like architecture for VFI-partitioned multicore designs. The performance gains and energy efficiency are achieved due to the low-power wireless shortcuts in conjunction with the small-world architecture. Indeed, our experimental results show energy improvements as large as 40% for multithreaded application benchmarks.

The millimeter (mm)-wave small-world wireless NoC (mSWNoC) is an enabling interconnect architecture to design high performance and low power multicore chips. As the mSWNoC has an overall irregular topology, it is extremely important to design suitable deadlock-free routing mechanisms for it. In this paper we quantify the latency, energy dissipation, and thermal profiles of mSWNoC architectures by incorporating irregular network routing strategies. We demonstrate that the latency, energy dissipation, and thermal profile are affected by the adopted routing methodologies. In presence of the benchmarks considered, the variation in latency and energy dissipation is small. However, the network hotspot temperature can vary considerably depending on the exact routing strategy and the characteristics of the benchmark.

Traditional multicore designs, based on the Network-on-Chip (NoC) paradigm, suffer from high latency, significant power consumption and temperature hotspots as the system size scales up due to the inherent multi-hop nature of the communication fabric. NoCs have been shown to achieve increased performance by inserting long-range wired links following the principles of small-world graphs [1]. Design and optimization of multi- and many-core systems on chip (SoCs) that exploit small-world effects have already been demonstrated [1]. Small-world graphs are characterized by many short-distance links between neighboring nodes as well as a few relatively long-distance direct shortcuts. These networks can be further improved by replacing the long-distance shortcuts with single-hop, energy-efficient, wireless links. Wireless NoCs (WiNoCs) with millimeter (mm)-wave wireless links working at 10-100 GHz is envisioned as an enabling technology to design low-power and high-bandwidth massive multicore architectures [2]. These mm-wave small-world NoC (mSWNoC) networks are used as the interconnection backbone in this work.

Continuing progress and unprecedented integration levels in current silicon technologies make possible complete end-user systems consisting of an extremely high number of cores integrated on a single chip for embedded or high-performance computing. However, without developing new paradigms for energy- and thermally-efficient design, meeting the computing, storage, and communication demands of the emerging applications is highly unlikely. Moreover, in order to sustain the predicted growth of number of embedded cores on a single die, it is extremely important to have a scalable, low power, and high bandwidth on-chip communication infrastructure. Towards this end, wireless Network-on-Chip (WiNoC) represents an emerging paradigm to design a low power yet high bandwidth interconnect infrastructure for multicore chips.

Traditional metal-wire-based networks-on-chip (NoC) suffer from high latency and power dissipation as the system size scales up in the number of cores. This limitation stems from the inherent multihop communication nature of larger NoCs. It has previously been shown that the performance of NoCs can be significantly improved by introducing long-range, low power, and high-bandwidth single-hop links between distant cores. While previous work has focused on specific NoC architectures and configurations, it remains an open question whether heterogeneous link types are beneficial in a broad range of NoC architectures. In this article, we show that a generic NoC architecture with heterogeneous link types allows for NoCs with higher bandwidth at a lower cost compared to homogeneous networks. We further show that such NoCs scale up significantly better in terms of performance and cost. We demonstrate these broadly-applicable results by using a technology-agnostic complex network approach that targets NoC architectures with various emerging link types.

The mm-wave small-world wireless NoC (mSWNoC) has emerged as an enabling interconnection infrastructure for designing high-bandwidth and energy-efficient multicore chips. In this mSWNoC architecture, long-range communication predominately takes place through the wireless shortcuts operating in the range of 10--100GHz, whereas short-range data exchange occurs through conventional metal wires. This results in performance advantages (lower latency and energy dissipation), mainly stemming from using the wireless links as long-range shortcuts between far-apart cores. The performance gain introduced by the wireless channels can be enhanced further if the wireline links of the mSWNoC are optimized according to the traffic patterns arising out of the application workloads. While there is significant energy savings, and hence temperature reduction, in the network due to the mSWNoC architecture, a load-imbalanced network is still susceptible to local temperature hotspots. In this work, we demonstrate that by incorporating congestion-avoidance routing with network-level dynamic voltage and frequency scaling (DVFS) in an mSWNoC, the power and thermal profiles can be improved without a significant impact on the overall network performance. In this work, we demonstrate how novel interconnect architectures enabled by the on-chip wireless links coupled with power management strategies can improve the energy and thermal characteristics of an mSWNoC significantly without introducing any performance degradation with respect to the conventional mesh-based NoC.

Wireless Network-on-Chip (WiNoC) has emerged as an enabling technology to design low power and high bandwidth massive multicore chips. WiNoCs based on small-world network architecture and designed with incorporating millimeter (mm)-wave on-chip wireless links offer significantly lower power and higher bandwidth compared to traditional mesh-based counterparts. In this mm-wave small-world WiNoC (mSWNoC), long distance communication predominately takes place through the wireless shortcuts whereas the short-range data exchange still occurs through the conventional metal wires. This results in performance advantages mainly stemming from using the wireless links as long-range shortcuts between far apart cores. This performance gain can be enhanced further if the wireline links and the processing cores of the WiNoC are optimized according to the traffic patterns and application workloads. In this work, we demonstrate that by incorporating both processor- and network-level dynamic voltage and frequency scaling (DVFS) in an mSWNoC, the power and thermal profiles can be improved without a significant impact on the overall execution time. We also show that depending on the applications, temperature hotspots can be formed either in the processing cores or in the network infrastructure. The proposed dual-level DVFS is capable of addressing both types of hotspots. In this work we will demonstrate how novel interconnect architectures enabled by the on-chip wireless links coupled with power management strategies can improve the energy and thermal characteristics of a NoC significantly.

The millimeter-wave small-world wireless network-on-chip (mSWNoC) is an enabling interconnect architecture to design high-performance and low-power multicore chips. As the mSWNoC has an overall irregular topology, it is essential to design and optimize suitable deadlock-free routing mechanisms for it. In this paper, we quantify the latency, energy dissipation, and thermal profiles of mSWNoC architectures by incorporating irregular network routing strategies. We demonstrate that the latency, energy dissipation, and thermal profile are affected by the adopted routing methodologies. The overall system performance and thermal profile are governed by the traffic-dependent optimization of the routing methods. Our aim is to establish the energy-thermal-performance trade-offs for the mSWNoC depending on the exact routing strategy and the characteristics of the benchmarks considered.

The Network-on-Chip (NoC) paradigm is used as a scalable interconnection infrastructure for multi-core chips. To enhance the performance of conventional interconnect-based multi-core chips, on-chip wireless interconnect has emerged as a radically different technology. However, this emerging interconnect paradigm imposes significant challenges pertaining to reliable integration and design. In this paper, we focus on two types of mm-wave wireless NoC architectures. One is a hierarchical architecture with long-range wireless shortcuts and the other is a power-law connectivity based small-world network without any hierarchy. We demonstrate that though the hierarchical architecture offers more bandwidth with lower energy dissipation than the small-world-based counterpart, it has significantly more area overhead. Also, the power-law connectivity based small-world wireless NoC is more robust in presence of wireless link failures.

Traditional multi-core designs, based on the Network-on-Chip (NoC) paradigm, suffer from high latency and power dissipation as the system size scales up due to the inherent multi-hop nature of communication. Introducing long-range, low power, and high-bandwidth, single-hop links between far apart cores can significantly enhance the performance of NoC fabrics. In this paper, we propose design of a small-world network based NoC architecture with on-chip millimeter (mm)-wave wireless links. The millimeter wave small-world NoC (mSWNoC) is capable of improving the overall latency and energy dissipation characteristics compared to the conventional mesh-based counterpart. The mSWNoC helps in improving the energy dissipation, and hence the thermal profile, even further in presence of network-level dynamic voltage and frequency scaling (DVFS) without incurring any additional latency penalty.

The Network-on-Chip (NoC) paradigm has emerged as a scalable interconnection infrastructure for modern multicore chips. However, with growing levels of integration, the traditional NoCs suffer from high latency and energy dissipation in on-chip data transfer due to conventional multihop metal/dielectric-based interconnects. Three-dimensional integration, on-chip photonics, RF, and wireless links have been proposed as radical low-power and low-latency alternatives to the conventional planar wire-based designs. Wireless NoCs with Carbon NanoTube (CNT) antennas are shown to outperform traditional wire-based NoCs significantly in achievable data rate and energy dissipation. However, such emerging and transformative technologies will be prone to high levels of failures due to various issues related to manufacturing challenges and integration. On the other hand, several naturally occurring complex networks such as colonies of microbes and the World Wide Web are known to be inherently robust against high rates of failures and harsh environments. This article advocates adoption of such complex network-based architectures to minimize the effect of wireless link failures on the performance of the NoC. Through cycle-accurate simulations it is shown that the wireless NoC architectures inspired by natural complex networks perform better than their conventional wired counterparts even in the presence of high degrees of link failures. We demonstrate the robustness of the proposed wireless NoC architecture by incorporating both uniform and application-specific traffic patterns.

The Network-on-chip (NoC) is an enabling technology to integrate large numbers of embedded cores on a single die. The existing methods of implementing a NoC with planar metal interconnects are deficient due to high latency and significant power consumption arising out of multihop links used in data exchange. To address these problems, we propose design of a hierarchical small-world wireless NoC architecture where the multihop wire interconnects are replaced with high-bandwidth and single-hop long-range wireless shortcuts operating in the millimeter (mm)-wave frequency range. The proposed mm-wave wireless NoC (mWNoC) outperforms the corresponding conventional wireline counterpart in terms of achievable bandwidth and is significantly more energy efficient. The performance improvement is achieved through efficient data routing and optimum placement of wireless hubs. Multiple wireless shortcuts operating simultaneously further enhance the performance, and provide an energy-efficient solution for design of communication infrastructures for multicore chips.

Traditional many-core designs based on the Network-on-Chip (NoC) paradigm suffer from high latency and power dissipation as the system size scales up due to their inherent multi-hop communication. NoC performance can be significantly enhanced by introducing long-range, low power, and high-bandwidth single-hop wireless links between far apart cores. This paper presents a design methodology and performance evaluation for a hierarchical small-world NoC with CMOS compatible on-chip millimeter (mm)-wave wireless long-range communication links. The proposed wireless NoC offers significantly higher bandwidth and lower energy dissipation compared to its conventional non-hierarchical wired counterpart in presence of both uniform and non-uniform traffic patterns. The performance improvement is achieved through efficient data routing and optimum placement of wireless hubs. Multiple wireless shortcuts operating simultaneously provide an energy efficient solution for design of many-core communication infrastructures.

Current commercial systems on chip (SoC) designs integrate an increasingly large number of pre-designed cores and their number is predicted to increase significantly in the near future. Specifically, molecular-scale computing will allow single or even multiple order-of-magnitude improvements in device densities. In the design of high-performance massive multi-core chips, power and temperature have become dominant constraints. Increased power consumption can raise chip temperature,which in turn can decrease chip reliability and performance and increase cooling costs.The new, ensuing possibilities in terms of single chip integration call for new paradigms, architectures, and infrastructures for high bandwidth and low-power interconnects. In this paper we demonstrate how small-world Network-on-Chip (NoC) architectures with long-range wireless links enable design of energy and thermally efficient sustainable multi-core platforms.

Massive levels of integration are making modern multicore chips all pervasive in several domains. High performance, robustness, and energy-efficiency are crucial for the widespread adoption of such platforms. Networks-on-Chip (NoCs) have emerged as communication backbones to enable a high degree of integration in multicore Systems-on-Chip (SoCs). Despite their advantages, an important performance limitation in traditional NoCs arises from planar metal interconnect-based multihop links with high latency and power consumption. This limitation can be addressed by drawing inspiration from the evolution of natural complex networks, which offer great performance-cost trade-offs. Analogous with many natural complex systems, future multicore chips are expected to be hierarchical and heterogeneous in nature as well. In this article we undertake a detailed performance evaluation for hierarchical small-world NoC architectures where the long-range communications links are established through the millimeter-wave wireless communication channels. Through architecture-space exploration in conjunction with novel power-efficient on-chip wireless link design, we demonstrate that it is possible to improve performance of conventional NoC architectures significantly without incurring high area overhead.

introduction Share on Introduction to the special issue on sustainable and green computing systems Editors: Partha Pratim Pande Washington State University Washington State UniversityView Profile , Amlan Ganguly Rochester Institute of Technology Rochester Institute of TechnologyView Profile Authors Info & Claims ACM Journal on Emerging Technologies in Computing SystemsVolume 8Issue 4Article No.: 26pp 1–3https://doi.org/10.1145/2367736.2367737Published:30 November 2012Publication History 1citation496DownloadsMetricsTotal Citations1Total Downloads496Last 12 Months10Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Maximum Parsimony phylogenetic tree reconstruction is based on finding the breakpoint median, given a set of species, and is represented by a bounded edge-weight graph model. This reduces the breakpoint median problem to one of solving multiple instances of the Traveling Salesman Problem (TSP), which is a classical NP-complete problem in graph theory. Exponential time algorithms that apply efficient runtime heuristics, such as branch-and-bound, to dynamically prune the search space are used to solve TSP. In this paper, we present the design and performance evaluation of a network-on-chip (NoC)-based implementation for solving TSP under the bounded edge-weight model, as used in the computation of breakpoint phylogeny. Our approach takes advantage of fine-grain parallelism from the multiple processing elements (PEs) and uses efficient NoC architecture for inter-PE communication. To accelerate the application on hardware, our PE design optimizes a particular lower bound calculation operation which typically tends to be the serial bottleneck in computation of a TSP solution. We also explore two representative NoC architectures-mesh and quad-tree-and show that the latter is more energy-efficient for this application domain. Experimental results show that this new implementation is able to achieve speedups of up to three orders of magnitude over state-of-the-art multithreaded software implementations.

In phylogenetic inference, which aims at finding a phylogenetic tree that best explains the evolutionary relationship among a given set of species, statistical estimation approaches such as maximum likelihood (ML) and Bayesian inference provide more accurate estimates than other nonstatistical approaches. However, the improved quality comes at a higher computational cost, as these approaches, even though heuristic driven, involve optimization over multidimensional real continuous space. The number of possible search trees in ML is at least exponential, thereby making runtimes on even modest-sized datasets to clock up to several million CPU hours. Evaluation of these trees, involving node-level likelihood vector computation and branch-length optimization, can be partitioned into tasks (or kernels), providing the application with the potential to benefit from hardware acceleration. The range of hardware acceleration architectures tried so far offer limited degree of fine-grain parallelism. Network-on-chip (NoC) is an emerging paradigm that can efficiently support integration of massive number of cores on a chip. In this paper, we explore the design and performance evaluation of 2-D and 3-D NoC architectures for RAxML, which is one of the most widely used ML software suites. Specifically, we implement the computation kernels of the top three functions consuming more than 85% of the total software runtime. Simulations show that through appropriate choice of NoC architecture, and novel core design, allocation and placement strategies, our NoC-based implementation can achieve individual function-level speedups of 390x to 847x, speed up the targeted kernels in excess of 6500x, and provide end-to-end runtime reductions up to 5x over state-of-the-art multithreaded software.

While traditional cluster computers are more constrained by power and cooling costs for solving extreme-scale (or exascale) problems, the continuing progress and integration levels in silicon technologies make possible complete end-user systems on a single chip. This massive level of integration makes modern multicore chips all pervasive in domains ranging from climate forecasting and astronomical data analysis, to consumer electronics, smart phones, and biological applications. Consequently, designing multicore chips for exascale computing while using the embedded systems design principles looks like a promising alternative to traditional cluster-based solutions. This paper aims to present an overview of new, far-reaching design methodologies and run-time optimization techniques that can help breaking the energy efficiency wall in massively integrated single-chip computing platforms.

The Network-on-Chip (NoC) paradigm has emerged as a scalable interconnection infrastructure for modern multi-core chips. However, with growing levels of integration, the traditional NoCs suffer from high latency and energy dissipation in on-chip data transfer due to conventional metal/dielectric based interconnects. Three-dimensional integration, on-chip photonic, RF and wireless links have been proposed as radical low-power and low-latency alternatives to the conventional planar wire-based designs. Wireless NoCs with Carbon Nanotube (CNT) antennas are shown to outperform traditional wire based NoCs by several orders of magnitude in power dissipation and latency. However such transformative technologies will be prone to high levels of faults and failures due to various issues related to manufacturing and integration. On the other hand, several naturally occurring complex networks such as colonies of microbes and the internet are known to be inherently fault-tolerant against high rates of failures and harsh environments. This paper proposes to adopt such complex network based architectures to minimize the effect of wireless link failures on the performance of the NoC. Through cycle accurate simulations it is shown that the wireless NoC architectures inspired by natural complex networks perform better than their conventional wired counterparts even in the presence of a high degree of faults.

Probability-based approaches for phylogenetic inference, like Maximum Likelihood (ML) and Bayesian Inference, provide the most accurate estimate of evolutionary relationships among species. But they come at a high algorithmic and computational cost. Network-on-chip (NoC), being an emerging paradigm, has not been explored yet to achieve fine-grained parallelism for these applications. In this paper, we present the design and performance evaluation of an NoC architecture for RAxML, which is one of the most widely used ML software suites. Specifically, we implement the top three function kernels that account for more than 85% of the total run-time. Simulations show that through novel core design, allocation and placement strategies our NoC-based implementation can achieve function-level speedups of 388x to 786x and system-level speedups in excess of 5000x over state-of-the-art multithreaded software.

Multicore platforms are emerging trends in the design of System-on-Chips (SoCs). Interconnect fabrics for these multicore SoCs play a crucial role in achieving the target performance. The Network-on-Chip (NoC) paradigm has been proposed as a promising solution for designing the interconnect fabric of multicore SoCs. But the performance requirements of NoC infrastructures in future technology nodes cannot be met by relying only on material innovation with traditional scaling. The continuing demand for low-power and high-speed interconnects with technology scaling necessitates looking beyond the conventional planar metal/dielectric-based interconnect infrastructures. Among different possible alternatives, the on-chip wireless communication network is envisioned as a revolutionary methodology, capable of bringing significant performance gains for multicore SoCs. Wireless NoCs (WiNoCs) can be designed by using miniaturized on-chip antennas as an enabling technology. In this paper, we present design methodologies and technology requirements for scalable WiNoC architectures and evaluate their performance. It is demonstrated that WiNoCs outperform their wired counterparts in terms of network throughput and latency, and that energy dissipation improves by orders of magnitude. The performance of the proposed WiNoC is evaluated in presence of various traffic patterns and also compared with other emerging alternative NoCs.

In a traditional Network-on-Chip (NoC), latency and power dissipation increase with system size due to its inherent multi-hop communications. The performance of NoC communication fabrics can be significantly enhanced by introducing long-range, low power, high bandwidth direct links between far apart cores. In this paper a design methodology for a scalable hierarchical NoC with on-chip millimeter (mm)-wave wireless links is proposed. The proposed wireless NoC offers significantly higher throughput and lower energy dissipation compared to its conventional multi-hop wired counterpart. It is also demonstrated that the proposed hierarchical NoC with long range wireless links shows significant performance gains in presence of various application-specific traffic and multicast scenarios.

Traveling Salesman Problem (TSP) is a classical NP-complete problem in graph theory. It aims at finding a least-cost Hamiltonian cycle that traverses all vertices of an input edge-weighted graph. One application of TSP is in breakpoint median-based Maximum Parsimony phylogenetic tree reconstruction, wherein a bounded edge-weight model is used. Exponential algorithms that apply efficient heuristics, such as branch-and-bound, to dynamically prune the search space are used. We adopted this approach in an NoC-based implementation for solving TSP targeted towards phylogenetics taking advantage of the fine-grained parallelism and efficient communication network. The largest fraction of the solution time for TSP is accounted for by a particular lower bound calculation operation that uses the graph's adjacency matrix. In this paper, we present the design and implementation of the processing elements with a highly optimized lower bound computation kernel and evaluate its performance. Additionally, we explore two major NoC architectures -mesh and quad-tree - and show that the latter is more suitable for this application domain.

Computing research has become a vital cog in the machinery required to drive biological discovery. Computing has made possible significant achievements over the last decade, especially in the genomics sector. An emerging area is the investigation of hardware accelerators for speeding up the massive scale of computation needed in large-scale biocomputing applications. Various hardware platforms, such as FPGA, Graphics Processing Unit (GPU), the Cell Broadband Engine (CBE) and multi-core processors are being explored. In this paper, we present a survey of hardware accelerators for biocomputing by choosing a representative set of each.

The most pervasive compute operation carried out in almost all bioinformatics applications is pairwise sequence homology detection (or sequence alignment). Due to exponentially growing sequence databases, computing this operation at a large-scale is becoming expensive. An effective approach to speed up this operation is to integrate a very high number of processing elements in a single chip so that the massive scales of fine-grain parallelism inherent in several bioinformatics applications can be exploited efficiently. Network-on-chip (NoC) is a very efficient method to achieve such large-scale integration. In this work, we propose to bridge the gap between data generation and processing in bioinformatics applications by designing NoC architectures for the sequence alignment operation. Specifically, we 1) propose optimized NoC architectures for different sequence alignment algorithms that were originally designed for distributed memory parallel computers and 2) provide a thorough comparative evaluation of their respective performance and energy dissipation. While accelerators using other hardware architectures such as FPGA, general purpose graphics processing unit (GPU), and the cell broadband engine (CBE) have been previously designed for sequence alignment, the NoC paradigm enables integration of a much larger number of processing elements on a single chip and also offers a higher degree of flexibility in placing them along the die to suit the underlying algorithm. The results show that our NoC-based implementations can provide above 102-103-fold speedup over other hardware accelerators and above 104-fold speedup over traditional CPU architectures. This is significant because it will drastically reduce the time required to perform the millions of alignment operations that are typical in large-scale bioinformatics projects. To the best of our knowledge, this work embodies the first attempt to accelerate a bioinformatics application using NoC.

Communication plays a crucial role in the design and performance of multi-core systems-on-chip (SoCs). Networks-on-chip (NoCs) have been proposed as a promising solution to simplify and optimize SoC design. However, it is expected that improving traditional communication technologies and interconnect organizations will not be sufficient to satisfy the demand for energy-efficient and high-performance interconnect fabrics, which continues to grow with each new process generation. Multiple options have been envisioned as compelling alternatives to the existing planar metal/dielectric communication structures. In this paper we outline the opportunities and challenges associated with three emerging interconnect paradigms: three-dimensional (3-D) integration, nanophotonic communication, and wireless interconnects.

The Network-on-Chip (NoC) paradigm has emerged as a revolutionary methodology for integrating a very high number of intellectual property (IP) blocks in a single die. The achievable performance benefit arising out of adopting NoCs is constrained by the performance limitation imposed by the metal wire, which is the physical realization of communication channels. With technology scaling, only depending on the material innovation will extend the lifetime of conventional interconnect systems a few technology generations. According to International Technology Roadmap for Semiconductors (ITRS) for the longer term, new interconnect paradigms are in need. The conventional two dimensional (2D) integrated circuit (IC) has limited floor-planning choices, and consequently it limits the performance enhancements arising out of NoC architectures. Three dimensional (3D) ICs are capable of achieving better performance, functionality, and packaging density compared to more traditional planar ICs. On the other hand, NoC is an enabling solution for integrating large numbers of embedded cores in a single die. 3D NoC architectures combine the benefits of these two new domains to offer an unprecedented performance gain. In this paper we evaluate the performance of 3D NoC architectures and demonstrate their superior functionality in terms of throughput, latency, energy dissipation and wiring area overhead compared to traditional 2D implementations.

Network-on-chip (NOC) is emerging as a revolutionary methodology to integrate numerous intellectual property blocks in a single die. It is the packet switching-based communications backbone that interconnects the components on multicore system-on-chip (SoC). A major challenge that NOC design is expected to face is related to the intrinsic unreliability of the interconnect infrastructure under technology limitations. By incorporating error control coding schemes along the interconnects, NOC architectures are able to provide correct functionality in the presence of different sources of transient noise and yet have lower overall energy dissipation. In this paper, designs of novel joint crosstalk avoidance and triple-error-correction/quadruple-error-detection codes are proposed, and their performance is evaluated in different NOC fabrics. It is demonstrated that the proposed codes outperform other existing coding schemes in making NOC fabrics reliable and energy efficient, with lower latency.

With the well-known trend of CMOS scaling as per Moore’s Law, traditional on-chip interconnect systems are reaching the point of having a very limited ability to meet the performance needs and specifications of Systems-on-Chip (SoCs). The conventional two-dimensional (2D) copper-based IC has inherent limitations due to the geometrical constraints of the planar structure. Innovative interconnect paradigms based on optical technologies, RF/wireless, carbon nanotubes, or 3D integration are promising alternatives that may indeed overcome the challenges encountered. In this paper we present an overview of different emerging non-traditional approaches to achieve massive degree of integration in a single chip. The advantages and underlying challenges of each method are highlighted.

Commercial designs are integrating from 10 to 100 embedded functional and storage blocks in a single system on chip (SoC) currently, and the number is likely to increase significantly in the near future. The communication requirements of these large Multi Processor SoCs (MP-SoCs) are convened by the emerging network-on-a-chip (NoC) paradigm. In the deep sub-micron (DSM) VLSI processes, it is difficult to guarantee correct fabrication with an acceptable yield without employing design techniques that take into account the intrinsic existence of manufacturing faults. To become a viable alternative IC design methodology the NoC paradigm must address the system-level reliability issues, which is going to be the dominant concern in the DSM and beyond silicon era. By incorporating adaptiveness in the data communication mechanism we are able to tolerate permanent manufacturing faults in the NoC interconnect architectures. The corresponding performance and cost figures must be carefully analyzed and weighted against the specific application requirements. In this paper we explore the performance tradeoffs associated with adaptive routing schemes in NoC fabrics.

Fault-tolerant design of network-on-chip communication architectures requires the addressing of issues pertaining to different elements described at different levels of design abstraction - these may be specific to architecture, interconnection, communication and application issues. Assessing the effectiveness of a particular fault-tolerant implementation can be a challenging task for designers, constrained with tight system performance specifications and other requirements In this paper, we provide a top-down view of fault-tolerance methods for NoC infrastructures, and present a range of metrics used for estimating their quality. We illustrate the use of these metrics by simulating a few simple but realistic fault-tolerant scenarios.

Three dimensional (3D) network on chip (NoC) has attracted researchers' attention recently. 3D NoCs are capable of achieving better system throughput and lower latency compared to the corresponding 2D implementations. To fully exploit the performance benefits of 3D architectures, it is imperative to address signal integrity issues in the design phase and its implications on energy dissipation. In this work we show that by incorporating joint crosstalk avoidance and multiple error correction schemes it is possible to enhance the robustness and reduce the energy dissipation simultaneously for both the 3D and more conventional planar NoC architectures. The achievable energy savings in 3D NoCs is significantly more than that in 2D structures.

Measuring and comparing performance, cost, and other features of advanced communication architectures for complex multi core/multiprocessor systems on chip is a significant challenge which has hardly been addressed so far. This document outlines the top-level view on a system of benchmarks for networks on chip (NoC), which intends to cover a wide spectrum of NoC design aspects, from application modeling to performance evaluation and post-manufacturing test and reliability. For performance benchmarking, requirements and features are described for application programs, synthetic micro-benchmarks, and abstract benchmark applications. Then, it proposes ways to measure and benchmark reliability, fault tolerance and testability of the on-chip communication fabric. This paper introduces the main concepts and ideas for benchmarking NoCs in a systematic and comparable way. It will be followed up by a report that will define a benchmark framework and the syntax of interfaces for benchmark programs that will allow the community to build-up a benchmark suite

Network-on-chip (NoC) communication fabrics will be increasingly used in many large multicore system-on-chip designs in the near future. A relevant challenge that arises from this trend is that the test costs associated with NoC infrastructures may account for a significant part of the total test budget. In this paper, we present a novel methodology for testing such NoC architectures. The proposed methodology offers a tradeoff between test time and on-chip self-test resources. The fault models used are specific to deep submicrometer technologies and account for crosstalk effects due to interwire coupling. The novelty of our approach lies in the progressive reuse of the NoC infrastructure to transport test data to the components under test in a recursive manner. It exploits the inherent parallelism of the data transport mechanism to reduce the test time and, implicitly, the test cost. We also describe a suitable test-scheduling approach. In this manner, the test methodology developed in this paper is able to reduce the test time significantly as compared to previously proposed solutions, offering speedup factors ranging from 2x to 34x for the NoCs considered for experimental evaluation.

Commercial designs are currently integrating from 10 to 100 embedded functional and storage blocks in a single SoC and the number is likely to increase significantly in the near future. With this ever increasing degree of integration, design of communication architectures for big SoCs is a challenge. The communication requirements of these large multi processor SoCs (MP-SoCs) are convened by the emerging network-on-a-chip (NoC) paradigm. The basic operations of NoC infrastructures are governed by on-chip packet-switched networks. Crosstalk between adjacent wires is an issue in NoC communication fabrics and it can cause timing violations and extra power dissipation. Crosstalk avoidance codes (CACs) can be used to improve signal integrity and also reduce the coupling capacitance and hence the energy dissipation of a wire segment. By incorporating crosstalk avoidance coding (CAC) in NoC data streams we are able to reduce communication energy, which will help to decrease the energy dissipation as a whole

A novel method for on-line fault detection and location in network-on-chip (NoC) communication fabrics is introduced. This approach is able to distinguish between faults in the communication links and faults in the NoC switches. The idea is based on the use of code-disjoint routing elements, combined with parity check encoding for the inter-switch links. We analyze the effect of our method on relevant performance parameters - power, latency, and throughput. Experiments show that our approach is effective and requires minimal modifications of the existing design methods for NoC interconnects.

In this paper, we present a novel built-in self-test methodology for testing the inter-switch links of network-on-chip (NoC) based chips. This methodology uses a high-level fault model that accounts for crosstalk effects due to inter-wire coupling. The novelty of our approach lies in the progressive reuse of the NoC infrastructure to transport test data to its own components under test in a bootstrap manner, and in extensively exploiting the inherent parallelism of the data transport mechanism to reduce the test time and implicitly the test cost

Over the past ten years, as integrated circuits became increasingly more complex and expensive, the industry began to embrace new design and reuse methodologies that are collectively referred to as system-on-chip (SoC) design. In this paper, we focus on the reuse and integration issues encountered in this paradigm shift. The reusable components, called intellectual property (IP) blocks or cores, are typically synthesizable register-transfer level (RTL) designs (often called soft cores) or layout level designs (often called hard cores). The concept of reuse can be carried out at the block, platform, or chip levels, and involves making the IP sufficiently general, configurable, or programmable, for use in a wide range of applications. The IP integration issues include connecting the computational units to the communication medium, which is moving from ad hoc bus-based approaches toward structured network-on-chip (NoC) architectures. Design-for-test methodologies are also described, along with verification issues that must be addressed when integrating reusable components.

Multiprocessor system-on-chip (MP-SoC) platforms are emerging as an important trend for SoC design. Power and wire design constraints are forcing the adoption of new design methodologies for system-on-chip (SoC), namely, those that incorporate modularity and explicit parallelism. To enable these MP-SoC platforms, researchers have recently pursued scaleable communication-centric interconnect fabrics, such as networks-on-chip (NoC), which possess many features that are particularly attractive for these. These communication-centric interconnect fabrics are characterized by different trade-offs with regard to latency, throughput, energy dissipation, and silicon area requirements. In this paper, we develop a consistent and meaningful evaluation methodology to compare the performance and characteristics of a variety of NoC architectures. We also explore design trade-offs that characterize the NoC approach and obtain comparative results for a number of common NoC topologies. To the best of our knowledge, this is the first effort in characterizing different NoC architectures with respect to their performance and design trade-offs. To further illustrate our evaluation methodology, we map a typical multiprocessing platform to different NoC interconnect architectures and show how the system performance is affected by these design trade-offs.

Multi-Processor (MP-SoC) platforms are emerging as the latest trend in SoC design. Monolithic bus-based interconnect architectures will not be able to support the clock cycle requirements of these high performance SoCs. Systems having multiple smaller buses, integrated through repeaters or bridges, are possible alternatives. But these kinds of solutions are ad-hoc in nature. By adopting a more structured network-based design paradigm, specific clock cycle requirements can easily be met. The precise focus of this paper is to show how the butterfly fat tree (BFT) can meet this objective when used as the overall MP-SoC interconnect architecture, thereby offering an attractive alternative for SoC interconnect that does not suffer from the non-scalability aspect of the buses in regards to the clock cycle.