Convolutional Neural Networks (CNNs) have shown a great deal of success in diverse application domains including computer vision, speech recognition, and natural language processing. However, as the size of datasets and the depth of neural network architectures continue to grow, it is imperative to design high-performance and energy-efficient computing hardware for training CNNs. In this paper, we consider the problem of designing specialized CPU-GPU based heterogeneous manycore systems for energy-efficient training of CNNs. It has already been shown that the typical on-chip communication infrastructures employed in conventional CPU-GPU based heterogeneous manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. To address this issue, we first analyze the on-chip traffic patterns that arise from the computational processes associated with training two deep CNN architectures, namely, LeNet and CDBNet, to perform image classification. By leveraging this knowledge, we design a hybrid Network-on-Chip (NoC) architecture, which consists of both wireline and wireless links, to improve the performance of CPU-GPU based heterogeneous manycore platforms running the above-mentioned CNN training workloads. The proposed NoC achieves 1.8× reduction in network latency and improves the network throughput by a factor of 2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC. For the considered CNN workloads, these network-level improvements translate into 25 percent savings in full-system energy-delay-product (EDP). This demonstrates that the proposed hybrid NoC for heterogeneous manycore architectures is capable of significantly accelerating training of CNNs while remaining energy-efficient.

Deep neural networks have seen tremendous success for different modalities of data including images, videos, and speech. This success has led to their deployment in mobile and embedded systems for real-time applications. However, making repeated inferences using deep networks on embedded systems poses significant challenges due to constrained resources (e.g., energy and computing power). To address these challenges, we develop a principled co-design approach. Building on prior work, we develop a formalism referred as coarse-to-fine networks (C2F Nets) that allow us to employ classifiers of varying complexity to make predictions. We propose a principled optimization algorithm to automatically configure C2F Nets for a specified tradeoff between accuracy and energy consumption for inference. The key idea is to select a classifier on-the-fly whose complexity is proportional to the hardness of the input example: simple classifiers for easy inputs and complex classifiers for hard inputs. We perform comprehensive experimental evaluation using four different C2F Net architectures on multiple real-world image classification tasks. Our results show that optimized C2F Net can reduce the energy delay product by 27% to 60% with no loss in accuracy when compared to the baseline solution, where all predictions are made using the most complex classifier in C2F Net.

As deep learning technology is increasingly employed in diverse applications domains, the demand for computational power to enable these algorithms also increases. In this respect, high-performance three-dimensional (3D) heterogeneous manycore systems present a promising direction. However, deep learning on these systems pose several design challenges. First, the network-on-chip (NoC) must handle the traffic requirements of both CPU and GPU communications. Second, 3D system designs must address thermal issues resulting from high-power density. In this work, we propose a design methodology for a heterogeneous 3D NoC architecture that not only satisfies the traffic requirements of both CPUs and GPUs, but also reduces thermal hotspots. To this end, we target the training of two widely employed convolutional neural networks (CNN), namely, LeNet and CIFAR. By using our joint performance-thermal optimization methodology to create a 3D NoC for training CNNs, we reduce the maximum temperature by 22% while incurring only 5% full-system energy-delay-product degradation over a solely performance optimized 3D NoC. This demonstrates that, our design methodology achieves considerable temperature reduction with negligible loss in performance.

The emergence of 3D network-on-chip (NoC) has revolutionized the design of high-performance and energy-efficient manycore chips. However, the anticipated performance gain can be compromised due to the degradation and failure of vertical links (VLs). The Through-Silicon-Via (TSV)-enabled VLs may fail due to workload-induced stress; the failure of a VL can affect the neighboring VLs, thereby causing a cascade of failures and reducing the lifetime of the chip. To enhance the reliability of 3D NoC-enabled manycore chips, we propose to incorporate a voltage-frequency island (VFI)-based power management strategy that helps to reduce the energy consumption and hence, the workload-induced stress of the highly utilized VLs. The adopted power-management strategy relies on control decisions about the voltage/frequency (V/F) levels on VLs. We demonstrate that compared to the well-known spare TSV allocation and adaptive routing strategies, power management is more effective in enhancing the reliability of a 3D NoC. VFI-based power management improves the reliability of the 3D NoC by one order of magnitude compared to both adaptive routing and spare allocation while running popular SPLASH-2 and PARSEC benchmarks. The principal benefit of power management is that it is capable of reducing the operating temperature of the system, which in turn enhances the Mean-Time-To-Failure (MTTF) of the VLs and reliability of the overall 3D NoC.

Manycore chips are widely employed in high-performance computing and large-scale data analysis. However, the design of high-performance manycore chips is dominated by power and thermal constraints. In this respect, voltage-frequency island (VFI) is a promising design paradigm to create scalable energy-efficient platforms. By dynamically tailoring the voltage and frequency of each island, we can further improve the energy savings within given performance constraints. Inspired by the recent success of imitation learning (IL) in many application domains and its significant advantages over reinforcement learning (RL), we propose the first architecture-independent IL-based methodology for dynamic VFI (DVFI) control in manycore systems. Due to its popularity in the EDA community, we consider an RL-based DVFI control methodology as a strong baseline. Our experimental results demonstrate that IL is able to obtain higher quality policies than RL (on average, 5% less energy with the same level of performance) with significantly less computation time and hardware area overheads (3.1X and 8.8X, respectively).

In recent years, designing specialized manycore heterogeneous architectures for deep learning kernels has become an area of great interest. However, the typical on-chip communication infrastructures employed on conventional manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. Hence, in this paper, our aim is to enhance the performance of heterogeneous manycore architectures through the design of a hybrid NoC consisting of both wireline and wireless links. To this end, we specifically target the resource-intensive backpropagation algorithm commonly used as the training method in deep learning. For backpropagation, the proposed hybrid NoC achieves 1.9X reduction in network latency and improves the network throughput by a factor of 2 with respect to a highly optimized mesh NoC. These network level improvements translate into 25% savings in full system energy-delay-product (EDP). This demonstrates the capability of the proposed hybrid and heterogeneous manycore architecture in accelerating deep learning kernels in an energy-efficient manner.

Multiple Voltage Frequency Island (VFI)-based designs can reduce the energy dissipation in multicore chips. Indeed, by tailoring the voltages and frequencies of each VFI domain, we can achieve significant energy savings subject to specific performance constraints. The achievable performance of VFI-based multicore platforms depends on the overall communication backbone, which relies predominantly on Networks-on-Chip (NoCs). Traditionally mesh-based NoCs have been used in VFI-based systems. However, the mesh-based NoCs have large latency and energy overheads due to their inherently long multihop paths. Emerging paradigms such as the millimeter (mm)-wave small-world wireless Networks-on-Chip (mSWNoCs) have lately been observed to help reduce the impact of the communication backbone on the performance of the multicore chips. In this work, we demonstrate that not only do mSWNoC-enabled VFI designs mitigate some of the full-system performance degradation inherent in VFI-partitioned multicore designs, but they also help in eliminating it entirely for certain applications. We also demonstrate that the VFI-partitioned designs used in conjunction with a novel NoC architecture like mSWNoC can achieve significant energy savings while minimizing the impact on the performance for each application under consideration.

Multiple voltage frequency island (VFI)-based designs can reduce the energy dissipation in multicore platforms by taking advantage of the varying nature of the application workloads. Indeed, the voltage/frequency (V/F) levels of the VFIs can be dynamically tailored by considering the workload-driven variations in the application. Traditionally, mesh-based networks-on-chip (NoCs) have been used in VFI-based systems; however, they have large latency and energy overheads due to the inherently long multihop paths. Consequently, in this paper, we explore the emerging paradigm of wireless NoC (WiNoC) and demonstrate that by incorporating WiNoC, VFI, and dynamic V/F tuning in a synergistic manner, we can design energy-efficient multicore platforms without introducing noticeable performance penalty. Our experimental results show that for the benchmarks considered, the proposed approach can achieve between 5.7% and 46.6% energy-delay product (EDP) savings over the state-of-the-art system and 26.8% and 60.5% EDP savings over a standard baseline non-VFI mesh-based system. This opens up a new of class of codesign approaches that can make WiNoCs the communication technology of choice for future multicore platforms.

In an era when power constraints and data movement are proving to be significant barriers for high-end computing, multicore architectures offer a low-power and highly scalable platform suitable for both data- and compute-intensive applications. MapReduce is a popular framework to facilitate the management and development of big-data workloads. In this work, we demonstrate that by using a wireless NoC-enabled Voltage Frequency Island (VFI)-based multicore platform it is possible to enhance the energy efficiency of MapReduce implementations without paying significant execution time penalties. Our experimental results show that for the benchmarks considered, the designed VFI system can achieve an average of 33.7% energy-delay product (EDP) savings over the standard baseline non-VFI mesh-based system while paying a maximum of 3.22% execution time penalty.

During the last decade, we have witnessed a major transition from computation- to communication-centric design of integrated circuits and systems. In particular, the network-on-chip (NoC) approach has emerged as the major design paradigm for multicore systems-on-chip (SoC). The major challenges in traditional wire-based NoCs are the high latency and power consumption of the multi-hop links. By inserting single-hop long-range wireless links in place of multi-hop wired links, the overall system performance can be significantly improved. We should adopt novel architectures inspired by the on-chip wireless links to design high-performance multi-core chips. In this regard, the small-world network-inspired wireless NoC (WiNoC) has emerged as an enabling interconnection infrastructure to design high-bandwidth and energy-efficient multicore chips. In this paper we present the various challenges and possible solutions for designing energy-efficient massive multicore chips enabled by the WiNoC paradigm.