Long-latency load requests continue to limit the performance of modern high-performance processors. To increase the latency tolerance of a processor, architects have primarily relied on two key techniques: sophisticated data prefetchers and large on-chip caches. In this work, we show that: (1) even a sophisticated state-of-the-art prefetcher can only predict half of the off-chip load requests on average across a wide range of workloads, and (2) due to the increasing size and complexity of on-chip caches, a large fraction of the latency of an off-chip load request is spent accessing the on-chip cache hierarchy to solely determine that it needs to go off-chip. The goal of this work is to accelerate off-chip load requests by removing the on-chip cache access latency from their critical path. To this end, we propose a new technique called Hermes, whose key idea is to: (1) accurately predict which load requests might go off-chip, and (2) speculatively fetch the data required by the predicted off-chip loads directly from the main memory, while also concurrently accessing the cache hierarchy for such loads. To enable Hermes, we develop a new lightweight, perceptron-based off-chip load prediction technique that learns to identify off-chip load requests using multiple program features (e.g., sequence of program counters, byte offset of a load request). For every load request generated by the processor, the predictor observes a set of program features to predict whether or not the load would go off-chip. If the load is predicted to go off-chip, Hermes issues a speculative load request directly to the main memory controller once the load’s physical address is generated. If the prediction is correct, the load eventually misses the cache hierarchy and waits for the ongoing speculative load request to finish, and thus Hermes completely hides the on-chip cache hierarchy access latency from the critical path of the correctly-predicted off-chip load. Our extensive evaluation using a wide range of workloads shows that Hermes provides consistent performance improvement on top of a state-of-the-art baseline system across a wide range of configurations with varying core count, main memory bandwidth, high-performance data prefetchers, and on-chip cache hierarchy access latencies, while incurring only modest storage overhead. The source code of Hermes is freely available at: https://github.com/CMU-SAFARI/Hermes.

Deep Neural Networks (DNN) are used in a variety of applications and services. With the evolving nature of DNNs, the race to build optimal hardware (both in datacenter and edge) continues. General purpose multi-core CPUs offer unique attractive advantages for DNN inference at both datacenter [60] and edge [71]. Most of the CPU pipeline design complexity is targeted towards optimizing general-purpose single thread performance, and is overkill for relatively simpler, but still hugely important, data parallel DNN inference workloads. Addressing this disparity efficiently can enable both raw performance scaling and overall performance/Watt improvements for multi-core CPU DNN inference.We present REDUCT, where we build innovative solutions that bypass traditional CPU resources which impact DNN inference power and limit its performance. Fundamentally, REDUCT’s "Keep it close" policy enables consecutive pieces of work to be executed close to each other. REDUCT enables instruction delivery/decode close to execution and instruction execution close to data. Simple ISA extensions encode the fixed-iteration count loop-y workload behavior enabling an effective bypass of many power-hungry front-end stages of the wide Out-of-Order (OoO) CPU pipeline. Per core performance scales efficiently by distributing light-weight tensor compute near all caches in a multi-level cache hierarchy. This maximizes the cumulative utilization of the existing architectural bandwidth resources in the system and minimizes movement of data.Across a number of DNN models, REDUCT achieves a 2.3× increase in convolution performance/Watt with a 2× to 3.94× scaling in raw performance. Similarly, REDUCT achieves a 1.8× increase in inner-product performance/Watt with 2.8× scaling in performance. REDUCT performance/power scaling is achieved with no increase to cache capacity or bandwidth and a mere 2.63% increase in area. Crucially, REDUCT operates entirely within the CPU programming and memory model, simplifying software development, while achieving performance similar to or better than state-of-the-art Domain Specific Accelerators (DSA) for DNN inference, providing fresh design choices in the AI era.

This paper analyzes the performance of two contemporary data-generating neural network-based workloads, Neural Style Transfer and Super Resolution GAN run on x86 hardware architecture. In understanding the impact of data-readiness, we find how certain layers benefit from forced data warming-up. In examining bandwidth utilization of these layers, we identify several memory-bound layers as not necessarily being bandwidthbound hinting at the feasibility of prefetch-based solutions for improved performance. We also observe layers with specific kernel sizes performing poorly because of their unoptimized library kernel implementation. Based on our findings, we suggest directions for removing these performance bottlenecks by utilizing available bandwidth margins ≥ 90% and realizing convolution operations through vector-based functional units with a scope of at least 20x more such software-to-hardware mappings than existing implementation.

Semiconductor industry has been aggressively following the Moore's Law ever since its was proposed in the late sixties in its pursuit for smaller device sizes and higher performance metrics. However, this vigorous scaling has brought in several scaling induced side effects like single event upsets into the technology regime. SRAMs are highly susceptible to such upsets since they are designed at minimum device sizes to keep the on-chip memory density high. This paper presents a novel SEU-hardened SRAM cell employing single bitline. The proposed cell is 4 times more immune than a standard 6T-SRAM cell and also achieves 68% reduction in bitline leakage.

In modern CMPs, Last Level Cache (LLC) is shared among cores for better utilization. Interference among data, mapped from multiple cores, increases conflict misses in shared LLCs. Such interference is highly dependent on cache behavior of applications and access rate difference among them. We observe that interference among applications is not eliminated completely even using existing state-of-the-art mechanism for applications having high cache access rate difference and different memory characteristic. Applications with highly diverse cache behavior can be observed in homogeneous as well as heterogeneous multicore processors. Streaming applications, having high access rate, can still interfere with cache friendly applications having low access rate. We propose Application behavior aware replacement policy that predicts re-reference interval of the block based on block locality as well as application behavior. By providing more priority to application behavior over cache block locality, we reduce interference between streaming applications and cache friendly applications. Our evaluation on set of SPEC CPU2006 workloads running on CMP with shared LLC shows that proposed replacement policy outperforms the state-of-the-art replacement policy, on throughput metric. We achieve performance gain up to 16.2% over SRRIP for application mixes of cache-friendly and streaming applications. Our replacement policy achieves maximum of 59.9% reduction in number of misses as compared to SRRIP with average misses per kilo instructions (mpki) reduction of 5.9% over SRRIP.

MOTIVATION: Synthetic lethal sets are sets of reactions/genes where only the simultaneous removal of all reactions/genes in the set abolishes growth of an organism. Previous approaches to identify synthetic lethal genes in genome-scale metabolic networks have built on the framework of flux balance analysis (FBA), extending it either to exhaustively analyze all possible combinations of genes or formulate the problem as a bi-level mixed integer linear programming (MILP) problem. We here propose an algorithm, Fast-SL, which surmounts the computational complexity of previous approaches by iteratively reducing the search space for synthetic lethals, resulting in a substantial reduction in running time, even for higher order synthetic lethals. RESULTS: We performed synthetic reaction and gene lethality analysis, using Fast-SL, for genome-scale metabolic networks of Escherichia coli, Salmonella enterica Typhimurium and Mycobacterium tuberculosis. Fast-SL also rigorously identifies synthetic lethal gene deletions, uncovering synthetic lethal triplets that were not reported previously. We confirm that the triple lethal gene sets obtained for the three organisms have a precise match with the results obtained through exhaustive enumeration of lethals performed on a computer cluster. We also parallelized our algorithm, enabling the identification of synthetic lethal gene quadruplets for all three organisms in under 6 h. Overall, Fast-SL enables an efficient enumeration of higher order synthetic lethals in metabolic networks, which may help uncover previously unknown genetic interactions and combinatorial drug targets. AVAILABILITY AND IMPLEMENTATION: The MATLAB implementation of the algorithm, compatible with COBRA toolbox v2.0, is available at https://github.com/RamanLab/FastSL CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

With the increase in process variations and diversity in workloads, it is imperative to holistically explore optimization techniques for power and temperature from the circuit layer right up to the compiler/ operating system (OS) layer. This article proposes one such holistic technique, called proactive workload aware temperature management framework for low-power chip multi-processors (ProWATCh). At the compiler level ProWATCh includes two techniques: (1) a novel compiler design for estimating the architectural parameters of a task at compile time; and (2) a model-based technique for dynamic estimation of architectural parameters at runtime. At the OS level ProWATCh integrates two techniques: (1) a workload- and temperature-aware process manager for dynamic distribution of tasks to different cores; and (2) a model predictive control-based task scheduler for generating the efficient sequence of task execution. At the circuit level ProWATCh implements either of two techniques: (1) a workload-aware voltage manager for dynamic supply and body bias voltage assignment for a given frequency in processors that support adaptive body bias (ABB); or (2) a workload-aware frequency governor for efficient assignment of upper and lower frequency bounds for frequency scaling in processors that do not support an ABB. Employing ProWATCh (with voltage manager) on an ABB-compatible 3D OpenSPARC architecture using MiBench benchmarks resulted in an average 18% ( 19ˆC ) reduction in peak temperature. Evaluating ProWATCh on an existing quad-core Intel Corei7 processor with frequency governor alone (as the processor does not support an ABB interface) resulted in 10% ( 8ˆC ) reduction in peak temperature when compared to what was obtained using the native Linux 3.0 completely fair scheduler (CFS). To study the effectiveness of the proposed framework across benchmark suites, ProWATCh was evaluated on a quad-core Intel Corei7 processor using CPU SPEC 2006 benchmarks which resulted in 7ˆC reduction in peak temperature as compared to the native Linux 3.0 CFS.

Aggressive prefetching improves system performance by hiding and tolerating off-chip memory latency. However, on a multicore system, prefetchers of different cores contend for shared resources and aggressive prefetching can degrade the overall system performance. The role of a prefetcher aggressiveness engine is to select appropriate aggressiveness levels for each prefetcher such that shared resource contention caused by prefetchers is reduced, thereby improving system performance. State-of-the-art prefetcher aggressiveness engines monitor metrics such as prefetch accuracy, bandwidth consumption, and last-level cache pollution. They use carefully tuned thresholds for these metrics, and when the thresholds are crossed, they trigger aggressiveness control measures. These engines have three major shortcomings: (1) thresholds are dependent on the system configuration (cache size, DRAM scheduling policy, and cache replacement policy) and have to be tuned appropriately, (2) there is no single threshold that works well across all the workloads, and (3) thresholds are oblivious to the phase change of applications. To overcome these shortcomings, we propose CAFFEINE, a model-based approach that analyzes the effectiveness of a prefetcher and uses a metric called net utility to control the aggressiveness. Our metric provides net processor cycles saved because of prefetching by approximating the cycles saved across the memory subsystem, from last-level cache to DRAM. We evaluate CAFFEINE across a wide range of workloads and compare it with the state-of-the-art prefetcher aggressiveness engine. Experimental results demonstrate that, on average (geomean), CAFFEINE achieves 9.5% (as much as 38.29%) and 11% (as much as 20.7%) better performance than the best-performing aggressiveness engine for four-core and eight-core systems, respectively.

Hardware prefetchers are commonly used to hide and tolerate off-chip memory latency. Prefetching techniques in the literature are designed for multiple independent sequential applications running on a multicore system. In contrast to multiple independent applications, a single parallel application running on a multicore system exhibits different behavior. In case of a parallel application, cores share and communicate data and code among themselves, and there is commonality in the demand miss streams across multiple cores. This gives an opportunity to predict the demand miss streams and communicate the predicted streams from one core to another, which we refer as cross-core stream communication.

A single parallel application running on a multi-core system shows sub-linear speedup because of slow progress of one or more threads known as critical threads. Some of the reasons for the slow progress of threads are (1) load imbalance, (2) frequent cache misses and (3) effect of synchronization primitives. Identifying critical threads and minimizing their cache miss latencies can improve the overall execution time of a program. One way to hide and tolerate the cache misses is through hardware prefetching. Hardware prefetching is one of the most commonly used memory latency hiding techniques. Previous studies have shown the effectiveness of hardware prefetchers for multiprogrammed workloads (multiple sequential applications running independently on different cores). In contrast to multiprogrammed workloads, the performance of a single parallel application depends on the progress of slow progress(critical) threads. This paper introduces a prefetcher aggressiveness control mechanism called Thread Criticality-aware Prefetcher Aggressiveness Control (TCPAC). TCPAC controls the aggressiveness of the prefetchers at the L2 prefetching controllers (known as TCPAC-P), DRAM controller (known as TCPAC-D) and at the Last Level Cache (LLC) controller (known as TCPAC-C) using prefetch accuracy and thread progress. Each TCPAC sub-technique outperforms the respective state-of-the-art techniques such as HPAC [2], PADC [4], and PACMan [3] and the combination of all the TCPAC sub-techniques named as TCPAC-PDC outperforms the combination of HPAC, PADC, and PACMan. On an average, on a 8 core system, in terms of improvement in execution time, TCPAC-PDC outperforms the combination of HPAC, PADC, and PACMan by 7.61%. For 12 and 16 cores, TCPAC-PDC beats the state-of-the-art combinations by 7.21% and 8.32% respectively.

A single parallel application running on a multicore system shows sub-linear speedup because of slow progress of one or more threads known as critical threads. Identifying critical threads and accelerating them can improve system performance. One of the metrics that correlate to thread criticality is the number of cache misses and the penalty associated with it. This paper proposes a throttling mechanism called TCPT which throttles hardware prefetchers by changing the prefetch degree based on the thread criticality.

Smooth approximations to half-perimeter wirelength are being investigated actively because of the recent increase in interest in analytical placement. It is necessary to not just provide smooth approximations but also to provide error analysis and convergence properties of these approximations. We present a new approximation scheme which uses a non-recursive approximation to the max function. We also show the convergence properties and the error bounds. The accuracy of our proposed scheme is better than those of the popular Logarithm-Sum-Exponential (LSE) wirelength model [7] and the recently proposed Weighted Average(WA) wirelength model[3]. We also experimentally validate the comparison by using global and detail placements produced by NTU Placer [1] on ISPD 2004 benchmark suite. The experimentations on benchmarks validate that the error bounds of our model are lower, with an average of 4% error in the total wirelength.

Existing low power testing techniques either focus on reducing the switching activity neglecting supply voltage, or perform supply voltage scaling without attempting to minimize switching activity. In this paper we propose LPScan (Low Power Scan), which integrates supply scaling and switching activity reduction in a single framework to reduce test power. For a shift frequency of 125MHz, the LPScan algorithm when applied to circuits from the ISCAS, OpenCores and ITC benchmark suite, produced power savings of 80% in the best case and 50% in the average case, compared to the best known algorithm [1].

Coalesced memory access patterns in CUDA yields high performance but achieving such patterns in an application can be tedious. We propose a tool, CUPL, which locates uncoalesced access patterns (UCAP) in a given kernel at compile-time. CUPL does static analysis of a given kernel using polyhedral model and reports warnings if the input kernel exhibits UCAP. CUPL has two-fold use 1) It can help the programmer to locate regions of the code to optimize 2) It can help a compiler to perform efficient data layout transformations. Initial experiments show that CUPL reports warnings at appropriate places in kernels from Rodinia benchmark and NVIDIA SDK suites.

Hardware prefetching has been studied in the past for multi-programmed workloads as well as GPUs. Efficient hardware prefetchers like stream-based or GHB-based ones work well for multiprogrammed workloads because different programs get mapped to different cores and are run independently. Parallel applications, however, pose a different set of challenges. Multiple threads of a parallel application share data with each other which brings in coherency issues. Also, local prefetchers do not understand the irregular spread of misses across threads. In this paper, we propose a hardware prefetching framework for L1 D-Cache that targets parallel applications. We show how to make efficient prefetch requests to the L2 cache by studying and classifying the patterns of L1 misses across all the threads. Our preliminary results show an improvement of 7% in execution time on an average on the PARSEC benchmark suite.

A program that is run on a General Purpose Graphics Processing Unit (GPGPU) has to stall if the data is not resident in the GPGPU. With CUDA 2.0 architecture, data can be streamed while the computation is still on. Exploiting this feature requires careful orchestration of data transfer and computation which typically requires a significant effort from the programmer. We propose an approach of transforming C programs to programs that can make use of CUDA streams. We identify the regions where data transfer and computation can be overlapped by using a polyhedral framework called PLUTO[2]. We use the PLUTO framework to do automatic tiling of source code and use the streaming capabilities to overlap data transfer with computation. Our results show an average speedup of 1.5X over CUDA programs without streaming optimizations.

Parallel applications are becoming mainstream and architectural techniques for multicores that target these applications are the need of the hour. Sharing of data by multiple threads and issues due to data coherence are unique to parallel applications. We propose CSHARP, a hardware framework that brings coherence and sharing awareness to any shared last level cache replacement policy. We use the degree of sharing of cache lines and the information present in coherence vectors to make replacement decisions. We apply CSHARP to a state-of-the-art cache replacement policy called TA-DRRIP to show its effectiveness. Our experiments on four core simulated system show that applying CSHARP on TA-DRRIP gives an extra 10% reduction in miss-rate at the LLC. Compared to LRU policy, CSHARP on TA-DRRIP shows a 18% miss-rate reduction and a 7% performance boost. We also show the scalability of our proposal by studying the hardware overhead and performance on a 8-core system.

A-priori interconnect prediction is the estimation of routing requirements of a design without even performing placement. All current a-priori models assume that the optimization objective during placement and routing is to minimize wirelength and congestion. However, design performance is a very important goal to all designers and interconnects requirements for a design that are optimized for timing have not been studied. We present an empirical methodology that takes c, a user defined input that trades off routability and timing, and estimate a-priori some important characteristics of the design. This tunable model will be very useful for design under uncertainties or preliminary feasibility study at system levels. We show how the length of source-sink pairs is important and how its distribution can be predicted. Our results show very accurate prediction for circuits that are placed and routed with VPR.

Interconnect prediction is very important for early feasibility studies in modern design flows. Most of the current interconnect estimation techniques estimate either the average or the total wirelength and some qualitative measure of routing demand for circuits. A priori techniques estimate these parameters without actually performing circuit placement. We propose a new a priori interconnect and wirelength estimation methodology for island style field programmable gate arrays (FPGAs). For a given design, we estimate bounding box semiperimeter wirelengths of all nets for an optimized placement and the minimum number of tracks per channel required for successful routing on an FPGA device. We analyze the structural characteristics of circuits and limitations posed by the FPGA architecture to derive a consistent model for wirelength and routing demand estimation. We identify reconvergences present in a circuit as an important global circuit characteristic in wirelength prediction. Our overall results show that we have an average error of 11.6% w.r.t. semiperimeter wirelength measured from the optimized layout using VPR. Also, the number of routing tracks per channel is predicted with an average error of 13.2% of the detailed routing results from VPR.

Interconnect management is a critical design issue for large field-programmable gate arrays (FPGA) based designs. One of the most important issues for planning interconnection is the ability to reliably and efficiently predict the interconnect requirements of a given design on a given FPGA architecture. Many interconnect estimation methods have been reported so far and the estimation problem is also under active research. From a CAD tool deployment point of view, comparing these estimation methods is very difficult because of the different reporting methods used by the authors. We make an argument for and propose a new uniform reporting metric, based on comparing the estimates with the results of an actual detailed router on both local and global levels. We then compare some of the well known and promising interconnect estimation methods using our new metric on a large number of benchmark circuits.

Interconnect management is a critical design issue for large FPGA based designs. One of the most important issues for planning interconnection is the ability to accurately and efficiently predict the routability of a given design on a given FPGA architecture. The recently proposed routability estimation procedure, fGREP [6], produced estimates within 3 to 4% of an actual detailed router. Other known routability estimation methods include RISA [5], Lois's [7] method and Rent's rule based methods [1] [11] [9]. Comparing these methods has been difficult because of the different reporting methods used by the authors. We propose a uniform reporting metric based on comparing the estimates produced with the results of an actual detailed router on both local and global levels. We compare all the above methods using our reporting metric on a large number of benchmark circuits and show that the enhanced fGREP method produces tight estimates that outperform most other techniques.