Integrating domain-specific hardware accelerators on modern systems on chips (SoCs) has enabled complex applications, such as vision, natural language processing, autonomous driving, and augmented reality, on small form factors. This leads to challenges in the integration of accelerators, with high memory bandwidth requirements and strict deadlines, on the system’s memory hierarchy. The system-level shared cache, or last-level cache (LLC), is a critical resource shared by multi-core processors, GPUs, and hardware accelerators in modern heterogeneous SoCs. It significantly reduces the bottleneck at the off-chip memory and delivers high performance. With the integration of accelerators on the LLC gaining momentum, the on-chip shared cache management becomes vital. If not managed intelligently, the interference between cache requests from the cores and the accelerators can significantly deteriorate their performance. Given the architectural differences between DRAM and cache systems, the off-chip memory management strategies explored by previous works cannot be extended to the LLC. We propose a deadline-aware flexible LLC arbitration and scheduling framework, FLASH , to dynamically partition the LLC bandwidth between the accelerators and multi-core processors to meet the deadline given for the accelerator while minimizing the impact on the performance of the cores. FLASH arbitrates between the requests from the cores and the accelerators and schedules the requests depending on the accelerator’s progress and its chances of meeting the deadline. We evaluate FLASH across different workloads and hardware accelerator configurations to show that it not only achieves significantly better performance for the cores than other static scheduling policies but also significantly reduces the deadline miss rates of the accelerator.

To keep up with the growing computational demands of current-day applications, SoC design has shifted towards heterogeneous architectures with CPUs and domain-specific accelerators. These accelerators demand high on-chip and off-chip memory bandwidth and require efficient management of shared system resources. We characterize Internet Protocol Security (IPsec), a high-throughput application, by collecting the memory traces of this application running on the accelerators and CPU cores of NXP LX2160A SoC. We use this characterization to design a simulation infrastructure for simulating IPsec on possible domain-specific architectural extensions and perform a design-space exploration across various general-purpose memory management policies. We propose APPAMM, an application-specific predictive packet-aware memory management policy using the knowledge of IPsec to improve performance for next-generation SoCs. Using our approach to manage memory for different input packet streams, we report improvements of up to 22x in the packet drop rate and peak throughput.

This paper proposes the use of empirical modeling techniques for building microarchitecture sensitive models for compiler optimizations. The models we build relate program performance to settings of compiler optimization flags, associated heuristics and key microarchitectural parameters. Unlike traditional analytical modeling methods, this relationship is learned entirely from data obtained by measuring performance at a small number of carefully selected compiler/microarchitecture configurations. We evaluate three different learning techniques in this context viz. linear regression, adaptive regression splines and radial basis function networks. We use the generated models to a) predict program performance at arbitrary compiler/microarchitecture configurations, b) quantify the significance of complex interactions between optimizations and the microarchitecture, and c) efficiently search for 'optimal' settings of optimization flags and heuristics for any given micro architectural configuration. Our evaluation using benchmarks from the SPEC CPU2000 suites suggests that accurate models (< 5% average error in prediction) can be generated using a reasonable number of simulations. We also find that using compiler settings prescribed by a model-based search can improve program performance by as much as 19% (with an average of 9.5%) over highly optimized binaries

Processor architects have a challenging task of evaluating a large design space consisting of several interacting parameters and optimizations. In order to assist architects in making crucial design decisions, we build linear regression models that relate processor performance to micro-architectural parameters, using simulation based experiments. We obtain good approximate models using an iterative process in which Akaike's information criteria is used to extract a good linear model from a small set of simulations, and limited further simulation is guided by the model using D-optimal experimental designs. The iterative process is repeated until desired error bounds are achieved. We used this procedure to establish the relationship of the CPI performance response to 26 key micro-architectural parameters using a detailed cycle-by-cycle superscalar processor simulator. The resulting models provide a significance ordering on all micro-architectural parameters and their interactions, and explain the performance variations of micro-architectural techniques.

Designing and optimizing high performance microprocessors is an increasingly difficult task due to the size and complexity of the processor design space, high cost of detailed simulation and several constraints that a processor design must satisfy. In this paper, we propose the use of empirical non-linear modeling techniques to assist processor architects in making design decisions and resolving complex trade-offs. We propose a procedure for building accurate non-linear models that consists of the following steps: (i) selection of a small set of representative design points spread across processor design space using latin hypercube sampling, (ii) obtaining performance measures at the selected design points using detailed simulation, (iii) building non-linear models for performance using the function approximation capabilities of radial basis function networks, and (iv) validating the models using an independently and randomly generated set of design points. We evaluate our model building procedure by constructing non-linear performance models for programs from the SPEC CPU2000 benchmark suite with a microarchitectural design space that consists of 9 key parameters. Our results show that the models, built using a relatively small number of simulations, achieve high prediction accuracy (only 2.8% error in CPI estimates on average) across a large processor design space. Our models can potentially replace detailed simulation for common tasks such as the analysis of key microarchitectural trends or searches for optimal processor design points

Several ILP limit studies indicate the presence of considerable ILP across dynamically far-apart instructions in program execution. This paper proposes a hardware mechanism, dynamic vectorization (DV), as a tool for quickly building up a large logical instruction window. Dynamic vectorization converts repetitive dynamic instruction sequences into vector form, enabling the processing of instructions from beyond the corresponding program loop to be overlapped with the loop. This enables vector-like execution of programs with relatively complex static control flow that may not be amenable to static, compile time vectorization. Experimental evaluation shows that a large fraction of the dynamic instructions of four of the six SPECInt92 programs can be captured in vector form. Three of these programs exhibit significant potential for ILP improvements from dynamic vectorization, with speedups of more than a factor of 2 in a scenario of realistic branch prediction and perfect memory disambiguation. Under perfect branch prediction conditions, a fourth program also shows well over a factor of 2 speedup from DV. The speedups are due to the overlap of post-loop processing with loop processing.

Software distributed shared memory (DSM) platforms on networks of workstations tolerate large network latencies by employing one of several weak memory consistency models. Fully asynchronous parallel iterative algorithms offer an additional degree of freedom to tolerate network latency. They behave correctly when supplied outdated shared data. However these algorithms can flood the network with messages in the presence of large delays. We propose a method of controlling asynchronous iterative methods wherein the reader of a shared datum imposes an upper bound on its age via use of a blocking Global Read primitive. This reduces the overall number of iteration is executed by the reader; thus controlling the amount of shared updates generated. Experiments for a fully asynchronous linear equation solver running on a network of 10 IBM RS/6000 workstations show that the proposed Global Read primitive provides significant performance improvement.