The currently dominant AI/ML workloads, such as Large Language Models (LLMs), rely on the efficient execution of General Matrix-Matrix Multiplication (GEMM) operations. Thus, most systems are equipped with dedicated matrix hardware accelerators based on square Systolic Arrays (SAs) of Processing Elements (PEs). While this organization was effective for traditional Deep Neural Networks (DNNs), LLMs introduce input-dependent and highly skewed matrices, leading to underutilized SA resources. To address this challenge, we propose SISA (Scale-In Systolic Array), a novel SA architecture that partitions the traditional square array into horizontal rectangular slabs. With minimal overhead, SISA exposes parallelism through independently scheduled slabs for efficient execution of small or skewed matrix shapes, while retaining full-array operation for large GEMMs. SISA achieves up to 8.52x speedup and 93% energy-delay-product (EDP) reduction for representative LLMs compared to a state-of-the-art monolithic SA with the same number of PEs.

Quantum computing has the potential to revolutionise multiple fields by solving complex problems that cannot be solved in reasonable time with current classical computers. Nevertheless, the development of quantum computers is still in its early stages and the available systems have still very limited resources. As such, currently, the most practical way to develop and test quantum algorithms is to use classical simulators of quantum computers. In addition, the development of new quantum computers and their components also depends on simulations. Given the characteristics of a quantum computer, their simulation is a very demanding application in terms of both computation and memory. As such, simulations do not scale well in current classical systems. Thus different optimisation and approximation techniques need to be applied at different levels. This review provides an overview of the components of a quantum computer, the levels at which these components and the whole quantum computer can be simulated, and an in-depth analysis of different state-of-the-art acceleration approaches. Besides the optimisations that can be performed at the algorithmic level, this review presents the most promising hardware-aware optimisations and future directions that can be explored for improving the performance and scalability of the simulations.

Convolutional Neural Networks (CNNs) serve various applications with diverse performance and resource requirements. Model-aware CNN accelerators best address these diverse requirements. These accelerators usually combine multiple dedicated Compute Engines (CEs). The flexibility of FieldProgrammable Gate Arrays (FPGAs) enables the design of such multiple Compute-Engine (multiple-CE) accelerators. However, existing multiple-CE accelerators differ in how they arrange their CEs and distribute the FPGA resources and CNN operators among the CEs. The design space of multiple-CE accelerators comprises numerous such arrangements, which makes a systematic identification of the best ones an open challenge. This paper proposes a Multiple-CE accelerator analytical Cost Model (MCCM) and an evaluation methodology built around MCCM. The model and methodology streamline the expression of any multiple-CE accelerator and provide a fast evaluation of its performance and efficiency. MCCM is in the order of$100000 \times$faster than traditional synthesis-based evaluation and has an average accuracy of$>90 \%$. The paper presents three use cases of MCCM. The first describes an end-to-end evaluation of state-of-the-art multiple-CE accelerators considering various metrics, CNN models, and resource budgets. The second describes fine-grained evaluation that helps identify performance bottlenecks of multiple-CE accelerators. The third demonstrates that MCCM fast evaluation enables exploring the vast design space of multiple-CE accelerators. These use cases show that no unique CE arrangement achieves the best results given different metrics, CNN models, and resource budgets. They also show that fast evaluation enables design space exploration, resulting in accelerator designs that outperform state-of-the-art ones. MCCM is available at https://github.com/fqararyah/MCCM.

The success of Artificial Intelligence (AI) applications is driven by efficient hardware accelerators. Recent trends show a rapid increase in the application demands, which in most cases surpass the available resources in the accelerators. As such, the efficient management of these limited resources becomes a critical factor in achieving high-performance.

Resource-efficient Convolutional Neural Networks (CNNs) are gaining more attention. These CNNs have relatively low computational and memory requirements. A common denominator among such CNNs is having more heterogeneity than traditional CNNs. This heterogeneity is present at two levels: intra-layer type and inter-layer type. Generic accelerators do not capture these levels of heterogeneity, which harms their efficiency. Consequently, researchers have proposed model-specific accelerators with dedicated engines. When designing an accelerator with dedicated engines, one option is to dedicate one engine per CNN layer. We refer to accelerators designed with this approach as single-engine single-layer (SESL). This approach enables optimizing each engine for its specific layer. However, such accelerators are resource-demanding and unscalable. Another option is to design a minimal number of dedicated engines such that each engine handles all layers of one type. We refer to these accelerators as single-engine multiple-layer (SEML). SEML accelerators capture the inter-layer-type but not the intra-layer-type heterogeneity. We propose the Fixed Budget Hybrid CNN Accelerator (FiBHA), a hybrid accelerator composed of an SESL part and an SEML part, each processing a subset of CNN layers. FiBHA captures more heterogeneity than SEML while being more resource-aware and scalable than SESL. Moreover, we propose a novel module, Fused Inverted Residual Bottleneck (FIRB), a fine-grained and memory-light SESL architecture building block. The proposed architecture is implemented and evaluated using high-level synthesis (HLS) on different Field Programmable Gate Arrays representing various resource budgets. Our evaluation shows that FiBHA improves the throughput by up to 4 x and 2.5 x compared to state-of-the-art SESL and SEML accelerators, respectively. Moreover, FiBHA reduces memory and energy consumption compared to an SEML accelerator. The evaluation also shows that FIRB reduces the required memory by up to 54%, and energy requirements by up to 35% compared to traditional pipelining.

The eProcessor project aims at creating a RISC-V full stack ecosystem. The eProcessor architecture combines a high-performance out-of-order core with energy-efficient accelerators for vector processing and artificial intelligence with reduced-precision functional units. The design of this architecture follows a hardware/software co-design approach with relevant application use cases from the high-performance computing, bioinformatics and artificial intelligence domains. Two eProcessor prototypes will be developed based on two fabricated eProcessor ASICs integrated into a computer-on-module.

Deep Learning (DL) applications are entering every part of our life given their ability to solve complex problems. Nevertheless, energy efficiency is still a major concern due to the large computational and memory requirements. State-of-the-art accelerators strive to address this issue by optimizing the architecture to the compute requirements of DL algorithms. However, there is always a mismatch between compute and memory requirements and what is offered by a particular design. A way to close this gap is by providing run-time adaptation or resource allocation to improve efficiency.

The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware platform, which leverages microserver technology to enable users to configure the hardware to meet the requirements of a diverse array of applications. Heterogeneous computing is used to boost performance and energy efficiency. In addition, the full spectrum of hardware accelerators is integrated, providing specialized ASICs as well as FPGAs for reconfigurable computing. The project's contributions span across trusted computing, remote attestation, and secure execution environments, with the ultimate goal of facilitating the design and deployment of robust and efficient AIoT systems. The overall architecture is validated on use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. Ten additional use cases are integrated via an open call, broadening the range of application areas.

Within VEDLIoT, a project targeting the development of energy-efficient Deep Learning for distributed AIoT applications, several accelerator platforms based on technologies like CPUs, embedded GPUs, FPGAs, or specialized ASICs are evaluated. The VEDLIoT approach is based on modular and scalable cognitive IoT hardware platforms. Modular microserver technology enables the integration of different, heterogeneous accelerators into one platform. Benchmarking of the different accelerators takes into account performance, energy efficiency and accuracy. The results in this paper provide a solid overview regarding available accelerator solutions and provide guidance for hardware selection for AIoT applications from far edge to cloud. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage. The focus is on the considerations of the performance and energy efficiency of hardware accelerators. Apart from the hardware and accelerator focus presented in this paper, the project also covers toolchain, security and safety aspects. The resulting technology is tested on a wide range of AIoT applications.

This book contains the proceedings of the 33rd edition of the International Conference on Field Programmable Logic and Applications (FPL) held in Gothenburg on September 4th to 8th, 2023.

Deep Learning (DL) is developing at an extremely fast pace. The increased number of applications, optimizations and hardware devices available, results in a multi-dimensional design space where the best performance is achieved with a detailed analysis of the hardware-software co-design process. Furthermore, the high demands for memory and the off-chip latency cost result in the on-chip memory becoming critical for achieving high performance and efficiency. In this work, we propose RAINBOW, a tool to assist in the hardware-software co-design for DL accelerators’ on-chip memory. The purpose is to help the design and/or deployment of a DL model to a dedicated accelerator. RAINBOW generates different analyses results and feeds them to the optimizers. The result is a heterogeneous execution plan combining different approaches and techniques depending on the dynamic requirements and constraints. In our analysis, we concluded that given the opportunity, RAINBOW’S heterogeneous plans are able to reduce the DRAM accesses to approximately half when compared to homogeneous plans.

In order to meet the increased computational demands and stricter power constraints of modern applications, architectures have evolved to include domain-specific accelerators. In order to design efficient accelerators, three main challenges need to be addressed: compute, memory, and control. Moreover, since SoCs usually contain multiple accelerators, selecting the right one for each task also become crucial. This becomes specially relevant in Flexible Processing Units (xPUs), processing units that provide multiple functionalities with the same hardware. While it is possible to use shared support components for all functionalities, this will lead to sub-optimal performance. In this work, we take one example of such xPU, and analyze the aspects which have not yet been fully addressed, showing that there is more potential to be exploited. By understanding the required memory patterns, we can achieve up to 72% speedup gains compared to using the memory support optimized for a different functionality. Furthermore, we propose an in-depth analysis of the different functionalities provided by the xPU. We then leverage the insights obtained from this analysis by providing a mechanism that selects the right functionality, maximizing hardware utilization.

The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide range of applications. VEDLIoT offers a complete design flow for Next-Generation IoT devices required for collaboratively solving complex Deep Learning applications across distributed systems. The methods are tested on various use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage with the first results available.

In order to deliver high performance efficiently, modern processors include dedicated hardware to accelerate different application domains. For example, several recent processors include dedicated Machine Learning (ML) accelerators. However, while adding dedicated hardware improves efficiency compared to general-purpose CPUs, it also requires a larger area, making it unfeasible for smaller devices. Therefore, exploring ways to use the existing hardware for different functionalities becomes desirable in those setups. In this work, we explore the reuse of the components in a Vector Processing Unit (VPU) to offer the functionality of a Systolic Array (SA) for General Matrix Multiplication (GEMM), a kernel extensively used in machine learning, big data, and scientific computing. This hybrid Vector-Systolic Architecture (VSA) can thus support Single Instruction Multiple Data (SIMD) instruction extensions with the VPU functionality and efficiently compute GEMM with the SA functionality. We present an implementation of VSA as a RISC-V co-processor that adds minimal hardware overhead of less than 0.1% compared to a baseline RISC-V implementation with a VPU. In our evaluation using different Deep Neural Network (DNN) models, VSA shows a speedup of up to 3.5x and a reduction in energy consumption of up to 70%.

Seeking the “sweet spot” in the accuracy-efficiency trade-off is increasing the heterogeneity of state-of-the-art Convolutional Neural Networks (CNNs). Such CNN models exhibit heterogeneity at two levels: intra- and inter-layer-type. Generic accelerators do not capture these levels of heterogeneity. Consequently, researchers have proposed model-specific accelerators with dedicated modules or engines. The proposed accelerators belong to two categories at the two ends of the design spectrum. In the first category, the accelerators contain a minimal number of dedicated engines such that all the layers of one type (e.g. depthwise convolutions) are handled by one engine. In the second, they have one dedicated engine per layer. While the first category addresses the inter-layer-type heterogeneity, it cannot capture the heterogeneity among layers of the same type. The second category is resource-demanding. In this paper, we propose a hybrid architecture that combines design concepts from both categories in a way that captures more heterogeneity than the first category and is more resource-efficient than the second. To derive a hybrid accelerator given a fixed resource budget, we propose a heuristic that splits the CNN and the available resources between the components of the hybrid architecture. The proposed architecture is implemented and evaluated using high-level synthesis (HLS) on an FPGA. For a fixed hardware budget, the hybrid accelerator achieves up to 1.7x and 4.1x of the throughput achieved by state-of-the-art accelerators of the two categories.

No abstract available.

The LEGaTO project leverages task-based programming models to provide a software ecosystem for Made in-Europe heterogeneous hardware composed of CPUs, GPUs, FPGAs and dataflow engines. The aim is to attain one order of magnitude energy savings from the edge to the converged cloud/HPC, balanced with the security and resilience challenges. LEGaTO is an ongoing three-year EU H2020 project started in December 2017.

This paper considers a hybrid memory system composed of memory technologies with different characteristics; in particular a small, near memory exhibiting high bandwidth, i.e., 3D-stacked DRAM, and a larger, far memory offering capacity at lower bandwidth, i.e., off-chip DRAM. In the past, the near memory of such a system has been used either as a DRAM cache or as part of a flat address space combined with a migration mechanism. Caches and migration offer different tradeoffs (between performance, main memory capacity, data transfer costs, etc.) and share similar challenges related to data-transfer granularity and metadata management. This paper proposes Hybrid2, a new hybrid memory system architecture that combines a DRAM cache with a migration scheme. Hybrid2does not deny valuable capacity from the memory system because it uses only a small fraction of the near memory as a DRAM cache; 64MB in our experiments. It further leverages the DRAM cache as a staging area to select the data most suitable for migration. Finally, Hybrid2alleviates the metadata overheads of both DRAM caches and migration using a common mechanism. Using near to far memory ratios of 1:16, 1:8 and 1:4 in our experiments, Hybrid2on average outperforms current state-of-the-art migration schemes by 7.9%, 9.1% and 6.4%, respectively. In the same system configurations, compared to DRAM caches Hybrid2gives away on average only 0.3%, 1.2%, and 5.3% of performance offering 5.9%, 12.1%, and 24.6% more main memory capacity, respectively.

This paper describes Approximate Value Reconstruction (AVR), an architecture for approximate memory compression. AVR reduces the memory traffic of applications that tolerate approximations in their dataset. Thereby, it utilizes more efficiently off-chip bandwidth improving significantly system performance and energy efficiency. AVR compresses memory blocks using low latency downsampling that exploits similarities between neighboring values and achieves aggressive compression ratios, up to 16:1 in our implementation. The proposed AVR architecture supports our compression scheme maximizing its effect and minimizing its overheads by (i) co-locating in the Last Level Cache (LLC) compressed and uncompressed data, (ii) efficiently handling LLC evictions, (iii) keeping track of badly compressed memory blocks, and (iv) avoiding LLC pollution with unwanted decompressed data. For applications that tolerate aggressive approximation in large fractions of their data, AVR reduces memory traffic by up to 70%, execution time by up to 55%, and energy costs by up to 20% introducing less than 1% error to the application output.

Although 3D-stacked DRAM offers substantially higher bandwidth than commodity DDR DIMMs, it cannot yet provide the necessary capacity to replace the bulk of the memory. A promising alternative is to use flat address space, hybrid memory systems of two or more levels, each exhibiting different performance characteristics. One such existing approach employs a near, high bandwidth 3D-stacked memory, placed on top of the processor die, combined with a far, commodity DDR memory, placed off-chip. Migrating data from the far to the near memory has significant performance potential, but also entails overheads, which may diminish migration benefits or even lead to performance degradation. This paper describes a new data migration scheme for hybrid memory systems that takes into account the above overheads and improves migration efficiency and effectiveness. It is based on the observation that migrating memory segments, which are (partly) present in the Last-Level Cache (LLC) introduces lower migration traffic. Our approach relies on the state of the LLC cachelines to predict future reuse and select memory segments for migration. Thereby, the segments are migrated when present (at least partly) in the LLC incurring lower cost. Our experiments confirm that our approach outperforms current state-of-the art migration designs improving system performance by 12.1% and reducing memory system dynamic energy by 13.2%.

Heterogeneous multicores offer flexibility in the form of different core types and Dynamic Voltage and Frequency Scaling (DVFS), defining a vast configuration space. The optimal configuration choice is not always straightforward, even for single applications, and becomes a very difficult problem for dynamically changing scenarios of concurrent applications with unpredictable spawn and termination times and individual performance requirements. This article proposes an integrated approach for runtime decision making for energy efficiency on such systems. The approach consists of a model that predicts performance and power for any possible decision and low-complexity heuristics that use this model to evaluate a subset of possible decisions to choose the best. The model predicts performance by projecting standalone application profiling data to the current status of the system and power by using a set of platform-specific parameters that are determined only once for a given system and are independent of the application mix. Our approach is evaluated with a plethora of dynamic, multi-application scenarios. When considering best effort performance to be adequate, our runtime achieves on average 3% higher energy efficiency compared to the powersave governor and 2× better compared to the other linux governors. Moreover, when also considering individual applications’ performance requirements, our runtime is able to satisfy them, giving away 18% of the system’s energy efficiency compared to the powersave, which, however, misses the performance targets by 23%; at the same time, our runtime maintains an efficiency advantage of about 55% compared to the other governors, which also satisfy the performance constraints.

DRAM caches have shown excellent potential in capturing the spatial and temporal data locality of applications capitalizing on advances of 3D-stacking technology; however, they are still far from their ideal performance. Besides the unavoidable DRAM access to fetch the requested data, tag access is in the critical path, adding significant latency and energy costs. Existing approaches are not able to remove these overheads and in some cases limit DRAM cache design options. For instance, caching DRAM cache tags adds constant latency to every access; accessing the DRAM cache using the TLB calls for OS support and DRAM cachelines as large as a page; reusing the last-level cache (LLC) tags to access the DRAM cache limits LLC performance as it requires indexing the LLC using higher-order address bits. In this article, we introduce Decoupled Fused Cache , a DRAM cache design that alleviates the cost of tag accesses by fusing DRAM cache tags with the tags of the on-chip LLC without affecting LLC performance. In essence, the Decoupled Fused Cache relies in most cases on the LLC tag access to retrieve the required information for accessing the DRAM cache while avoiding additional overheads. Compared to current DRAM cache designs of the same cacheline size, Decoupled Fused Cache improves system performance by 6% on average and by 16% to 18% for large cacheline sizes. Finally, Decoupled Fused Cache reduces DRAM cache traffic by 18% and DRAM cache energy consumption by 7%.

LEGaTO is a three-year EU H2020 project which started in December 2017. The LEGaTO project will leverage task-based programming models to provide a software ecosystem for Made-in-Europe heterogeneous hardware composed of CPUs, GPUs, FPGAs and dataflow engines. The aim is to attain one order of magnitude energy savings from the edge to the converged cloud/HPC.

The explosive growth of Internet-connected devices will soon result in a flood of generated data, which will increase the demand for network bandwidth as well as compute power to process the generated data. Consequently, there is a need for more energy efficient servers to empower traditional centralized Cloud data-centers as well as emerging decentralized data-centers at the Edges of the Cloud. In this paper, we present our approach, which aims at developing a new class of micro-servers - the UniServer - that exceed the conservative energy and performance scaling boundaries by introducing novel mechanisms at all layers of the design stack. The main idea lies on the realization of the intrinsic hardware heterogeneity and the development of mechanisms that will automatically expose the unique varying capabilities of each hardware. Low overhead schemes are employed to monitor and predict the hardware behavior and report it to the system software. The system software including a virtualization and resource management layer is responsible for optimizing the system operation in terms of energy or performance, while guaranteeing non-disruptive operation under the extended operating points. Our characterization results on a 64-bit ARMv8 micro-server in 28nm process reveal large voltage margins in terms of Vmin variation among the 8 cores of the CPU chip, among three different sigma chips, and among different benchmarks with the potential to obtain up-to 38.8% energy savings. Similarly, DRAM characterizations show that refresh rate and voltage can be relaxed by 35x and 5%, respectively, leading to 23.2% power savings on average.

DRAM caches have been shown to be an effective way to utilize the bandwidth and capacity of 3D stacked DRAM. Although they can capture the spatial and temporal data locality of applications, their access latency is still substantially higher than conventional on-chip SRAM caches. Moreover, their tag access latency and storage overheads are excessive. Storing tags for a large DRAM cache in SRAM is impractical as it would occupy a significant fraction of the processor chip. Storing them in the DRAM itself incurs high access overheads. Attempting to cache the DRAM tags on the processor adds a constant delay to the access time. In this paper, we introduce FusionCache, a DRAM cache that offers more efficient tag accesses by fusing DRAM cache tags with the tags of the on-chip Last Level Cache (LLC). We observe that, in an inclusive cache model where the DRAM cachelines are multiples of on-chip SRAM cachelines, LLC tags could be re-purposed to access a large part of the DRAM cache contents. Then, accessing DRAM cache tags incurs zero additional latency in the common case.

Increasing user demands for timely and complex analysis of large amounts of collected data in the IoT era are pushing the computing servers from the cloud to the Edge where the energy budgets are much tighter. Building a HPC for the Edge is thus a considerable challenge. In terms of processors, we are observing a trend of making traditional high-performance processors more energy-efficient. On the other end of the spectrum we are observing the development of higher performance versions of low-power mobile processors.In this work we consider using commodity sub-5W processors as a building block for a Edge HPC cluster. We consider the processors in mobile phones, tablets, and low-end laptops. We evaluate the energy-to-solution and energy efficiency of different commodity sub-5W processors against a regular 100 Watt processor using HPC applications. We observed that commodity sub-5W processors, which have been designed for a totally different purpose, can actually deliver an average reduction of 70% of energy-to-solution for the ten HPC applications tested. These results are very encouraging for the use of simple low-cost sub-5W processors as building blocks for Edge HPC systems.

High throughput and low latency stream aggregation - and stream processing in general - is critical for many emerging applications that analyze massive volumes of continuously produced data on-the-fly, to make real time decisions. In many cases, high speed stream aggregation can be achieved incrementally by computing partial results for multiple windows. However, for particular problems, storing all incoming raw data to a single window before processing is more efficient or even the only option. This paper presents the first FPGA-based single window stream aggregation design. Using Maxeler's dataflow engines (DFEs), up to 8 million tuples-per-second can be processed (1.1 Gbps) offering 1-2 orders of magnitude higher throughput than a state-of-the-art stream processing software system. DFEs have a direct feed of incoming data from the network as well as direct access to off-chip DRAM processing a tuple in less than 4 μsec, 4 orders of magnitude lower latency than software. The proposed approach is able to support challenging queries required in realistic stream processing problems (e.g. holistic functions). Our design offers aggregation for up to 1 million concurrently active keys and handles large windows storing up to 6144 values (24 KB) per key.

As the number of cores increases in a single chip processor, several challenges arise: wire delays, contention for out-of-chip accesses, and core heterogeneity. In order to address these issues and the applications demands, future large-scale many-core processors are expected to be organized as a collection of NUMA clusters of heterogeneous cores. In this work we propose a scheduler that takes into account the non-uniform memory latency, the heterogeneity of the cores, and the contention to the memory controller to find the best matching core for the application's memory and compute requirements. Scheduler decisions are based on an on-line classification process that determines applications requirements either as memory- or compute-bound. We evaluate our proposed scheduler on the 48-core Intel SCC using applications from SPEC CPU2006 benchmark suite. Our results show that even when all cores are busy, migrating processes to cores that match better the requirements of applications results in overall performance improvement. In particular we observed a reduction of the execution time from 15% to 36% compared to a random static scheduling policy.

SWITCHES is a task-based dataflow runtime that implements a lightweight distributed triggering system for runtime dependence resolution and uses static scheduling and compile-time assignment policies to reduce runtime overheads. Unlike other systems, the granularity of loop-tasks can be increased to favor data-locality, even when having dependences across different loops. SWITCHES introduces explicit task resource allocation mechanisms for efficient allocation of resources and adopts the latest OpenMP Application Programming Interface (API), as to maintain high levels of programming productivity. It provides a source-to-source tool that automatically produces thread-based code. Performance on an Intel Xeon-Phi shows good scalability and surpasses OpenMP by an average of 32%.

Single Instruction Multiple Data (SIMD) Extensions become popular in computer architectures as a simple and efficient way to exploit the data parallelism hidden in applications. The compiler research community has proposed automatic vectorization as the answer to the complexity of low-level programming of vector units. Despite recent advances in compilation techniques, modern compilers miss opportunities to automatically vectorize code. One of the biggest challenges is to evaluate the changes against the best hand-written code. This paper presents a benchmark suite based on video encoding and decoding kernels. The suite contains hand-written versions of the kernels provided by the open source community that support the latest SIMD extensions. The paper also compares the performance of three available compilers (GCC, LLVM, and ICC) against the hand-written kernels. A performance evaluation, using an i7-4790 processor, shows that the auto-vectorized version produced by the best compiler achieves on average only 28% of the hand-tuned kernels.

Summary form only given. Processors have evolved dramatically in the last years and current multicore systems deliver very high performance. We are observing a rapid increase in the number of cores per processor thus resulting in more dense and powerful systems. Nevertheless,this evolution will meet several challenges such as power consumption, and reliability. It is expected that, in order to improve the efficiency, future processors will contain units that are able to operate at a very low power consumption with the draw back of not guaranteeing the correctness of the produced results. This model is known as Approximate Computing. One interesting approach to exploit Approximate Computing is to make applications aware of the errors and react accordingly. For this work we focus on the Decision Support System Workloads and in particular the standard TPC-H set of queries. We first define a metric that quantifies the correctness of a query result - Quality of Result (QoR). Using this metric we analyse the impact of relaxing the correctness in the DBMS on the accuracy of the query results. In order to improve the accuracy of the results we propose a dynamic adaptive technique that is implemented as a tool above the DBMS. Using heuristics, this tool spawns a number of replica query executions on different cores and combines the results as to improve the accuracy. We evaluated our technique using real TPC-H queries and data on PostgreSQL with a simple fault-injection to emulate the Approximate Computing model. The results show that for the selected scenarios, the proposed technique is able to increase the QoR with a cost in parallel resources smaller than any alternative static approach. The results are very encouraging since the QoR is within 7% of the best possible.

The current trend in processor design is to increase the number of cores as to achieve a desired performance. While having a large number of cores on a chip seems to be feasible in terms of the hardware, the development of the software that is able to exploit that parallelism is one of the biggest challenges. In this paper we propose a Data-Flow based system that can be used to exploit the parallelism in large-scale many-core processors in an effective and efficient way. Our proposed system - TFlux SCC - is an extension of the TFlux Data-Driven Multithreading (DDM), which evolved to exploit the parallelism of the 48-core Intel Single-chip Cloud Computing (SCC) processor. With TFlux SCC we achieve scalable performance using a global address space without the need of cache-coherency support. Our scalability study shows that application's performance can scale, with speedup results reaching up to 48x for 48 cores. The findings of this work provide insight towards what a Data-Flow implementation requires and what not from a many-core architecture in order to scale the performance.

Thanks to the improvements in semiconductor technologies, extreme-scale systems such as teradevices (i.e., composed by 1000 billion of transistors) will enable systems with 1000+ general purpose cores per chip, probably by 2020. Three major challenges have been identified: programmability, manageable architecture design, and reliability. TERAFLUX is a Future and Emerging Technology (FET) large-scale project funded by the European Union, which addresses such challenges at once by leveraging the dataflow principles. This paper describes the project and provides an overview of the research carried out by the TERAFLUX consortium.

Decision Support System (DSS) workloads are known to be one of the most time-consuming database workloads that process large data sets. Traditionally, DSS queries have been accelerated using large-scale multiprocessors. In this work we exploit the benefits of using future many-core architectures, more specifically on-chip clustered many-core architectures. To achieve this goal we propose different representative data parallel versions of the original database scan and join algorithms. We also study the impact on the performance when on-chip memory, shared among all cores, is used as a prefetching buffer. For our experiments we study the behaviour of three queries from the standard DSS benchmark TPC-H executing on the Intel Single chip Cloud Computer experimental processor (Intel SCC). Our results show that parallelism can be well exploited by such architectures and how important it is to have a balance between computation and data intensity. Moreover, from our experimental results we show that performance improvement of 5x and 10x for the corresponding query implementation without data prefetching. Finally we show how we could efficiently use the system in order to achieve high power-performance efficiency when using the proposed prefetching buffer.

Low-Density Parity-Check (LDPC) codes are powerful error correcting codes used today in communication standards such as DVB-S2 and WiMAX to transmit data inside noisy channels with high error probability. LDPC decoding is computationally demanding and requires irregular accesses to memory which makes it suitable for parallelization. The recent introduction of the many-core Single-chip Cloud Computer (SCC) from Intel research Labs has created new opportunities and also new challenges for programmers that wish to exploit conveniently the high level of parallelism available in the architecture. In this paper we propose three different implementations: a distributed, a shared and a multi-codeword implementation, for LDPC decoding algorithms that explore the Intel SCC scaling opportunities. From the experimental results we observed that the distributed memory model couldn't scale due to the large number of messages exchanged by the parallel kernels, while the shared memory model had a limited scaling due to the overhead added by the uncacheable shared memory. On the other hand, the multi-codeword implementation scales almost linearly acheving a relative throughput of 28 for 32 cores.

We are currently faced with the situation where applications have increasing computational demands and there is a wide selection of parallel processor systems. In this paper we focus on exploiting fine-grain parallelism for a demanding bioinformatics application - MrBayes - and its phylogenetic likelihood functions (PLF) using different architectures. Our experiments compare side-by-side the scalability and performance achieved using general-purpose multi-core processors, the cell/BE, and graphics processor units (GPU). The results indicate that all processors scale well for larger computation and data sets. Also, GPU and Cell/BE processors achieve the best improvement for the parallel code section. Nevertheless, data transfers and the execution of the serial portion of the code are the reasons for their poor overall performance. The general-purpose multi-core processors prove to be simpler to program and provide the best balance between an efficient parallel and serial execution, resulting in the largest speedup.

The need to exploit multi-core systems for parallel processing has revived the concept of dataflow. In particular, the dataflow multithreading architectures have proven to be good candidates for these systems. In this work we propose an abstraction layer that enables compiling and running a program written for an abstract dataflow multithreading architecture on different implementations. More specifically, we present a set of compiler directives that provide the programmer with the means to express most types of dependencies between code segments. In addition, we present the corresponding toolchain that transforms this code into a form that can be compiled for different implementations of the model. As a case study for this work, we present the usage of the toolchain for the TFlux and DTA architectures.

In this paper we present thread flux (TFlux), a complete system that supports the data-driven multithreading (DDM) model of execution. TFlux virtualizes any details of the underlying system therefore offering the same programming model independently of the architecture. To achieve this goal, TFlux has a runtime support that is built on top of a commodity operating system. Scheduling of threads is performed by the thread synchronization unit (TSU), which can be implemented either as a hardware or a software module. In addition, TFlux includes a preprocessor that, along with a set of simple compiler directives, allows the user to easily develop DDM programs. The preprocessor then automatically produces the TFlux code, which can be compiled using any commodity C compiler, therefore automatically producing code to any ISA. TFlux has been validated on three platforms. A Simics-based multicore system with a TSU hardware module (TFluxHard), a commodity 8-core Intel Core2 QuadCore-based system with a software TSU module (TFluxSoft), and a Cell/BE system with a software TSU module (TFluxCell). The experimental results show that the performance achieved is close to linear speedup, on average 21x for the 27 nodes TFluxHard, and 4.4x on a 6 nodes TFluxSoft and TFluxCell. Most importantly, the observed speedup is stable across the different platforms thus allowing the benefits of DDM to be exploited on different commodity systems.

Due to limitations in the traditional microprocessor design, such as high complexity and power, all current commercial high-end processors contain multiple cores on the same chip (multicore). This trend is expected to continue resulting in increasing number of cores on the chip. While these cores may be used to achieve higher throughput, improving the execution of a single application requires careful coding and usually results in smaller benefit as the scale of the system is increased. In this work we propose an alternative way to exploit the multicore technology where certain cores execute code which indirectly improves the performance of the application. We call this approach the Helper Core approach. The main contribution of this work is the proposal and evaluation of HelperCoreDB, a Helper Core designed to improve the performance of database workloads by performing efficient data prefetching. We validate the proposed approach using a HelperCoreDBimplementation for the PostgreSQL DBMS. The approach is evaluated with native execution on a dual core processor system using the standard TPC-H benchmark. The experimental results show a significant performance improvement specially considering that the baseline system is a modern optimized processor that includes advanced features such as hardware prefetching. In particular, for the 22 queries of TPC-H, our approach achieves a large reduction of the secondary cache misses, 75% on average, and an improvement in the execution time of up to 21.5%.

The increased complexity and operating frequency in current microprocessors is resulting in a decrease in the performance improvements. In order to keep up with the expected performance gains, major manufacturers have started to offer chip-multiprocessor architectures. Nevertheless, the integration of several cores on the same chip leads to increased heat dissipation and consequently additional costs, decrease of the reliability, and performance loss, among others. In this paper we propose thermal-aware scheduling (TAS) a technique that aims to minimize all these problems. When assigning processes to cores, TAS takes their temperature into account avoiding thermal violation events. As a side effect, the performance is improved. Simulation results show that for a 25-core CMP, a simple TAS heuristic reduces the performance loss that is introduced by excessive temperature, from 52% to 18%. At the same time, TAS decreases the chip's temperature by 2.6degC

Microprocessor development costs are considerably high. To minimize these costs, manufacturers produce a single design that better satisfies, in average, a wide range of applications. Nevertheless, as applications have different characteristics and users have different demands, this single design is suboptimal in many situations. As a consequence there is a need to design microprocessors that are "flexible" and adapt to the different scenarios. This paper presents a simple multiple parameter selection algorithm to achieve high power-performance efficiency gains for each application. The proposed algorithm is tested using detailed simulation and a set of different applications from multimedia, scientific and database workloads. The experimental results show that when the selection algorithm is applied to a baseline configuration optimized for the corresponding workload, the efficiency gains are up to 30%. When the baseline used is optimized for a different workload then the gains may go up by an order of magnitude. The effectiveness of the simple selection algorithm with the proposed optimizations is impressive as the efficiency gains obtained represent 96% of the gains obtained with an ideal parameter selection algorithm

The properties of computer system such as performance, energy, reliability, etc. are commonly evaluated by running benchmarks. However, the benchmarking process is complicated to set-up and use and running the benchmarks takes a substantial amount of time. Furthermore, when designing a computer, architects resort to simulation of the system, increasing the benchmarking time by several orders of magnitude. This problem can be alleviated by reducing the execution time of the benchmark suite. In this paper, we investigate a method to reduce the number of queries in TPC-H, a decision support system benchmark. Our evaluation shows that out of the 22 original queries, a subset of only 6 achieves a high representativeness at only 40% of the original execution time. The results also show that subsets built for a particular database size may be used for evaluating computer systems with other database sizes. Finally, we validate our approach against the full benchmark execution for a set of architecture case studies. The validation results show that the proposed subsets lead to similar conclusions than the ones drawn using the full benchmark.

This paper describes the data-driven multithreading (DDM) model and how it may be implemented using off-the-shelf microprocessors. Data-driven multithreading is a nonblocking multithreading execution model that tolerates internode latency by scheduling threads for execution based on data availability. Scheduling based on data availability can be used to exploit cache management policies that reduce significantly cache misses. Such policies include firing a thread for execution only if its data is already placed in the cache. We call this cache management policy the CacheFlow policy. The core of the DDM implementation presented is a memory mapped hardware module that is attached directly to the processor's bus. This module is responsible for thread scheduling and is known as the thread synchronization unit (TSU). The evaluation of DDM was performed using simulation of the data-driven network of workstations (D2NOW). D2NOW is a DDM implementation built out of regular workstations augmented with the TSU. The simulation was performed for nine scientific applications, seven of which belong to the SPLASH-2 suite. The results show that DDM can tolerate well both the communication and synchronization latency. Overall, for 16 and 32-node D2NOW machines the speedup observed was 14.4 and 26.0, respectively

Current microprocessors are optimized for the average use. Nevertheless, it is known that different applications impose different demands on the system. This work focuses on the reconfiguration of the first-level caches. In order to achieve good performance, the first-level cache is split physically into two parts, one for instruction and one for data. This separation has the benefit of avoiding interference between instructions and data. Nevertheless, this separation is strict and determined at design-time. In this work we show a cache design that is able to change the split dynamically at runtime. The proposed design was tested using simulation of a variety of benchmark applications from the MiBench suite on two baseline architectures: embeddedXScale and high-end PowerPC. The results show that, while the average misses rate reduction may seem small; certain applications show a benefit larger than 90%. For miss rate reduction, the dynamic split cache seems to be more relevant for the cache with the smaller associativity (PowerPC). Lastly, the dynamic split cache was also used to reduce the energy consumption without loss of performance. This feature resulted in a significant energy reduction and the results showed that it has a bigger impact for the caches with larger associativity (42% energy reduction for the XScale and 28% for the PowerPC for a large data set size).

Graphics processors are designed to perform many floating-point operations per second. Consequently, they are an attractive architecture for high-performance computing at a low cost. Nevertheless, it is still not very clear how to exploit all their potential for general-purpose applications. In this work we present a comprehensive study of the performance of an application executing on the GPU. In addition, we analyze the possibility of using the graphics card to extend the life-time of a computer system. In our experiments we compare the execution on a mid-class GPU (NVIDIA GeForce FX 5700LE) with a high-end CPU (Pentium 4 3.2 GHz). The results show that to achieve high speedup with the GPU you need to: (1) format the vectors into two-dimensional arrays; (2) process large data arrays; and (3) perform a considerable amount of operations per data element. Finally, we study the performance when upgrading a low-end system by simply adding a GPU. This solution is cheaper, results in smaller power consumption and achieves higher speedup (8.1x versus 1.3x) than a full upgrade to a new high-end system.

High-performance processors usually require large amounts of power for their operation. Adapting resources in such processors is very relevant as each application and distinct use of the system has a different demand. A frequent use of adaptation is to have the processor operating in either high-performance or low-power mode. In this work we analyze how power and performance change while modifying most of the resources in a processor. This leads to identifying which resources are more important for adaptation. This analysis was performed for three distinct workloads: multimedia, scientific, and database. Four criteria were defined to limit the adaptation range considering the cost and/or the efficiency of the system. The results of our experiments showed that the resource to be adapted depends on the: (1) operating mode (e.g. low-power or high-performance); (2) the workload; and (3) the criteria. Overall, for our baseline processor configuration, the dominant resources to be adapted are the voltage-frequency and the first-level instruction cache. Adapting resources may lead to an increase in the performance by up to 44% or a reduction in power by up to 93% with no restrictions. For configurations of the same cost and efficiency as the baseline, adaptation of resources may improve the performance by 33% or reduce the power by 34%.

While database workloads consume a major fraction of the cycles in today's machines, there are only a few public-domain performance studies that characterize in detail how these workloads exercise the machines. This fact is due to the complexity of setting up and tuning database workloads, the high cost of the equipment required to evaluate them, and the frequent use of proprietary systems. In this paper, we help redress this problem by presenting a detailed performance characterization of the TPC-D benchmark running on a 4-processor Pentium Pro SMP multiprocessor with Windows NT and Microsoft's SQL Server. We use the Pentium Pro built-in hardware event counters and software tools that monitor system activity. Our results show that TPC-D queries have a relatively low CPI. The CPIs, which are 1.27 on average for the 17 read-only queries, are comparable to values observed for technical workloads. The major sources of processor stall cycles are the instruction fetch bottleneck and data misses in the secondary cache. Kernel time is negligible, as queries spend less than 6% of their time on average in the kernel. Other results show that branch prediction is effective in TPC-D and that the exclusive state in the cache tags is largely unnecessary. Finally, we compare our results to the ones published for TPC-C.

Dramatic increases in the main-memory size of computers is allowing some applications to shift their main data storage area from disk to main memory and, as a result, increase their performance. This trend is at work in some databases, resulting in what is called memory-resident databases. However, because of the increasing gap between processor and main memory speed, in these systems, effective use of the cache hierarchy is crucial to high performance. Unfortunately, there has been relatively little work on building cache-friendly database systems. We present several cache-oriented optimizations to enable effective exploitation of caches in memory-resident decision support databases. The main optimization involves developing a query optimizer that includes the cost of cache misses in its cost metrics. The other optimizations are sophisticated data blocking and software prefetching. These optimizations require no custom-designed hardware support and are effective for the more complicated TPC-D queries. In a simple database, these queries run about 13% faster with the cache-oriented optimizer and blocking, and a total of 31% faster if, in addition, we add prefetching. The effectiveness of these optimizations is stable across a range of cache sizes, cache line sizes, and miss penalties.

Although cache-coherent shared-memory multiprocessors are often used to run commercial workloads, little work has been done to characterize how well these machines support such workloads. In particular, we do not have much insight into the demands of commercial workloads on the memory subsystem of these machines. In this paper, we analyze in detail the memory access patterns of several queries that are representative of Decision Support System (DSS) databases. Our analysis shows that the memory use of queries differs largely depending on how the queries access the database data, namely via indices or by sequentially scanning the records. The former queries, which we call Index queries, suffer most of their shared-data misses on indices and on lock-related metadata structures. The latter queries, which we call Sequential queries, suffer most of their shared-data misses on the database records as they are scanned. An analysis of the data locality in the queries shows that both Index and Sequential queries exhibit spatial locality and, therefore, can benefit from relatively long cache lines. Interestingly, shared data is reused very little inside queries. However, there is data reuse across Sequential queries. Finally, we show that the performance of Sequential queries can be improved moderately with data prefetching.