We present a flexible and scalable FPGA overlay architecture for data-flow applications. The overlay consists of a 2D grid of tiles consisting of compute units that can be exchanged at runtime. The overlay can deal with unbalanced data-flow graphs and is based on AXI-Stream to facilitate extensibility using IP and High-Level Synthesis. To support compute units that also require random access to the system memory, AXI4-ports can be enabled per tile at design-time. We implement a prototype of the overlay architecture tailored to the application domain of analytical query processing on a Xilinx Alveo U280 board. In this system, an overlay grid of$11\times 4$compute unit tiles occupies one-third of the available resources. with a design I/O throughput of$11\times 3.75\ \text{GB}/\mathrm{s}$. The overlay and HLS-based SIMD compute units provide full-throughput data processing, but are limited by the memory subsystem implemented with vendor IPs.

Today’s processor landscape is increasingly heterogeneous with the availability of co-processors. This landscape impacts query engines, as they need to be reworked to keep competitive performance by leveraging the underlying architectures. Such a rework might be costly if, for each external processor or SDK, peripheral components needed to be developed as well; resulting in redundant effort and adoption difficulties. In this paper, we propose an approach to overcome these shortcomings through ADAMANT – a query executor equipped with interfaces to plug-in new co-processors without reworking other components of a query engine. ADAMANT consists of 1) pluggable interfaces that allow interaction with co-processors, encapsulating operator implementations, and 2) a unified runtime that handles the execution on arbitrary co-processors, with a chunked execution model for scalable query processing. To evaluate ADAMANT’s versatility, we plug different implementations of a CPU/GPU-based system (using OpenCL, OpenMP, & CUDA) and analyze their performance on TPC-H queries. We identify a 4x performance difference between an arbitrary chunked execution vs. a more architecturally conscious pipelined execution. Furthermore, our comparisons with HeavyDB show complex performance variations from speed-ups up to a factor of 2x from our hardware-conscious execution. We envision initiatives like ADAMANT to ease the study of complex optimizations required in co-processor systems, paving the way for efficient and portable data management tools without cutbacks.

Abstract Using heterogeneous processing devices, like GPUs, to accelerate relational database operations is a well-known strategy. In this context, the operation is highly interesting for two reasons. Firstly, it incurs large processing costs. Secondly, its results (i.e., aggregates) are usually small, reducing data movement costs whose compensation is a major challenge for heterogeneous computing. Generally, for computation on GPUs, one relies either on sorting or hashing. Today, empirical results suggest that hash-based approaches are superior. However, by concept, hashing induces an unpredictable memory access pattern conflicting with the architecture of GPUs. This motivates studying why current sort-based approaches are generally inferior. Our results indicate that current sorting solutions cannot exploit the full parallel power of modern GPUs. Experimentally, we show that the issue arises from the need to synchronize parallel threads that access the shared memory location containing the aggregates via . Our quantification of the optimal performance motivates us to investigate how to minimize the overhead of atomics. This results in different variants using atomics, where the best variants almost mitigate the atomics overhead entirely. The results of a large-scale evaluation reveal that our approach achieves a 3x speed-up over existing sort-based approaches and up to 2x speed-up over hash-based approaches.

Abstract GPU accelerated query execution is still ongoing research in the database community, as GPUs continue to be heterogeneous in their architectures varying their capabilities (e.g., their newest selling point: tensor cores). Hence, many researchers come up with optimal operator implementations for a specific device generation involving tedious operator tuning by hand. Alternatively, there is a growing availability of GPU libraries providing optimized operators for various applications. However, the question arises of how mature these libraries are and whether they are fit to replace handwritten operator implementations not only w.r.t. implementation effort and portability but also performance. In this paper, we investigate various general-purpose libraries that are both portable and easy to use for arbitrary GPUs to test their production readiness on the example of database operations. To this end, we develop a framework to show the support of GPU libraries for database operations that allows a user to plug-in new libraries and custom-written code. Our framework allows for easy pluggability of new libraries for query execution using a simple task model. Using this framework, we develop multiple libraries (ArrayFire, Thrust, and boost.compute) supporting many database operations. We use these libraries to experiment with different devices to see the impact of the underlying device. Based on our experiments, we see a significant diversity in terms of performance among libraries. Furthermore, one of the fundamental database primitives—hashing, and thus hash joins—is currently not supported, leaving important tuning potential unused.

Many modern database engines use OpenCL to target heterogeneous hardware. Queries are evaluated by execution of chains of low-level operators. The common paradigm for OpenCL workloads facilitates communication between kernels using buffers in off-chip memory. This poses a severe performance limitation due to weak memory systems of FPGAs in contrast to the memory hierarchy available in CPUs and GPUs. To overcome this bottleneck, we propose the use of structural optimizations of kernel code. On-chip pipelining and code fusion are analyzed as alternatives to buffer-based inter-kernel communication. We assess the impact on resource utilization and system throughput and thereby demonstrate that properly structured code achieves a speedup of more than 4x over the default paradigm. This shows that it is essential for chains of kernels to consider not only optimization techniques for individual kernels, but also optimization of inter-kernel communication.