Creating dataflow circuits with HLS tools to exploit memory- or task-level parallelism is challenging. These tools can generate functional designs easily, but optimizing them for performance often requires extensive code restructuring, pragma tuning, and even RTL integration. Writing low-level RTL, however, is rarely productive. We propose Chext, a domain-specific language that provides a thin abstraction layer over RTL for building complex dataflow circuits with elastic interfaces. Chext enables a safe and agile development flow with its novel approach and offers a rich component library to accelerate design, making such circuits more accessible on FPGAs. In this paper, we introduce Chext, outline its design principles, and present two case studies comparing it with Vitis HLS.

Modern FPGAs integrate High Bandwidth Memory (HBM), offering up to 12× the DDR bandwidth distributed across multiple memory interfaces. To utilize the most of HBM's theoretical bandwidth, accelerators typically issue long bursts and exploit data locality. However, some applications like sparse matrix-vector multiplication (SpMV) and graph analytics often exhibit irregular, nonbursting memory access patterns, which hinder performance. Additionally, the HBM interconnect, essential for accessing multiple interfaces, may stall requests under certain conditions. This work introduces HBMex, a novel module designed to enhance HBM throughput for accelerators with irregular access patterns. Positioned between the accelerator and the HBM, HBMex improves parallelism by distributing memory requests across interfaces and mitigates stalls caused by the interconnect. We evaluate HBMex using memory access microbenchmarks and an SpMV accelerator, demonstrating throughput gains of up to 37% across real-world workloads compared to vendor-provided solutions. HBMex is distributed as a highly-configurable and open-source RTL generator.

High-level synthesis (HLS) helps to develop hard-ware accelerators for field-programmable gate arrays (FPGAs) using C/C++ descriptions. HLS is tailored to exploit instruction-level parallelism and, where available, data-level parallelism in applications. Yet, some applications mostly display another type of parallelism known as task-level parallelism (TLP): they may have massive amounts of available parallelism but paralleliz-able execution threads follow starkly different control paths or make completely independent memory accesses. Alas, there is very limited support for TLP in HLS tools (with much of the support being for statically scheduled coarse-grained tasks) whereas TLP is widely supported on conventional CPU platforms via libraries like OpenCilk, Intel Threading Building Blocks, and OpenMP. In this paper, we introduce a framework for supporting software-like TLP on FPGAs. The framework provides a parameterized architectural template that implements all hardware modules needed to support TLP primitives and task management. The emphasis is on providing programmers with a software-like experience: limited hardware resource constraints should only impact performance and never functionality. For this, all queues in the limited BRAMs are virtually extended to larger HBM/DDR FPGA memory and potentially beyond. We provide an open-source Chisel hardware generator that creates a dedicated task management system from a given Cilk-like application description. It is then straightforward to integrate it with HLS-based processing elements to realize full TLP-enabled applications on FPGAs. In the evaluation, we focus on the efficiency and scalability of our hardware task management system and compare it with OpenCilk, a recent software TLP framework. The results show that the architectural building blocks consume a small percentage of resources on modern FPGAs designed for data centres and achieve linear hardware scalability. For a reasonable range of parameters, the system shows near-perfect efficiency and speedup scales linearly when increasing the number of processing elements.

Shared Virtual Memory (SVM) is a mechanism that allows host-side applications and FPGA accelerators to access the same virtual address space. It enables accelerating algorithms with unpredictable memory access patterns by making transparent pointer sharing possible. Even for applications with predictable memory access patterns, SVM helps by eliminating manual data movement. FlexiMem is a customizable SVM system that uses FPGA-addressable memory resources to store virtual memory pages, rather than directly accessing the host memory, to achieve high throughput and low latency. On the FPGA side, FlexiMem features a highly-flexible interconnect that, for the first time, allows for configuring address and payload data paths independently for SVM systems. It supports multiple master and slave devices sharing the same address space, such as accelerators issuing many memory requests in parallel to multiple memory banks. With our interconnect design, we provide numerous specialization dimensions to optimize the SVM system for a given workload. For example, irregular applications with short accesses to memory require an address translation for each data transfer. Such applications can take advantage of a highly parallelized address data path and an increased number of TLBs working in parallel. In contrast, regular and bursting applications transfer a small number of address packets per many data packets, and they can tolerate a lightweight address data path with a single translation unit. FlexiMem also provides address translation units with reconfigurable capacities and page sizes to be tailored to the needs of the application. On the host side, FlexiMem leverages the Linux userfaultfd API and vendor-provided IPs and drivers for automatic data movement initiated by software. Blocks of memory allocated by the FlexiMem API can be passed freely to the FPGA or other host-side libraries, without requiring any kind of explicit data movement. We evaluate several experimental setups with various FlexiMem configurations to showcase the effect of customizability on performance.

Multi-pod systolic arrays are emerging as the architecture of choice in DNN inference accelerators. Despite their potential, designing multi-pod systolic arrays to maximize effective throughput/Watt—i.e., throughput/Watt adjusted when accounting for array utilization—poses a unique set of challenges. In this work, we study three key pillars in multi-pod systolic array designs, namely array granularity, interconnect, and tiling. We identify optimal array granularity across workloads and show that state-of-the-art commercial accelerators use suboptimal array sizes for single-tenancy workloads. We, then evaluate the bandwidth/latency trade-offs in interconnects and show that Butterfly networks offer a scalable topology for accelerators with a large number of pods. Finally, we introduce a novel data tiling scheme with custom partition size to maximize utilization in optimally sized pods. We propose Scale-out Systolic Arrays , a multi-pod inference accelerator for both single- and multi-tenancy based on these three pillars. We show that SOSA exhibits scaling of up to 600 TeraOps/s in effective throughput for state-of-the-art DNN inference workloads, and outperforms state-of-the-art multi-pod accelerators by a factor of 1.5 ×. 1