Hardware development critically depends on cycle-accurate RTL simulation.However, as chip complexity increases, conventional single-threaded simulation becomes impractical due to stagnant single-core performance.Parendi is an RTL simulator that addresses this challenge by exploiting the abundant fine-grained parallelism inherent in RTL simulation and efficiently mapping it onto the massively parallel Graphcore IPU (Intelligence Processing Unit) architecture.Parendi scales up to 5888 cores on 4 Graphcore IPU sockets.It allows us to run large RTL designs up to 4× faster than the most powerful state-of-the-art x64 multicore systems.To achieve this performance, we developed new partitioning and compilation techniques and carefully quantified the synchronization, communication, and computation costs of parallel RTL simulation: The paper comprehensively analyzes these factors and details the strategies that Parendi uses to optimize them.

This paper presents the implementation of Manticore: a manycore accelerator for parallel RTL simulation. Manticore packs up to 225 custom soft processors running at 475 MHz on a large FPGA. Implementing manycore accelerators on FPGAs is challenging as designers must reconcile the conflicting goals of maximizing the number of cores on the chip and clocking them at the highest possible frequency. Designers face two classes of constraints: (1) architectural constraints imposed by a large FPGA's multi-die structure, and (2) physical constraints imposed by the FPGA shell's size and placement. Physical design therefore plays a critical role in the implementation of manycore accelerators. We present physical design challenges faced during Manticore's implementation on the AMD Alveo U200 card---a large FPGA with a poorly-placed, wide shell that challenges physical implementation.

The demise of Moore's Law and Dennard Scaling has revived interest in specialized computer architectures and accelerators. Verification and testing of this hardware depend heavily upon cycle-accurate simulation of register-transfer-level (RTL) designs. The fastest software RTL simulators can simulate designs at 1--1000 kHz, i.e., more than three orders of magnitude slower than hardware. Improved simulators can increase designers' productivity by speeding design iterations and permitting more exhaustive exploration.

FPGAs play an increasing role in the reconfigurable accelerator landscape. A key challenge in designing FPGA-based systems is partitioning computation between processor cores and FPGAs. An appropriate division of labor is difficult to predict in advance and requires experiments and measurements. When an investigation requires rewriting part of the system in a new language or with a new programming model, its high cost can delay design-space exploration. A single-language system with an appropriate programming model and compiler that targets both platforms transforms this tedious exploration to a simple recompile with new compiler directives.

As the usage of FPGAs spreads, engineers will inevitably employ them in ways unforeseen-or unwanted-by their manufacturers. Xilinx's toolchains offer multiple points for customizing the FPGA compilation flow, but all flows must end with Vivado as it is the only tool capable of generating the bitstream to program an FPGA. Xilinx does not document its bitstream format, so users who wish to bypass Vivado and modify a bitstream directly must reverse-engineer it to discover the location and format of cells. Prior work has reverse-engineered parts of the bitstream format for security or debugging/instrumentation activities, but no paper has explained how to do this reverse engineering systematically! Code from prior efforts (when available) is hard-coded to reverse engineer a specific device and is difficult or impossible to use for another one. These efforts-focused on applications instead of reverse-engineering-compel engineers who need to modify a bitstream to rediscover unwritten practice. Our work bridges this gap by explaining: (1) the various parameters needed to navigate a bitstream correctly, (2) the experiments to obtain them, and (3) the many pitfalls and erroneous assumptions to avoid while undertaking this endeavor. We demonstrate our technique by using it to extract the bitstream format of initial LUT equations, LUTRAM contents, BRAM contents, and register values in Xilinx UltraScale and UltraScale+ FPGAs. Our methods are implemented in an open-source tool, Bitfiltrator [1], that can extract device layouts and architecture-specific bitstream formats for these cells automatically and without physical access to an FPGA.

Hardware-software codesign for FPGAs requires flexible and changeable boundaries between hardware and software. Design space exploration is facilitated by expressing programs in a language that can be compiled for both CPU and FPGA execution. Such an approach requires efficient and general communication mechanisms between hardware and software. We present a practical solution to this problem for heterogeneous programs expressed in CAL, an actor based language running on a PCIe-based FPGA system where communication between a processor and FPGA is relatively expensive. We show how a network of continuously executing software and hardware actors with fine-grained communication can be expressed as a coprocessor model that executes the network in discrete steps with efficient coarse-grained transfers across the PCIe bus.

The use of parallelism has increased drastically in recent years. Parallel platforms come in many forms: multi-core processors, embedded hybrid solutions such as multi-processor system-on-chip with reconfigurable logic, and cloud datacenters with multi-core and reconfigurable logic. These heterogeneous platforms can offer massive parallelism, but it can be difficult to exploit, particularly when combining solutions constructed with multiple architectures. To program a heterogeneous platform, a developer must master different programming languages, tools, and APIs to program each aspect of platform separately and then must find a means to connect them with communication interfaces. The motivation of this work is to provide a single programming model and framework for hardware-software stream programs on heterogeneous platforms. Our framework, StreamBlocks, starts with a dataflow programming model for both embedded and datacenter platforms. Dataflow programming is an alternative model of computation that captures both data and task parallelism. We describe a compiler infrastructure for CAL dataflow programs for hardware code generation. CAL is a dataflow programming language that can express multiple dataflow models of computation. StreamBlocks is based on the Tycho compiler infrastructure, which transforms each actor in a dataflow program to an abstract machine model, called Actor Machine. Actor Machines provides a unified model for executing actors in both hardware and software and permit our compiler extension and backend to generate efficient FPGA code. Unlike other systems, the programming model and compiler directly support hardware-software systems in which an FPGA functions as a coprocessor to a CPU. This permits easy integration with existing workflows.