Spiking Neural Networks (SNNs) are a promising class of algorithms for hardware acceleration, even outper-forming traditional neural networks in some cases. However, existing SNN accelerator approaches do not perform exhaustive explorations of possible network parameters, including neuron models and spike codings; instead, they often focus on a single network setting and a given fixed hardware architecture for its implementation. Chisel is a hardware construction language that allows the modeling of high-level abstractions from the Register-Transfer Level (RTL) and above. It promises a more transparent and more performance-predictable approach than compiler-based methodologies, like High-Level Synthesis (HLS). In this paper, we propose a novel multi-layer Domain-Specific Language (DSL) for SNN accelerator design based on Chisel, allowing for design space explorations that vary neuron models, spike codings, reset behaviors, and even accelerator topologies. Moreover, we propose an SNN accelerator generation framework using this DSL, which covers training to deployment. We explore and evaluate implementations and provide results regarding execution time, Field-Programmable Gate Array (FPGA) resource usage, power consumption, and accuracy.

Transformer-type Neural Networks (NNs) have shown impressive accuracy numbers in Natural Language Processing (NLP) applications where Recurrent Neural Networks (RNNs) have been in use before, even surpassing them. However, differing considerably from common types of NNs, existing accelerator designs, particularly for Field-Programmable Gate Arrays (FPGAs), cannot be used to implement them. Previous research has shown FPGAs to be platforms superior to CPUs and even GPUs for accelerating NNs when it comes to energy efficiency. Following the development of automated compiler-based design flows for NNs, there is still a lack of such an approach for transformers and FPGA targets. In this realm, this paper presents a novel compiler called TRAC as well as a library of operators and modules for implementing transformer accelerators on FPGAs. Based on optimization and code generation settings in the compiler using an integrated approach combining weight compression techniques with according adaptations of the accelerator modules, a design space of accelerators is defined and explored. For each design, a system-level data path and control unit architecture is generated, which integrates module-level designs using hierarchical High-Level Synthesis (HLS). We evaluate our implementation for the BERT network and provide results regarding the trade-off between execution time, accuracy, and FPGA resource usage.

Thanks to the enormous computing power of GPUs, Machine Learning (ML) based on artificial neural networks has found its way into many important application fields. Sophisticated compiler infrastructures facilitate the task of mapping neural networks onto these accelerators. Recently, new developments have also led to compilation and design automation flows that target FPGA-based accelerators. Although not being as mature as their GPU counterparts, there exists a multitude of published and actively developed approaches with differing support levels for network classes, file formats, and target platforms. Neural network exchange file formats advance jointly with the modeling frameworks. In this paper, we take a quick safari through the jungle of neural network compilation flows for FPGA-based targets by reporting qualitative and quantitative metrics. For comparison, we study the classes of supported neural network architectures of each approach, and the corresponding compatibility of exchange formats, emphasizing ONNX, by examining available conversion tools. Besides, we look at several non-functional properties, including FPGA resource utilization and performance numbers for selected neural networks, but also soft criteria such as licensing, community support, and development activity. Finally, we also assess and discuss some deficiencies currently still affecting some approaches. We hope that our study supports interested readers to orient themselves in the jungle of available flows concerning both functionality and usability, as well as to guide further development and research activities in the endeavor of automated ML acceleration on FPGAs.