While routing failures are rare in contemporary FPGAs, difficulties in routing a design can translate into timing closure failure. However, unlike many customer-visible metrics, such as performance, routability of a design on a given FPGA is challenging to quantify. Nevertheless, FPGA architects and CAD developers monitor several metrics, such as heap operations, wire usage, number of routing iterations, and outstanding congestion during routing iterations, to indirectly assess and compare the routability of different designs on various FPGAs. In academic research these metrics can be correlated against channel width sweeps and experiments varying a small set of connectivity parameters on a fixed architecture. In modern commercial FPGAs routing topology can vary significantly between architectures and detailed connectivity patterns are highly optimized to the physical characteristics of the target process node, making channel width sweeps not applicable to evaluating and optimizing commercial FPGAs. In this work, we introduce a new metric to quantify routability in FPGAs. To this end, we start by extracting routability metrics across a set of experiments on an Agilex device as we reduce its available routing tracks. Next, we introduce our proposed routing struggle metric, derived from heap operations trajectory, to quantify and compare the routability of designs as we progressively maim the routing fabric of the device. Next, we perform several comparisons between Stratix 10 and Agilex FPGAs and show that the Agilex device is significantly more routable, but this routability difference is not captured using traditional routability metrics. However, routing struggle can provide a comparison in terms of routability between the Agilex and Stratix 10 devices that aligns with the trend of maximum operating frequency in progressively shrinking placement regions in our area-constrained experiments. Finally, we show that routing struggle is not limited to Quartus experiments and can also be applied in the academic VTR flow.

Reducing the precision of deep neural network (DNN) inference accelerators can yield large efficiency gains with little or no accuracy degradation compared to half or single precision floating-point by enabling more multiplication operations per unit area. A wide range of precisions fall on the pareto-optimal curve of hardware efficiency vs. accuracy with no single precision dominating, making the variable precision capabilities of FPGAs very valuable. We propose three types of logic block architectural enhancements and fully evaluate a total of six architectures that improve the area efficiency of multiplications and additions implemented in the soft fabric. Increasing the LUT fracturability and adding two adders to the ALM (4-bit Adder Double Chain architecture) leads to a 1.5× area reduction for arithmetic heavy machine learning (ML) kernels, while increasing their speed. In addition, this architecture also reduces the logic area of general applications by 6%, while increasing the critical path delay by only 1%. However, our highest impact option, which adds a 9-bit shadow multiplier to the logic clusters, reduces the area and critical path delay of ML kernels by 2.4× and 1.2×, respectively. These large gains come at a cost of 15% logic area increase for general applications.

Recent work has shown that using low-precision arithmetic in Deep Neural Network (DNN) inference acceleration can yield large efficiency gains with little or no accuracy degradation compared to half or single precision floating-point by enabling more MAC operations per unit area. The most efficient precision is a complex function of the DNN application, structure and required accuracy, which makes the variable precision capabilities of FPGAs very valuable. We propose three logic block architecture enhancements to increase the density and reduce the delay of multiply-accumulate (MAC) operations implemented in the soft fabric. Adding another level of carry chain to the ALM (extra carry chain architecture) leads to a 1.5x increase in MAC density, while ensuring a small impact on general designs as it adds only 2.6% FPGA tile area and a representative critical path delay increase of 0.8%. On the other hand, our highest impact option, which combines our 4-bit Adder architecture with a 9-bit Shadow Multiplier, increases MAC density by 6.1x, at the cost of larger tile area and representative critical path delay overheads of 16.7% and 9.8%, respectively.

FPGAs are becoming more heteregeneous to better adapt to different markets, motivating rapid exploration of different blocks/tiles for FPGAs. To evaluate a new FPGA architectural idea, one should be able to accurately obtain the area, delay, and energy consumption of the block of interest. However, current FPGA circuit design tools can only model simple, homogeneous FPGA architectures with basic logic blocks and also lack DSP and other heterogeneous block support. Modern FPGAs are instead composed of many different tiles, some of which are designed in a full custom style and some of which mix standard cell and full custom styles. To fill this modelling gap, we introduce COFFE 2, an open-source FPGA design toolset for automatic FPGA circuit design. COFFE 2 uses a mix of full custom and standard cell flows and supports not only complex logic blocks with fracturable lookup tables and hard arithmetic but also arbitrary heterogeneous blocks. To validate COFFE 2 and demonstrate its features, we design and evaluate a multi-mode Stratix III-like DSP block and several logic tiles with fracturable LUTs and hard arithmetic. We also demonstrate how COFFE 2’s interface to VTR allows full evaluation of block-routing interfaces and various fracturable 6-LUT architectures.

Some modern datacenters are augmenting their compute infrastructure by deploying field-programmable gate arrays (FPGAs) to provide users with specialized accelerators that offer superior compute capability, increased energy efficiency, lower latency, and more programming flexibility than CPUs. However, the higher programming flexibility of FPGAs also gives more capabilities to malicious users to remotely sniff data from other applications running on the same FPGA. This has created a challenge for efficient utilization of FPGAs in datacenters: FPGAs in datacenters are currently not shared between users due to potential security risks. In this paper, we propose different techniques to defeat data-sniffing attacks in datacenter FPGAs by encrypting/decrypting the user application's data. We describe techniques that are appropriate to different trust levels and rigorously evaluate the costs of these data confidentiality techniques in current virtualized FPGAs. In addition, for each trust level, we propose an architectural change to the FPGA to mitigate the costs of providing data confidentiality. We also investigate the role of interconnect in these architectural changes and demonstrate that more efficient security features can be implemented together with the interconnect if the FPGAs use a hard network on chip.

Use of reduced precisions in Deep Learning (DL) inference tasks has recently been shown to significantly improve accelerator performance and greatly reduce both model memory footprint and the required external memory bandwidth. With appropriate network retuning, reduced precision networks can achieve accuracy close or equal to that of full-precision floating-point models. Given the wide spectrum of precisions used in DL inference, FPGAs' ability to create custom bit-width datapaths gives them an advantage over other acceleration platforms in this domain. However, the embedded DSP blocks in the latest Intel and Xilinx FPGAs do not natively support precisions below 18-bit and thus can not efficiently pack low-precision multiplications, leaving the DSP blocks under-utilized. In this work, we present an enhanced DSP block that can efficiently pack 2× as many 9-bit and 4× as many 4-bit multiplications compared to the baseline Arria-10-like DSP block at the cost of 12% block area overhead which leads to only 0.6% total FPGA core area increase. We quantify the performance gains of using this enhanced DSP block in two state-of-the-art convolutional neural network accelerators on three different models: AlexNet, VGG-16, and ResNet-50. On average, the new DSP block enhanced the computational performance of the 8-bit and 4-bit accelerators by 1.32× and 1.6× and at the same time reduced the utilized chip area by 15% and 30% respectively.

FPGAs are being deployed in modern datacenters to provide users with specialized accelerators that offer superior compute capability, increased energy efficiency, lower latency, and more programming flexibility than CPUs. However, FPGAs are not utilized as efficiently in datacenters: unlike CPUs, FPGAs in datacenters are currently not shared between users due to potential security risks. The higher flexibility that comes with FPGAs also gives more capabilities to malicious users. Several recent studies have demonstrated examples of FPGA user applications capable of remotely sniffing data from other applications running on the same FPGA. In this work, we look at various ways to ameliorate these threats by encrypting/decrypting the user application's data under different trust levels for current virtualized FPGAs. We also discuss the role of interconnect and discuss the potential of more efficient security features that can be implemented together with the interconnect if the FPGAs use a hard network on chip.

Recently, deep learning (DL) has become best-in-class for numerous applications but at a high computational cost that necessitates high-performance energy-efficient acceleration. The reconfigurability of FPGAs is appealing due to the rapid change in DL models but also causes lower performance and area-efficiency compared to ASICs. In this article, we implement three state-of-the-art computing architectures (CAs) for convolutional neural network (CNN) inference on FPGAs and ASICs. By comparing the FPGA and ASIC implementations, we highlight the area and performance costs of programmability to pinpoint the inefficiencies in current FPGA architectures. We perform our experiments using three variations of these CAs for AlexNet, VGG-16 and ResNet-50 to allow extensive comparisons. We find that the performance gap varies significantly from 2.8× to 6.3×, while the area gap is consistent across CAs with an 8.7 average FPGA-to-ASIC area ratio. Among different blocks of the CAs, the convolution engine, constituting up to 60% of the total area, has a high area ratio ranging from 13 to 31. Motivated by our FPGA vs. ASIC comparisons, we suggest FPGA architectural changes such as increasing DSP block count, enhancing low-precision support in DSP blocks and rethinking the on-chip memories to reduce the programmability gap for DL applications.

While plentiful on-chip memory is necessary for many designs to fully utilize an FPGA’s computational capacity, SRAM scaling is becoming more difficult because of increasing device variation. An alternative is to build FPGA block RAM (BRAM) from magnetic tunnel junctions (MTJ), as this emerging embedded memory has a small cell size, low energy usage, and good scalability. We conduct a detailed comparison study of SRAM and MTJ BRAMs that includes cell designs that are robust with device variation, transistor-level design and optimization of all the required BRAM-specific circuits, and variation-aware simulation at the 22nm node. At a 256Kb block size, MTJ-BRAM is 3.06× denser and 55% more energy efficient and its F max is 274MHz, which is adequate for most FPGA system clock domains. We also detail further enhancements that allow these 256 Kb MTJ BRAMs to operate at a higher speed of 353MHz for the streaming FIFOs, which are very common in FPGA designs and describe how the non-volatility of MTJ BRAM enables novel on-chip configuration and power-down modes. For a RAM architecture similar to the latest commercial FPGAs, MTJ-BRAMs could expand FPGA memory capacity by 2.95× with no die size increase.

FPGAs are being incorporated into contemporary datacenters in order to improve computational capacity, power consumption, and processing latency. Efficiently integrating FP-GAs in datacenters is, however, quite challenging. Ideally, smaller tasks could share a device and the cloud management layer would be able to partially reconfigure the device to allocate its free resources to incoming tasks. Moreover, to facilitate FPGA hardware upgrades without undue porting effort for previously developed accelerator tasks, the complexities associated with board-specific system-level integration should be abstracted away from designers. By meeting these requirements, FPGAs in the cloud would become multi-user virtualized resources with increased availability and elasticity. The virtualization of FPGAs, however, comes with two major costs in current FPGAs: lower application operating frequency, and extravagant use of routing resources. In this paper, we quantify the costs of FPGA virtualization and demonstrate that for an FPGA that supports four independent tasks, virtualization reduces the task average frequency by 18% to 46% and increases wire usage to 2.6×. We also investigate the cause of these costs and show that the use of hard NoCs in future datacenter-optimized FPGAs would facilitate FPGA virtualization without sacrificing operating frequency or routing resources.

Contemporary FPGAs are composed of a mix of full custom and standard cell-based circuitry, organized into many heterogeneous blocks and the programmable routing. To explore new FPGA architectures, in particular those incorporating new hard blocks, we must estimate the area, power and delay of any new block of interest, and would like to do so efficiently so that many ideas can be evaluated. Unfortunately, existing open source academic FPGA circuit modelling tools cannot target such a wide range of blocks and cannot mix custom and standard cell circuitry. In this work, we present an enhanced (hybrid) COFFE flow that automatically optimizes and models a wide variety of blocks using an intelligent mix of full custom transistor sizing and standard cell flows. Hybrid COFFE can generate models for both advanced logic blocks enriched with hard arithmetic and fracturable LUTs using its full custom flow and arbitrary hard blocks (such as DSP blocks) described in standard Hardware Description Languages (HDLs) that are fabricated using a mix of standard cell and full custom flows. To validate this hybrid flow we model a Stratix-III like DSP block in 65 nm CMOS and find good agreement with published commercial data. The resulting block and routing models are output in the Verilog-To-Routing (VTR) architecture format to facilitate architectural exploration of advanced FPGA blocks.

Many important applications demand large amounts of on-chip memory both to fully utilize an FPGA's computational capacity and to minimize energy-consuming off-chip memory accesses, leading some recent commercial FPGAs to add higher-capacity on-chip block RAMs (BRAMs). While memory is becoming more important to FPGA designs, SRAM scaling is becoming more difficult because of increasing device variation. An alternative is to build FPGA BRAM from magnetic tunnel junction (MTJ) cells as this emerging embedded memory features a small cell size, low energy usage, and good scalability. In this work, we conduct a detailed comparison study of SRAM and MTJ BRAMs that includes cell designs that are robust with device variation, transistor-level design and optimization of all the required BRAM-specific circuits, and variation-aware simulation at the 22nm node. We find that as the capacity of a BRAM increases, the MTJ benefits of high-density and low-energy increase and its drawback of lower speed is mitigated. At a 256 Kb block size, MTJ-BRAM is 3.06× denser and 55% more energy efficient and its Fmaxis 274 MHz, which is adequate for most FPGA system clock domains. We detail how the non-volatility of an MTJ-BRAM saves energy, especially for narrow write operations which are common for the width-configurable BRAMs of FPGAs. For a RAM architecture similar to the latest commercial FPGAs, MTJ-based block RAMs reduce the FPGA fabric area by 28%, or alternatively could expand FPGA memory capacity by 2.95× with no die size increase.

Nanoelectromechanical (NEM) relays are a promising emerging technology that has gained widespread research attention due to its zero leakage current, sharp ON-OFF transitions, and complementary metal-oxide-semiconductor compatibility. As a result, NEM relays have been significantly investigated as highly energy-efficient design solutions. A major shortcoming of NEMs preventing their widespread use is their limited switching endurance. Hence, in order to utilize the low-power advantages of NEM relays, further device, circuit, and architectural techniques are required. In this paper, we introduce the concept of shadow NEM relays, which is a circuit-level technique to leverage the energy efficiency of the NEM relays despite their low switching endurance. This technique creates two virtual ground nodes in a block to allow: 1) a low power mode with functional NEM relays and 2) a normal mode with failed NEM relays. To demonstrate the applicability of this concept, we have applied it to a six-transistor SRAM cell as an illustrative example. We also investigate the applicability of this SRAM cell in field-programmable gate arrays and on-chip caches. Experimental results reveal that shadow NEM relays can reduce the power consumption of SRAM cells by up to 80% while addressing the limited switching endurance of NEM relays.

With the continual scaling of feature size, system failure due to soft errors is getting more frequent in CMOS technology. Soft errors have particularly severe effects in static random-access memory (SRAM)-based reconfigurable devices (SRDs) since an error in SRD configuration bits can permanently change the functionality of the system. Since interconnect resources are the dominant contributor to the overall configuration memory upsets in SRD-based designs, the system failure rate can be significantly reduced by mitigating soft errors in routing fabric. This paper first presents a comprehensive analysis of SRD switch box susceptibility to short and open faults. Based on this analysis, we present a dependable routing fabric by efficiently employing asymmetric SRAM cells in configuration memory of SRDs. The proposed scheme is highly scalable and capable of achieving any desired level of dependability. In the proposed scheme, we also present a fault masking mechanism to mitigate the effect of soft errors in the routing circuitry. A routing algorithm is also proposed to take the advantage of the proposed routing fabric. Experimental results over the Microelectronics Center of North Carolina benchmarks show that the proposed scheme can mitigate both single and multiple event upsets in the routing fabric and can reduce system failure rate orders of magnitude as compared with the conventional protection techniques.