Emerging MPEG Video Coding for Machines (MPEG VCM) standardization activities address the growing demand for machine-to-machine visual applications, including video surveillance, autonomous driving, etc. This paper proposes an evaluation methodology tailored to MPEG VCM, with an emphasis on semantic segmentation tasks using the Pandaset dataset. This is a challenging target as standardization works must follow several limitations, such as dataset's licensing, fixed tools and software, and compliance with existing common test conditions (CTC) for the standard's development. The proposed evaluation methodology includes a step to align Pandaset and COCO labels. Two task networks, Detectron2 and Mask2Former, are used to evaluate the Rate-Performance behavior for semantic segmentation under various coding configurations. The performance of MPEG VCM is benchmarked against traditional codecs (VVC and HEVC), with a detailed analysis of MPEG VCM's coding tools. The extensive evaluations reveal interesting observations. (1) Although VCM achieved reasonably good segmentation performance, some of its developed tools, such as temporal resampling and region-of-interest coding, were not well suited for segmentation task. (2) The Hybrid NNVVC Inner Codec outperformed the VVC Inner Codec. (3) VCM's performance varies significantly for segmented classes. (4) Despite significant differences in human vision performance, VVC and HEVC exhibit relatively similar performance in machine vision. The main contributions of this work are to (1) enable evaluation of MPEG VCM in a real-world semantic segmentation use case, which is one of VCM's targeted tasks, and (2) to provide a detailed assessment of VCM's performance in semantic segmentation.

Recent work on FPGA hardware security showed a substantial potential risk through power-hammering, which uses high switching activity in order to create excessive dynamic power loads. Virtually all present power-hammering attack scenarios are based on some kind of ring oscillators for which mitigation strategies exist. In this paper, we use a different strategy to create excessive dynamic power consumption: glitch amplification. By carefully designing XOR trees, fast switching wires can be implemented that, while driving high fan-out nets, can draw enough power to crash an FPGA. In addition to the attack (which is crashing an Ultra96 board), we will present a scanner for detecting malicious glitch amplifying FPGA designs.

Since FPGAs are now available in datacenters to accelerate applications, providing FPGA hardware security is a high priority. FPGA security is becoming more serious with the transition to FPGA-as-a-Service where users can upload their own bitstreams. Full control over FPGA hardware through the bitstream enables attacks to weaken an FPGA-based system. These include physically damaging the FPGA equipment and leaking of sensitive information such as the secret keys of crypto algorithms. While there is no known attacks in the commercial settings so far, it is not so much a question of if but more of when? The tutorial will show concrete attacks applicable on datacenter FPGAs. The goal of this tutorial is to prepare the FPGA community to impending security issues in order to pave way for a proactive security. First, we will give a tour through the FPGA hardware security jungle surveying practical attacks and potential threats. We will reinforce this with live demos of denial of service attacks. Less than 10% of the logic resources on an FPGA can draw enough dynamic power to crash a datacenter FPGA card. In the second part of the tutorial, we will show different mitigations that are either vendor supported or proposed by the academic community. In summary, the tutorial will communicate that while FPGA hardware security is complicated to bring about, there are acceptable solutions for known FPGA security problems.

As FPGAs are now offered on the cloud, this exposes many potential security issues. This PhD project investigates current security issues and challenges when deploying FPGAs in the cloud as well as using FPGAs in a multi-tenancy scenario. By addressing practical threats, and most importantly, proposing feasible countermeasures, this paper shows preliminary results on protecting FPGAs for multi-tenant scenarios.

As FPGAs are now offered on the cloud and widely used in critical applications (infrastructure, military, medical), this exposes many potential security issues in which attackers can deploy attacks remotely. This demo will take a closer look at malicious bitstreams and demo our FPGA bitstream virus scanner FPGADefender that can scan for signatures relating to malicious circuits and many forms of bitstream manipulations. Additionally, we show a deny-of-service power-hammering attack, which can serve as a template for hardware Trojans.

With the introduction of Zynq UltraScale+ MPSoCs equipped with powerful 64-bit ARM CPUs along with a 16nm UltraScale+ FPGA fabric in the same die, we can now build full hardware-software programmable systems and configure the FPGA with accelerators, as needed. Traditionally, accelerators had been developed offline using powerful servers running heavy lifting CAD toolchains. In this demo, we show a self-compilation system supporting a user-friendly Jupyter Notebook GUI and multi-tenancy use of the FPGA for educational purposes. From a user perspective, this system compiles accelerators at run-time directly on the ARM CPU without any involvement of the vendor tools. The final bitstreams then execute on the FPGA fabric using PR.

Sharing configuration bitstreams rather than netlists is a very desirable feature to protect IP or to share IP without longer CAD tool processing times. Furthermore, an increasing number of systems could hugely benefit from serving multiple users on the same FPGA, for example, for resource pooling in cloud infrastructures. This article researches the threat that a malicious application can impose on an FPGA-based system in a multi-tenancy scenario from a hardware security point of view. In particular, this article evaluates the risk systematically for FPGA power-hammering through short-circuits and self-oscillating circuits, which potentially may cause harm to a system. This risk includes implementing, tuning, and evaluating all FPGA self-oscillators known from the literature but also developing a large number of new power-hammering designs that have not been considered before. Our experiments demonstrate that malicious circuits can be tuned to the point that just 3% of the logic available on an Ultra96 FPGA board can draw the power budget of the entire FPGA board. This fact suggests a waste power potential for datacenter FPGAs in the range of kilowatts. In addition to carefully analyzing FPGA hardware security threats, we present the FPGA virus scanner FPGAD efender , which can detect (possibly) any self-oscillating FPGA circuit, as well as detecting short-circuits, high fanout nets, and a tapping onto signals outside the scope of a module for protecting data center FPGAs, such as Xilinx UltraScale+ devices at the bitstream level.

With FPGAs now being deployed in the cloud and at the edge, there is a need for scalable design methods that can incorporate the heterogeneity present in the hardware and software components of FPGA systems. Moreover, these FPGA systems need to be maintainable and adaptable to changing workloads while improving accessibility for the application developers. However, current FPGA systems fail to achieve modularity and support for multi-tenancy due to dependencies between system components and the lack of standardised abstraction layers. To solve this, we introduce a modular FPGA operating system – FOS, which adopts a modular FPGA development flow to allow each system component to be changed and be agnostic to the heterogeneity of EDA tool versions, hardware and software layers. Further, to dynamically maximise the utilisation transparently from the users, FOS employs resource-elastic scheduling to arbitrate the FPGA resources in both time and spatial domain for any type of accelerators. Our evaluation on different FPGA boards shows that FOS can provide performance improvements in both single-tenant and multi-tenant environments while substantially reducing the development time and, at the same time, improving flexibility.

This paper combines a chain of academic tools to form an FPGA compilation flow for building partially reconfigurable modules on lightweight embedded platforms. Our flow - EFCAD - supports the entire stack from RTL (Verilog) to (partial) bitstream, and we demonstrate early results from the onchip ARM processor of, and targeting, the latest 16nm generation of a Zynq UltraScale+ MPSoC device. With this, we complement Xilinx's PYNQ initiative to not only facilitate System-on-Chip research and education entirely within an embedded system, but also to allow building new and specialising existing customcomputing accelerators without needing access to a workstation.

With the introduction of Zynq FPGAs that provide an ARM SoC with an attached FPGA fabric, it is possible to build complex software-centric systems that are software and hardware programmable. To harness the full potential of this approach, we developed FOS an FPGA Operating System which is built on open-source FPGA community and Xilinx vendor components. A distinct feature shown in this demo is a heterogeneous resource elastic scheduler that can dynamically and automatically adjust the allocation of tasks to hardware and software resources with respect to the present load scenario. We will also show the FOS ecosystem that allows easily implementing relocatable partially reconfigurable modules directly from RTL or HLS.

FPGA accelerators are being applied in various types of systems ranging from embedded systems to cloud computing for their high performance and energy efficiency. Given the scale of deployment, there is a need for efficient application development, resource management, and scalable systems, which make FPGA virtualization extremely important. Consequently, FPGA virtualization methods and hardware infrastructures have frequently been proposed in both academia and industry for addressing multi-tenancy execution, multi-FPGA acceleration, flexibility, resource management and security. In this survey, we identify and classify the various techniques and approaches into three main categories: 1)Resource level, 2)Node level, and 3)Multi-node level. In addition, we identify current trends and developments and highlight important future directions for FPGA virtualization which require further work.

FPGAs are rising in popularity for acceleration in all kinds of systems. However, even in cloud environments, FPGA devices are typically still used exclusively by one application only. To overcome this, and as an approach to manage FPGA resources with OS functionality, this paper introduces the concept of resource elastic virtualization which allows shrinking and growing of accelerators in the spatial domain with the help of partial reconfiguration. With this, we can serve multiple applications simultaneously on the same FPGA and optimize the resource utilization and consequently the overall system performance. We demonstrate how an implementation of resource elasticity can be realized for OpenCL accelerators along with how it can achieve 2.3x better FPGA utilization and 49% better performance on average while simultaneously lowering waiting time for tasks.

FPGAs are currently being deployed at a large scale across data-centres for various applications because of their performance and power benefits. In particular, cloud service operators are now offering FPGAs as a Service. However, to completely integrate FPGAs in a data-centre environment like standard software systems, support for fault tolerance and task migration is essential. In this paper, we propose a live migration technique for FPGA accelerators to provide support for fault tolerance, system maintenance, and resource management. Our technique allows migration of OpenCL accelerators not only within a single FPGA but also across FPGAs with zero downtime. It achieves this by overlapping the computation with datamovements transparently from the user for OpenCL kernels. Moreover, distributed check-pointing mechanisms can be employed to recover from unknown faults with minimal loss of completed work. Altogether it enables system updates such as changing the static FPGA configuration or upgrading the OS without an interruption of service.

To fully support the partial reconfiguration capabilities of FPGAs, this paper introduces the tool and API BitMan for generating and manipulating configuration bitstreams. Bit-Man supports recent Xilinx FPGAs that can be used by the ISE and Vivado tool suites of the FPGA vendor Xilinx, including latest Virtex-6, 7 Series, UltraScale and UltraScale− series FPGAs. The functionality includes high-level commands such as cutting out regions of a bitstream and placing or relocating modules on an FPGA as well as low-level commands for modifying primitives and for routing clock networks or rerouting signal connections at run-time. All this is possible without the vendor CAD tools for allowing BitMan to be used even with embedded CPUs. The paper describes the capabilities, API and performance evaluation of BitMan.

Reconfigurable architectures have found use in a wide range of application domains, but mostly as static accelerators for computationally intensive functions. Commodity computing adoption has not taken off due primarily to design complexity challenges. Yet reconfigurable architectures offer significant advantages in terms of sharing hardware between distinct isolated tasks, under tight time constraints. Trends towards amalgamation of computing resources in the automotive and aviation domains have so far been limited to non-critical systems, because processor approaches suffer from a lack of predictability and isolation. Hybrid reconfigurable platforms may provide a promising solution to this, by allowing physically isolated access to hardware resources, and support for computationally demanding applications, but with improved programmability and management. We propose virtualized execution and management of software and hardware tasks using a microkernel-based hypervisor running on a commercial hybrid computing platform (the Xilinx Zynq). We demonstrate a framework based on the CODEZERO hypervisor, which has been modified to leverage the capabilities of the FPGA fabric. It supports discrete hardware accelerators, dynamically reconfigurable regions, and regions of virtual fabric, allowing for application isolation and simpler use of hardware resources. A case study demonstrating multiple independent (and isolated) software and hardware tasks is presented.