ShapeFuture will drive innovation in fundamental Electronic Components and Systems (ECS) that are essential for robust, powerful, fail-operational and integrated perception, cognition, AI-enabled decision making, resilient automation and computing, as well as communications, for highly automated vehicles. The overarching vision of ShapeFuture is to bring ECS Innovation to the heart of Europe’s Mobility Transformation, thereby elevating sovereignty by perfecting programmable ECS solutions for intelligent, safe, connected, and highly automated vehicles. In this paper, we detail not only the vision and mission of the ShapeFuture project, but we also showcase the results achieved during the first year.

The rapid deployment of modern video coding standards underscores the need for hardware implementations that enable real-time coding, support interoperability across codecs, and are openly accessible to community. This paper presents the first known high-level synthesis (HLS) implementation of an accurate full-search fractional motion estimation (FME). The proposed FME core is released as open-source and is compatible with High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) standards. It implements 1) an accurate multiplierless constant multiplication (MCM) unit that performs quarter-pixel interpolation over 9 × 9 pixels at a time; 2) a transform-exempted sum of absolute transformed differences (TE-SATD) unit that is optimized for area and speed; and 3) fixed-point Lagrangian optimizations for rate-distortion optimization (RDO). On an Intel Arria 10 FPGA, the FME core consumes 122 kALUTs and operates at up to 210 MHz. Our profiling results show that a single FME core can support practical HEVC and VVC encoding of 2160p video at 30–120 fps, depending on the video content and encoder preset.

High Efficiency Video Coding (HEVC) and Multi-access Edge Computing (MEC) technologies can make real-time streaming media services available to users with reasonable bandwidth, but the computational complexity of HEVC tends to lead to increased energy consumption in these schemes. In this paper, we investigate the energy saving opportunities of utilizing a field-programmable gate array (FPGA) based HEVC encoder in edge media servers and devices. In practice, we analyze the energy impact of migrating our Kvazaar software HEVC intra encoder to Intel Arria 10 PCIe FPGA(s) on two platforms: 1) Nokia Airframe Cloud Server with 2.4 GHz dual 14-core Intel Xeon processors and 2) an embedded Jetson AGX Orin board with 2.2 GHz 12-core ARM processor. According to our experiments, FPGA encoding on these two platforms saved 76% and 86% of the energy taken up by software only encoding on Airframe, respectively. These results indicate the potential of FPGA-based video encoder acceleration in future green MEC architectures.

High Efficiency Video Coding (HEVC) is the key enabling technology for numerous modern media applications. Overcoming its computational complexity and customizing its rich features for real-time HEVC encoder implementations, calls for automated design methodologies. This article introduces the first complete High-Level Synthesis (HLS) implementation for HEVC intra encoder on FPGA. The C source code of our open-source Kvazaar HEVC encoder is used as a design entry point for HLS that is applied throughout the whole encoder design process, from data-intensive coding tools like intra prediction and discrete transforms to more control-oriented tools such as context-adaptive binary arithmetic coding (CABAC). Our prototype is run on Nokia AirFrame Cloud Server equipped with 2.4 GHz dual 14-core Intel Xeon processors and two Intel Arria 10 PCIe FPGA accelerator cards with 40 Gigabit Ethernet. This proof-of-concept system is designed for hardware-accelerated HEVC encoding and it achieves real-time 4K coding speed up to 120 fps. The coding performance can be easily scaled up by adding practically any number of network-connected FPGA cards to the system. These results indicate that our HLS proposal not only boosts development time, but also provides previously unseen design scalability with competitive performance over the existing FPGA and ASIC encoder implementations.

This paper presents the first known high-level synthesis (HLS) implementation for the Sum of Absolute Transformed Differences (SATD) calculation. The proposed hardware architecture is designed for two SATD algorithms: a widespread Fast Walsh-Hadamard Transform (FWHT-SATD) and a recently introduced Transform Exempted scheme (TE- SATD). This 2-stage architecture is made up of two 1-D Walsh- Hadamard Transform (WHT) stages and a transpose buffer (TB) between them. The chosen HLS approach cuts down design time over contemporary design methods and thereby made it feasible to implement a set of dedicated FWHT-SATD and TE- SATD architectures for 4x4, 8x8, and 16x16 pixel blocks. All these six architectures were synthesized for 28 nm and 45 nm standard cell technologies, and their area and energy consumptions were analysed. TE-based implementations provide 6.0-8.3% total cell area savings and 6.9-12.7% better energy-efficiency than traditional FWHT approaches. Our proposal is the first to introduce TE-SATD architectures for up to 16x16 blocks and each of these tailored architectures was shown to provide better trade-off between silicon area and performance than their reference implementations.

This paper describes a demonstration setup to visualize dynamic resource allocation for real-time HEVC encoding services in FPGA-accelerated cloud. The demonstrated application is Kvazaar HEVC intra encoder, whose functionality is partitioned between FPGAs and processors. During the demonstration, several encoding services can be invoked with requests to the resource manager, which is responsible for allocation, deallocation, and load balancing of resources in the network. The manager provides JSON data to the visualizer, which uses D3 JavaScript library to visualize 1) the physical network structure; 2) running services; and 3) performance of the network elements. This interactive demonstration allows users to request new video streams, view the encoded streams, observe the visualization of the network and services, and manually turn on/off resources to test the robustness of the system.

To increase productivity in designing digital hardware components, high-level synthesis (HLS) is seen as the next step in raising the design abstraction level. However, the quality of results (QoRs) of HLS tools has tended to be behind those of manual register-transfer level (RTL) flows. In this paper, we survey the scientific literature published since 2010 about the QoR and productivity differences between the HLS and RTL design flows. Altogether, our survey spans 46 papers and 118 associated applications. Our results show that on average, the QoR of RTL flow is still better than that of the state-of-the-art HLS tools. However, the average development time with HLS tools is only a third of that of the RTL flow, and a designer obtains over four times as high productivity with HLS. Based on our findings, we also present a model case study to sum up the best practices in comparative studies between HLS and RTL. The outcome of our case study is also in line with the survey results, as using an HLS tool is seen to increase the productivity by a factor of six. In addition, to help close the QoR gap, we present a survey of literature focused on improving HLS. Our results let us conclude that HLS is currently a viable option for fast prototyping and for designs with short time to market.

This paper presents a hardware-accelerated Kvazaar HEVC intra encoder for 4K real-time video coding at up to 120 fps. The encoder is implemented on a Nokia AirFrame Cloud Server featuring a 2.4 GHz dual 14-core Intel Xeon processor and two Arria 10 PCI Express FPGA accelerator cards. The presented encoder is a speed-optimized version of our 1st generation 4K40p HEVC intra encoder. The proposed speedup techniques include 1) Increasing the number of FPGA cards to two; 2) Remapping the simplest multiplications from DSP blocks to logic for better FPGA utilization; 3) Making task scheduling more flexible to improve utilization rate of hardware accelerators; and 4) Increasing the pipeline depth and duplicating time-sensitive resources in the hardware accelerator. As a result, up to three hardware accelerator instances can be accommodated in a single Arria 10 so the encoder is able to make use of six accelerators. According to our experiments, the proposed encoder obtains threefold speedup over our 1st generation encoder. Our proposal is also shown to outperform all other encountered FPGA and ASIC implementations.

This paper describes a demonstration setup for real-time 4K HEVC intra coding. The system is built on Kvazaar open-source HEVC encoder partitioned between 22-core Xeon processor and two Arria 10 FPGAs. The demonstrator supports 1) live streaming of up to three 4K30p videos; or 2) offline video streaming up to 4K100p format. Live feeds are shot by three cameras whereas offline video is accessed from a local hard drive. In both cases, encoded bit stream is sent over a wired connection and played back by laptop(s). The demonstrated HEVC coding speed is over three times as fast as that of a pure software solution.

This paper presents the first known high-level synthesis (HLS) implementation of integer discrete cosine transform (DCT) and discrete sine transform (DST) for High Efficiency Video Coding (HEVC). The proposed approach implements these 2-D transforms by two successive 1-D transforms using a well-known row-column and Even-Odd decomposition techniques. Altogether, the proposed architecture is composed of a 4-point DCT/DST unit for the smallest transform blocks (TBs), an 8/16/32-point DCT unit for the other TBs, and a transpose memory for intermediate results. On Arria II FPGA, the low-cost variant of the proposed architecture is able to support encoding of 1080p format at 60 fps and at the cost of 10.0 kALUTs and 216 DSP blocks. The respective figures for the proposed high-speed variant are 2160p at 30 fps with 13.9 kALUTs and 344 DSP blocks. These cost-performance characteristics outperform respective non-HLS approaches on FPGA.

This paper presents efficient inverse discrete cosine transform (IDCT) and inverse discrete sine transform (IDST) implementations for High Efficiency Video Coding (HEVC). The proposal makes use of high-level synthesis (HLS) to implement a complete HEVC 2-D IDCT/IDST architecture directly from the C code of a well-known Even-Odd decomposition algorithm. The final architecture includes a 4-point IDCT/IDST unit for the smallest transform blocks (TB), an 8/16/32-point IDCT unit for the other TBs, and a transpose memory for intermediate results. On Arria II FPGA, it supports real-time (60 fps) HEVC decoding of up to 2160p format with 12.4 kALUTs and 344 DSP blocks. Compared with the other existing HLS approach, the proposed solution is almost 5 times faster and is able to utilize available FPGA resources better.

This paper presents a High-Level Synthesis (HLS) flow for mapping a software HEVC encoder into Altera CycloneV SoC-FPGA. The starting point is a C implementation of an open-source Kvazaar HEVC intra encoder, which is minimally refined for SystemC design space exploration and automatic Catapult-C RTL generation. The final implementation involves Kvazaar encoder executed in Linux on dual-core ARM, and HW accelerated intra prediction on FPGA. Changing the SW/HW partitioning or modifying the implementation takes hours instead of weeks with Catapult-C HLS. In addition, the design is portable to other platforms without major manual re-writing. We obtained 9 fps full-HD intra prediction speed with a single accelerator on Altera Cyclone V SX on Terasic VEEK-MT-C5SoC board including video capture and HEVC video streaming via Ethernet. To the best of our knowledge, this is the first reported HLS assisted implementation of HEVC encoder on SoC-FPGA.