DSP blocks are one of the efficient solutions to implement multiply-accumulate (MAC) operations on FPGA’s. However, since the DSP blocks have wide multiplier and adder blocks, MAC operations using low bit-length parameters lead to an underutilization. Hence, an efficient approximation technique is introduced. The technique includes manipulation and approximation of the low bit-length parameters based upon a Single DSP - Multiple Multiplication (SDMM) execution. The accuracy of the developed optimization technique was evaluated for different CNN weight bit precisions using the Alexnet and VGG-16 networks and the ImageNet ILSVRC-2012 dataset. The optimization can be implemented without loss of accuracy in almost all cases, while it causes slight accuracy losses in a few cases. Through these optimizations, multiple parameter multiplications are performed in a single DSP block at the cost of a small hardware overhead. As a result of our optimizations, the parameters are represented in a different format on off-chip memory, providing up to 33% compression without any hardware cost. A prototype systolic array architecture was implemented employing our optimizations on a Xilinx Zynq FPGA. It reduced the number of DSP blocks by 66.6%, 75%, and 83.3% for 8, 6, and 4-bit input variables, respectively.

Autonomous driving is disrupting the automotive industry as we know it today. For this, fail-operational behavior is essential in the sense, plan, and act stages of the automation chain in order to handle safety-critical situations on its own, which currently is not reached with state-of-the-art approaches.The European ECSEL research project PRYSTINE realizes Fail-operational Urban Surround perceptION (FUSION) based on robust Radar and LiDAR sensor fusion and control functions in order to enable safe automated driving in urban and rural environments. This paper showcases some of the key exploitable results (e.g., novel Radar sensors, innovative embedded control and E/E architectures, pioneering sensor fusion approaches, AI-controlled vehicle demonstrators) achieved until its final year 3.

Quantization techniques are widely used in CNN inference to reduce the cost of hardware at the expense of small accuracy losses. However, after the quantization, there is still a multiplication cost for the fixed-point quantized CNN weights. Therefore, a novel CNN quantization technique is introduced, which can be implemented without using any multiplier. We evaluated our quantization technique using VGG-16 and Alexnet networks, and the Tiny ImageNet dataset. The quantization technique causes 0.39% and 0.98% accuracy losses for the 8-bit CNN weights compared to floating-point implementations of VGG-16 and Alexnet, respectively. After, a fine-tuning method for our quantization is introduced, which further reduces the accuracy loss. The fine-tuning reduced the accuracy losses on 8-bit quantized VGG-16 and Alexnet to 0.24% and 0.39%, respectively. Two different processing element architectures, which do not include any multiplier hardware, are designed to perform multiply-accumulate (MAC) operations of CNN models quantized by our technique. Two different systolic array prototypes are designed employing the two PE architectures to compare with the traditional fixed-point MAC implementation. The systolic array architectures containing our processing element designs reduced the power consumption of the systolic array up to 14.2% and 21.6%.

Versatile Video Coding (VVC) is a new international video compression standard offering much better compression efficiency than previous video compression standards at the expense of much higher computational complexity. In this paper, an efficient FPGA implementation of VVC intra prediction for angular prediction modes of 4x4, 8x8, 16x16 and 32x32 prediction unit sizes is proposed. In the proposed FPGA implementation, four constant multiplications used in one intra angular prediction equation are implemented using two DSP blocks and two adders in FPGA. The proposed FPGA implementation of VVC intra prediction, in the worst case, can process 34 full HD (1920x1080) frames per second.

Approximate hardware designs have higher performance, smaller area or lower power consumption than exact hardware designs at the expense of lower accuracy. Absolute difference (AD) operation is heavily used in many applications such as motion estimation (ME) for video compression, ME for frame rate conversion, stereo matching for depth estimation. Since most of the applications using AD operation are error tolerant by their nature, approximate hardware designs can be used in these applications. In this paper, novel approximate AD hardware designs are proposed. The proposed approximate AD hardware implementations have higher performance, smaller area and lower power consumption than exact AD hardware implementations at the expense of lower accuracy. They also have less error, smaller area and lower power consumption than the approximate AD hardware implementations which use approximate adders proposed in the literature.

Fractional interpolation is one of the most computationally complex parts of video compression standards. Fractional interpolation in Versatile Video Coding (VVC) standard has much higher computational complexity than fractional interpolation in previous video compression standards. In this paper, a reconfigurable VVC fractional interpolation hardware for motion compensation is designed and implemented using Verilog HDL. The proposed hardware is the first VVC fractional interpolation hardware for motion compensation in the literature. It interpolates necessary fractional pixels for 1/16 pixel accuracy for all prediction unit sizes. The proposed VVC fractional interpolation hardware, in the worst case, can process 66 quad full HD (3840×2160) frames per second. It has up to 77% less power consumption than baseline VVC fractional interpolation hardware.

Multiple constant multiplication (MCM) operation multiplies an input variable with multiple constants. MCM operations are widely used in many applications such as video processing and compression. In this paper, a method is proposed for efficient implementation of MCM operations using DSP blocks in Xilinx FPGAs. The proposed method reduces number of DSP blocks used for implementing a given MCM operation by manipulating the multiple constants used in this MCM operation. In this paper, a high level synthesis tool implementing the proposed method is also proposed. The proposed tool takes the input variable bit length and multiple constants as inputs, and generates a Verilog RTL code which efficiently implements this MCM operation using DSP blocks. The proposed method and tool are used for one of the most complex video compression algorithms, HEVC 2D DCT. They reduced number of DSP blocks used in the FPGA implementation of HEVC 2D DCT algorithm by 35.8%.

Sub-pixel interpolation is one of the most computationally intensive parts of High Efficiency Video Coding (HEVC) video encoder and decoder. Therefore, in this paper, a low energy HEVC sub-pixel (half-pixel and quarter-pixel) interpolation hardware, which uses Hcub multiplierless constant multiplication algorithm, is proposed. The proposed HEVC sub-pixel interpolation hardware, in the worst case, can process 30 quad full HD (3840×2160) video frames per second. It has up to 48% less energy consumption than original HEVC sub-pixel interpolation hardware.

Intra prediction algorithm in the recently developed High Efficiency Video Coding (HEVC) standard has very high computational complexity. Therefore, in this paper, we propose novel techniques for reducing amount of computations performed by HEVC intra prediction algorithm, and therefore reducing energy consumption of HEVC intra prediction hardware. The proposed techniques significantly reduce the amount of computations performed by 4×4 and 8×8 angular prediction modes with a small comparison overhead without any PSNR and bit rate loss. We also designed and implemented a high performance HEVC intra prediction hardware for 4×4 and 8×8 angular prediction modes including the proposed techniques using Verilog HDL, and mapped it to a Xilinx Virtex 6 FPGA. The proposed techniques significantly reduce the energy consumption of the proposed hardware on this FPGA.