The performance of object detection systems in automotive solutions must be as high as possible, with minimal response time and, due to the often battery-powered operation, low energy consumption. When designing such solutions, we therefore face challenges typical for embedded vision systems: the problem of fitting algorithms of high memory and computational complexity into small low-power devices. In this paper we propose PowerYOLO - a mixed precision solution, which targets three essential elements of such application. First, we propose a system based on a Dynamic Vision Sensor (DVS), a novel sensor, that offers low power requirements and operates well in conditions with variable illumination. It is these features that may make event cameras a preferential choice over frame cameras in some applications. Second, to ensure high accuracy and low memory and computational complexity, we propose to use 4-bit width Powers-of- Two (PoT) quantisation for convolution weights of the YOLO detector, with all other parameters quantised linearly. Finally, we embrace from PoT scheme and replace multiplication with bit-shifting to increase the efficiency of hardware acceleration of such solution, with a special convolution-batch normalisation fusion scheme. The use of specific sensor with PoT quantisation and special batch normalisation fusion leads to a unique system with almost 8x reduction in memory complexity and vast computational simplifications, with relation to a standard approach. This efficient system achieves high accuracy of mAP 0.301 on the GENt DVS dataset, marking the new state-of-the-art for such compressed model.

Pedestrian detection algorithms have a wide range of applications: from video surveillance, to driver assistance systems and autonomous vehicles. The performance of these systems must be as high as possible, with minimal response time and, due to the often battery-powered operation (like in electric vehicles), low energy consumption. When designing such solutions, we therefore face challenges typical for embedded vision systems: the problem of fitting algorithms of high memory and computational complexity into small low-power devices. In this paper, we propose a system based on a Dynamic Vision Sensor (DVS), which has low power requirements and operates well in conditions with variable illumination. It is these features that may make event cameras a preferential choice over frame cameras in some applications. To ensure high accuracy, for pedestrian detection we use the YOLO (You Only Look Once) deep network on event data representation. Due to the high complexity of the applied algorithm, we propose a low precision architecture: the weights of the convolution layers are quantized logarithmically to 4 bits powers-of-two values (PoT quantization). Such compression reduces not only the memory complexity by almost 8x, but also the computational complexity by replacing most multiplication operations with bit-shifting, due to use of powers-of-two weights. At the same time, the proposed system achieves the accuracy on par with the floating point baseline, of mAP0.5 0.708.

This demo presents a hardware architecture for an object tracker based on a quantised Siamese neural network. The system is designed to work with a 4K/UltraHD input video stream and to meet the real time and energy efficiency constraints. The network is designed using Brevitas and implemented with FINN tool from Xilinx. The Siamese tracker runs on the ZCU 104 board, with the Zynq UltraScale+ MPSoC device from Xilinx.