We are developing an SoC FPGA-based unmanned mobile vehicle for the FPGA design competition. For the vehicle to follow roads successfully, it must be able to detect not only straight lines but also curved lines accurately. Therefore, we implemented a lane detection algorithm that is robust not only against straight lines but also against curves to improve driving performance. We implemented an autonomous driving system employing this algorithm on Digilent Zybo Z7-20. We evaluated the lane detection algorithm based on simulations and showed that this algorithm can reduce false detection of lane features compared to the classical Canny filter.

This paper presents an FPGA implementation of real-time high dynamic range (HDR) synthesis, which expresses a wide dynamic range by combining multiple images with different exposures using image pyramids. We have implemented a pipeline that performs streaming processing on images without using external memory. However, implementation for high-resolution images has been difficult due to large memory usage for line buffers. Therefore, we propose an image compression algorithm based on adaptive differential pulse code modulation (ADPCM). Compression modules based on the algorithm can be easily integrated into the pipeline. When the image resolution is 4K and the pyramid depth is 7, memory usage can be halved from 168.48 % to 84.32 % by introducing the compression modules, resulting in better quality.

We are developing the unmanned mobile vehicle implemented on SoC FPGA for the FPGA design competition. For highly productive development of image-based self-driving mobile vehicles, a remote verification and debugging environment with real-time image transmission is important. This paper presents an image transmission system with which we can monitor onboard camera images of the vehicle and feature detection results over VNC Wi-Fi connection. We implemented the whole system on a Xilinx Zynq-7000 with a maximum operating frequency of 125 MHz. The evaluation of the system showed that the resolution of 640×720 is the most beneficial for VNC in this experiment in terms of the performance ratio of VNC to SSH X11 forwarding. We also shortly describe other components to be used to develop the autonomous driving system in this paper.

A super-resolution technology is used for filling the gap between high-resolution displays and lower-resolution images. One of various algorithms to interpolate the lost information is to use a convolutional neural network (CNN). This paper shows an FPGA implementation and a performance evaluation of our CNN-based super-resolution system, which can process moving images in real time. We apply horizontal and/or vertical flips to input images instead of pre-enlargement. This method prevents information loss and enables the network to make the best use of its input size. In addition, we adopted the residue number system (RNS) to reduce resource utilization. The proposed system can perform super-resolution from 960×540 to 1920×1080 at 60fps with a latency of less than 1ms. In spite of resource restriction of the FPGA, the system generates clear super-resolution images with smooth edges. The evaluation results also revealed the superior quality in terms of the peak signal-to-noise ratio (PSNR), compared to other systems using pre-enlargement.

Super-resolution technologies are used to fill the gap between high-resolution displays and lower-resolution contents. There are various algorithms to interpolate information, one of which is using a convolutional neural network (CNN). This paper shows FPGA implementation and performance evaluation of a CNN-based super-resolution system, which can process moving images in real time. We apply horizontal and/or vertical flips to network input images instead of commonly used pre-enlargement techniques. This method prevents information loss and enables the network to utilize the best of its input image size. Our system can perform super-resolution from 960×540 pixels to 1920×1080 pixels at not less than 48fps with a latency of less than 1 ms. Even though the network scale and the size of filters are limited due to resource restriction of the FPGA, the system generates clear super-resolution images with smooth edges. The evaluation results also reveal that the proposed system achieves superior quality in terms of the structural similarity (SSIM) index, compared to other systems using pre-enlargement.