Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers.

A resource-efficient hardware architecture for connected component analysis (CCA) of streamed video data is presented, which reduces the required hardware resources, especially for larger image widths. On-chip memory requirements increase with image width and dominate the resources of state-of-the-art CCA single-pass hardware architectures. A reduction in on-chip memory resources is essential to meet the ever increasing image sizes of high-definition (HD) and ultra-HD standards. The proposed architecture is resource efficient due to several innovations. An improved label recycling scheme detects the last pixel of an image object in the video stream only a few clock cycles after its occurrence, allowing the reuse of a label in the following image row. The coordinated application of these techniques leads to significant memory savings of more than two orders in magnitude compared with classical two-pass connected component labeling architectures. Compared with the most memory-efficient state-of-the-art single-pass CCA hardware architecture, 42% or more of on-chip memory resources are saved depending on the features extracted. Based on these savings, it is possible to realize an architecture processing video streams of larger images sizes, or to use a smaller and more energy-efficient field-programmable gate array device, or to increase the functionality of already existing image processing pipelines in reconfigurable computing and embedded systems.

In this paper, an adaptive architecture for dynamic management and allocation of on-chip FPGA Block Random Access Memory (BRAM) resources is presented. This facilitates the dynamic sharing of valuable and scarce on-chip memory among several processing elements (PEs), according to their dynamic run-time memory requirements. Different real-time applications are becoming increasingly dynamic which leads to unexpected and variable memory footprints, and static allocation of the worst-case memory requirements would result in costly overheads and inefficient memory utilization. The proposed scalable BRAM memory management architecture adaptively manages these dynamic memory requirements and balances the buffer memory over several PEs to reduce the total memory required, compared to the worst-case memory footprint for all PEs. The run-time adaptive system allocates BRAM to each PE sufficiently fast enough as required and utilized. In a case study, a significant improvement in BRAM utilization with limited overhead has been achieved due to the adaptive memory management architecture. The proposed system supports different BRAM types and configurations, and automated dynamic allocation and deallocation of BRAM resources, and is therefore well suited for the dynamic memory footprints of FPGA-based reconfigurable architectures.

A memory efficient architecture for single-pass connected components analysis suited for high throughput embedded image processing systems is proposed which achieves a high throughput by partitioning the image into several vertical slices processed in parallel. The low latency of the architecture allows reuse of labels associated with the image objects. This reduces the amount of memory by a factor of more than 5 compared to previous work. This is significant, since memory is a critical resource in embedded image processing on FPGAs.

In classical connected component labeling algorithms the image has to be scanned two times. The amount of memory required for these algorithms is at least as high as for storing a full image. By using single pass connected component labeling algorithms, the memory requirement can be reduced by one order of magnitude to only a single image row. This memory reduction which avoids the requirement of high bandwidth external memory is essential to obtain a hardware efficient implementation on FPGAs. These single pass algorithms mapped one-to-one to hardware resources on FPGAs can process only one pixel per clock cycle in the best case. In order to enhance the performance a scalable parallel memory-efficient single pass algorithm for connected component labeling is proposed. The algorithm reduces the amount of memory required by the hardware architecture by a factor of 100 or more, for typical image sizes, compared to a recently proposed parallel connected component labeling algorithm. The architecture is also able to process an image stream with high throughput without the need of buffering a full image.

In this article, a technique to relax particle image velocimetry (PIV) based on the analysis of motion blurring of particles is proposed in order to reduce the frame rate required for measurement of particle velocities. Another advantage is the relaxation of the requirements of the measurement setup while retaining the principle of measurement analysis. This technique is based on convolutions with 5×5 spatial filters that can be computed in linear time, independently from the contents of the image. Due to multi-scale processing, the technique adapts its parameters autonomously, according to the image and particle properties to provide accurate results. The accuracy of the results is guaranteed by integrating the filter parameters, the measured angle and eventually the measured size of the particles into a linear model. The low complexity and high parallelizability of this method enable the online measurement of particles' velocities.

The fast fermat number transform (FNT) enables fast correlation and fast convolution similar to fast correlation based on fast fourier transform (FFT). In contrast to fixed-point FFT with dynamic scaling, FNT is based on integer operations, which are free of rounding error, and maintains full dynamic range for convolution and correlation. In this paper, a technique to calculate FNT based on two's complement (TFNT) is presented and the correctness of the technique is proven. The TFNT is data flow driven without conditional assignments, which enables high performance pipelined implementations on digital signal processors and field programmable gate arrays. By taking the example of 2D correlation and based on a Radix-4 algorithm, it is shown that TFNT requires less operations than fixed-point FFT as well as less operations than FNT based on the previously presented diminished-1 approach.

A spatial domain perceptual image codec based on subsampling and perceptual quantization (SSPQ) guided by the just-noticeable distortion (JND) profile is proposed. SSPQ integrates perceptual image coding and progressive transmission in one framework. The input image is first subsampled by a factor of two in both dimensions and the subsampled image is compressed without loss. The subsampled image provides a basis for both predicting the input pixels by interpolation and estimating the JND values for each pixel. Residual quantization thresholds are set to the estimated JND values for a perceptually tuned compression. Quantized residuals are progressively encoded by a context-based Golomb coder with run-length coding capacity. Experimental results show over 50% improvement in compression performance on average for the proposed SSPQ codec compared to the lossless JPEG-LS.