Quantization-aware neural architecture search is an efficient way to automatically search for the best quantized model that can meet the limited resource constraints on edge devices. Existing methods utilize the straight-through estimator for training the quantized supernet, but lead to oscillation problem of learned weights. To address this issue, we introduce pseudo quantization noise (PQN) in quantization-aware NAS. Accurate range of PQN is vital to ensuring the high accuracy of the quantized networks. However, producing accurate noise for activation quantization during supernet training is challenging, as it requires precise estimation of the quantization parameters for each subnet. Different distributions of different subnets will result in different noise range. To this end, we propose PQNAS, a mixed-precision quantization-aware NAS framework with Pseudo Quantizer. We propose adaptive quantization parameters (AQP) which are trained with the distribution of activation in the pseudo quantizer. With AQP, we can obtain accurate noise range for different subnets during training. Experimentally, comparing with the existing method, the proposed PQNAS achieves 0.99%∼6.48% Top-1 accuracy improvement on ImageNet 1K dataset and 1.1% mAP improvement on COCO dataset.

Computer vision applications are rapidly gaining popularity in embedded systems, which typically involve a difficult trade-off between vision performance and energy consumption under a constraint of real-time processing throughput. Recently, hardware (FPGA and ASIC-based) implementations have emerged that significantly improve the energy efficiency of vision computation. These implementations, however, often involve intensive memory traffic that retains a significant portion of energy consumption at the system level. To address this issue, we present a lossy embedded compression framework to exploit the trade-off between vision performance and memory traffic for input images. Differential pulse-code modulation-based gradient-oriented quantization is developed as the lossy compression algorithm. We also present its hardware design that supports up to 12-scale 1080p@60fps real-time processing. For histogram of oriented gradient-based deformable part models on VOC2007, the proposed framework achieved a 49.6%-60.5% memory traffic reduction at a detection rate degradation of 0.05%-0.34%. For AlexNet on ImageNet, memory traffic reduction achieved up to 60.8% with less than 0.61% classification rate degradation.

Convolutional neural networks (CNNs) are rapidly gaining popularity in artificial intelligence applications and employed in mobile devices. However, this is challenging because of the high computational complexity of CNNs and the limited hardware resource in mobile devices. To address this issue, compressing the CNN model is an efficient solution. This work presents a new framework of model compression, with the sparseness ratio allocation (SRA) and the neuron re-pruning (NRP). To achieve a higher overall spareness ratio, SRA is exploited to determine pruned weight percentage for each layer. NRP is performed after the usual weight pruning to further reduce the relative redundant neurons in the meanwhile of guaranteeing the accuracy. From experimental results, with a slight accuracy drop of 0.1%, the proposed framework achieves 149.3× compression on lenet-5. The storage size can be reduced by about 50% relative to previous works. 8-45.2% computational energy and 11.5-48.2% memory traffic energy are saved.

This paper presents a measurement-domain intra prediction coding framework that is compatible with compressive sensing (CS) based image sensors. In this framework, we propose a low-complexity intra prediction algorithm that can be directly applied to the measurements captured by the image sensor. Moreover, we propose a structural random 0/1 measurement matrix, embedding the block boundary information that can be extracted from the measurements for intra prediction. Experiment results show that our proposed framework can compress the measurements and increase coding efficiency, with 30% BD-rate reduction compared to the direct output of CS based sensors. This can significantly save both the energy consumption and the bandwidth in communication of wireless camera systems to be massively deployed in the era of IoT.

FRC (frame re-compression) is a kind of widely used technique in reducing the SDRAM (synchronous dynamic random access memory) bandwidth of HEVC video system. However, in previous research works, FRC imposes requirements on accessing pattern and hence its usage are only limited in HEVC video codecs. While in a typical HEVC VLSI video system, there exists many other video IPs with high bandwidth requirements. Therefore, in this article, we propose a new FRC architecture to overcome the limitation and make it applicable to all the video IPs in a HEVC VLSI video system, which raises the overall bandwidth reduction rate of the whole video system. Our proposal has two points: firstly we propose a system internal bus based FRC architecture, which is independent, transparent, and easily connected to all other video IPs. Secondly, we propose a FA (freely access) scheme to remove the requirements on access pattern in previous work. By using this proposal, the bandwidth reduction rate in our VLSI video system model is raised from 92.4% to 69.6%.

Motion estimation and motion compensation in HEVC and similar video codecs involve huge memory traffic in storing and loading reference frames. The resulting memory power composes a significant portion of system energy consumption. This paper presents a memory power reduction framework that losslessly compresses and decompresses reference frames on-the-fly. We first present the architecture that supports the random access of frame data compressed in variable ratios. The latest recompression algorithms and the corresponding VLSI implementation are also introduced. The framework has been implemented in several UHDTV ASIC and FPGA codecs. Experiments under HEVC common test conditions show an average memory traffic reduction of near 60%.

Video encoders and decoders for HEVC-like compression standards require huge external memory bandwidth, which occupies a significant portion of the codec power consumption. To reduce the memory bandwidth, this paper presents a new lossless reference frame recompression algorithm along with a high-throughput hardware architecture. Firstly, hybrid spatial -domain prediction is proposed to combine the merits of DPCM scanning and averaging. The prediction is then enhanced with multiple modes to accommodate various image characteristics. Finally, efficient residual regrouping based on semi-fixed-length (SFL) coding is used to improve the compression performance. Compared to no compression, the proposed scheme can reduce data traffic by an average of 57.6% with no image quality degradation. The average compression ratio is 2.49, an improvement of at least 12.2-13.2%, relative to the state-of-the-art algorithms. By applying a reordered two-step architecture and the two optimizations, residual reuse and simplified coding mode decision, the hardware cost is similar to that of previous reference frame recompression architectures. The computational complexity increase caused by multi-mode prediction affects the HW cost slightly. This work can be implemented with 45.1 k gates for the compressor and 34.5 k gates for the decompressor at 300 MHz, enough to support a 3840 ×2160@60fps video encoder and decoder.