This paper presents the design and VLSI implementation of a pyramid block-matching motion estimation engine, which is consisted of cascaded Integer Motion Estimation (IME) and Fractional Motion Estimation (FME). The IME is further divided into cascaded 3-stage, quarter sub-sample search, half sub-sample search, and integer sample search, while FME is divided into cascaded 2-stage half-sample interpolation and quarter-sample interpolation. Global Motion Estimation (GME) is introduced to compensate drastic objects moving within limited search range of ±160 × ±96. We also employ a lossless compression algorithm based on pixel tiles to reduce DRAM bandwidth by 50%. The design is integrated into a 4K realtime HEVC video encoder and fabricated with TSMC 28nm technology. The total ME hardware costs are 1,094k gates and 75.5KB SRAM, which leads to reduction of 40% ~ 55% and 64% ~ 86% as compared with reference designs. The measured results show that our implementation is able to achieve 4096×2160@30fps real-time encoding when running at 350 MHz while consumes 47mW, and 0.18nJ/pixel of energy efficiency with power reduction of 18%.

This paper presents the design and VLSI implementation of a low-power HEVC main profile encoder, which is able to process up to [email protected] 4:2:0 encoding in real-time with five-stage pipeline architecture. A pyramid ME (Motion Estimation) engine is employed to reduce search complexity. To compensate for the video sequences with fast moving objects, GME (Global Motion Estimation) are introduced to alleviate the effect of limited search range. We also implement an alternative 5x5 search along with 3x3 to boost video quality. For intra mode decision, original pixels, instead of reconstructed ones are used to reduce pipeline stall. The encoder supports DVFS (Dynamic Voltage and Frequency Scaling) and features three operating modes, which helps to reduce power consumption by 25%. Scalable quality that trades encoding quality for power by reducing size of search range and intra prediction candidates, achieves 11.4% power reduction with 3.5% quality degradation. Furthermore, a lossless frame buffer compression is proposed which reduced DDR bandwidth by 49.1% and power consumption by 13.6%. The entire video surveillance SoC is fabricated with TSMC 28nm technology with 1.96 mm2 area. It consumes 2.88M logic gates and 117KB SRAM. The measured power consumption is 103mW at 350MHz for 4K encoding with high-quality mode. The 0.39nJ/pixel of energy efficiency of this work, which achieves 42% ~ 97% power reduction as compared with reference designs, make it ideal for real-time low-power smart video surveillance applications.

Depth videos shall be efficiently compressed and transmitted to the client for view synthesis in Three-Dimensional (3D) video system. Since depth video may contain noise that reduce the coding efficiency, we propose a depth filtering algorithm for 3D depth coding, which exploits the Allowable Depth Distortion (ADD) in view synthesis and is able to improve the coding performance of the depth encoder. Firstly, the depth values has the same rendering position based on the ADD model are clustered. Then, the clustered depth are filtered and set to the optimal depth value for each group by minimizing the view synthesis error. The filtered depth videos are smoother and can be more effectively compressed by the existing 3D High Efficiency Video Coding (HEVC) depth encoder. Experimental results show that the proposed depth filtering method can assist the depth encoder achieve 5.87% bit rate reduction in terms of Bjonteggard Delta Bit Rate (BDBR) and 0.25dB quality gain in terms of Bjonteggard Delta Peak-Signal-to-Noise Ratio (BDPSNR) on average as compared with that of coding the original depth maps.

In 3D video systems, imperfect depth images often induce annoying temporal noise, e.g., flickering, to the synthesized video. However, the quality of synthesized view is usually measured with peak signal-to-noise ratio or mean squared error, which mainly focuses on pixelwise frame-by-frame distortion regardless of the obvious temporal artifacts. In this paper, a novel full reference synthesized video quality metric (SVQM) is proposed to measure the perceptual quality of the synthesized video in 3D video systems. Based on the proposed SVQM, an improved rate-distortion optimization (RDO) algorithm is developed with the target of minimizing the perceptual distortion of synthesized view at given bit rate. Then, the improved RDO algorithm is incorporated into the 3D High Efficiency Video Coding (3D-HEVC) software to improve the 3D depth video coding efficiency. Experimental results show that the proposed SVQM metric has better consistency with human perception on evaluating the synthesized view compared with the state-of-the-art image/video quality assessment algorithms. Meanwhile, this SVQM metric maintains low complexity and easy integration to the current video codec. In addition, the proposed SVQM-based depth coding scheme can achieve approximately 15.27% and 17.63% overall bit rate reduction or 0.42- and 0.46-dB gain in terms of SVQM quality score on average as compared with the latest 3D-HEVC reference model and the state-of-the-art depth coding algorithm, respectively.

The quality assessment for synthesized video with texture/depth compression distortion is important for the design, optimization, and evaluation of the multi-view video plus depth (MVD)-based 3D video system. In this paper, the subjective and objective studies for synthesized view assessment are both conducted. First, a synthesized video quality database with texture/depth compression distortion is presented with subjective scores given by 56 subjects. The 140 videos are synthesized from ten MVD sequences with different texture/depth quantization combinations. Second, a full reference objective video quality assessment (VQA) method is proposed concerning about the annoying temporal flicker distortion and the change of spatio-temporal activity in the synthesized video. The proposed VQA algorithm has a good performance evaluated on the entire synthesized video quality database, and is particularly prominent on the subsets which have significant temporal flicker distortion induced by depth compression and view synthesis process.

Smart phones is bringing about emerging potentials in mobile visual search. Extensive research efforts have been made in compact visual descriptors. However, directly extracting visual descriptors on a mobile device is computationally intensive and time consuming. Towards low bit rate visual search, we propose to deeply compress query images by learning a customized JPEG quantization table in the context of visual search. Distinct from traditional image compression, by incorporating pair-wise image matching precision into distortion measure, we optimize quantization table to seek a better trade-off between image compression rate and visual search performance. An evolutionary algorithm is employed to learn an optimal quantization table. Under MPEG CDVS evaluation framework, extensive evaluation has been done including image retrieval and pair-wise matching over 1 million database images. Experimental results have demonstrated that our optimized quantization table works much better than JPEG default one in terms of retrieval/matching performance vs. a set of different operating points. The proposed low bit rate solution may be easily deployed to smart phones without hardware support, as a useful complement to the ongoing MPEG CDVS standardization efforts.