With a lot of enhanced coding tools introduced, High Efficiency Video Coding (HEVC) achieves significant improvement in coding efficiency at the cost of increased computational complexity. To efficiently reduce the encoding time of HEVC, a wavefront-based high parallel (WHP) solution integrating novel data-level and task-level methods is proposed in this paper. On data level, optimal single-instruction-multiple-data algorithms are designed for the enhanced coding tools, i.e., replacing the multiplication in motion compensation by add and shift operations with reduced instruction cycles, removing the transpose in transform via realignment of coefficients, and minimizing the memory access in sum of absolute difference/sum of squared differences calculation by fully reusing the registers. On task level, a novel inter-frame wavefront (IFW) method is developed by effectively decreasing the dependence of wavefront parallel processing (WPP). In addition, a coding tree block level parallelism analysis method is presented to prove the superior of IFW method compared with other HEVC representative parallel methods. Besides, a three-level thread management scheme is proposed to best exploit the parallelism of IFW method and achieve corresponding encoding speedup. Extensive experimental results show that, the overall WHP solution can bring up to $57.65\times $ , $65.55\times $ , and $88.17\times $ speedup for HEVC encoding of Wide Video Graphics Array, 720p and 1080p standard test sequences, while maintaining the same coding performance as with WPP. The proposed solution is also applied in several leading video companies in China, providing HEVC video service for more than 1.3 million users everyday.

To cope with the challenges brought about by bandwidth fluctuation and improve the experience of watching online videos, an adaptive video streaming system that can adjust video quality according to actual network conditions is proposed based on the scalable video coding (SVC) extension of H.264/AVC. First, a simple and effective linear error model is proposed and verified for quality scalability of SVC. The model exploits the linear feature of pixel value errors and can be used to accurately estimate the distortion caused by discarding any combination of enhancement data packets in an SVC bitstream. On that basis, a greedy-like algorithm is designed to assign each data packet a priority value according to its rate-distortion (R-D) impact, thus enabling R-D optimized bitstream extraction under certain bitrate constraints. Finally, the proportional-integral-derivative (PID) method is utilized to control the video quality adjustment and determine a suitable bitrate for transmission. By monitoring and predicting the past, current, and future bandwidth information, the PID-based quality control algorithm is able to reduce quality fluctuation, while still preserving a high quality level. Experimental results show that compared with the baseline software, the proposed system that integrates the above algorithms can achieve much lower video quality fluctuation, with PSNR variance reduced from 1.24 to 0.69, and at the same time deliver higher video quality, with the PSNR average increased by 0.83 dB.

This paper decouples the complex Rate-Distortion (R-D) analysis problem by inspecting the respective influence of source distribution and quantizer design on R-D performance. First, a universal R-D property is theoretically revealed that, for any source distribution can be expressed as the product of a Scaling Factor (SF) and its remaining part, SF does not affect its derivative R-D function. Second, efficient quantizer design principles are deduced for different source distributions, which can be used as convenient R-D performance classifier and indicator when the dead-zone plus uniform threshold scalar quantizer with nearly-uniform reconstruction quantizer (DZ+UTSQ/NURQ) is applied. These two contributions bring new insight and inspiration towards the R-D analysis of various different sources, being solid infrastructure to benefit various video/image applications.

In HEVC coding standard, the encoder employs a quad-tree-based Coding Unit (CU) structure to adapt to various texture characteristics of images. Although it provides better compression performance, the computation load increases drastically. In this paper, a two-stage fast CU size decision method is proposed for HEVC intra coding. In the first stage, we utilize the combination of the weighted variance and the maximum value of the Hadamard costs of the four sub CUs to represent the CU complexity, and classify each CU into three categories: compound, homogeneous, and undetermined. The former two kinds of CUs will be made early split and pruned decisions respectively. In the second stage, an improved SATD-based estimated R-D cost is employed to decide whether the prev undetermined CUs should be early pruned. Experimental results demonstrate that compared with the reference software HM13.0, the proposed fast CU size decision method provides 49% time reduction with slight quality degradation using the HEVC all intra test condition. This proposal has been adopted to provide HEVC video and picture encoding services at the server-side for UC mobile browser which covers over 100 million people in China.

In the above paper [ibid., vol. 16, no. 7, p. 1915, Nov. 2014], the sentence "has provided HEVC service to over 1500 million people in China via the Xunlei Kankan video client" should have appeared as "has provided HEVC services to over 150 million people in China via the Xunlei Kankan video client." Also. J. Sun should have been noted as the corresponding author.

This paper investigates the no-reference SVC bitstream extraction problem and presents an efficient solution to approximate the “optimal” extracted sub-stream. First, we introduce a linear error model to accurately estimate the distortion caused by discarding any combination of packets, even when the original sequence is not available. Then we propose a greedy algorithm to decide each packet's priority according to its R-D impact. The priority value of packets can be stored in the bitstream and used for R-D optimized extraction. Experimental results show that our bitstream extraction method can achieve a significant PSNR gain compared to the extractors of JSVM, without computational complexity increment. Comparison with other methods also demonstrates the advantage of the proposed method.

High Efficiency Video Coding (HEVC) is the new generation video coding standard which achieves significant improvement in coding efficiency. Although HEVC is promising in many applications, the increased computational complexity is a serious problem, which makes parallelization necessary in HEVC encoding. To better understand the bottleneck of parallelization and improve the encoding speed, in this paper, we propose a Coding Tree Blocks (CTB) level parallelism analysis method as well as a novel Inter-Frame Wavefront (IFW) parallel encoding method. First, by establishing the relationship between parallelism and dependence, parallelism is precisely described by CTB-level dependence as a criterion to evaluate different parallel methods of HEVC. On this basis, by effectively decreasing the dependence based on Wavefront Parallel Processing (WPP), IFW method is developed. Finally, with the proposed parallelism analysis method, IFW is theoretically proved to be of higher parallelism compared with other HEVC representative parallel methods. Extensive experimental results show that, the proposed method and implementation can bring up to 17.81x, 14.34x and 24.40x speedup for HEVC encoding of WVGA, 720p and 1080p standard test sequences with the same ignorable coding performance degradation as WPP, thus showing a promising technology for future large-scale HEVC video application.

In this paper, we propose a novel design and optimized implementation of the HEVC decoder. First, a novel decoder prototype with refined decoding workflow and efficient memory management is designed. Then on this basis, a series of single-instruction-multiple-data (SIMD) based algorithms are used to speed up several time-consuming modules in HEVC decoding. Finally, a frame-based parallel framework is applied to exploit the multi-threading technology on multicore processors. With the highly optimized HEVC decoder, decoding speed of 246fps on Intel i7-2400 3.4GHz quad-core processor for 1080p videos and 52fps on ARM Cortex-A9 1.2GHz dual-core processor for 720p videos can be achieved in our experiments.

This paper demonstrates a peer-to-peer (P2P) live and VOD streaming system called Immedia based on Scalable Video Coding (SVC) with low latency. With features of layered coding in SVC, Immedia system could automatically adapt output video rate to varying network. Especially, the live system combining P2P streaming media transmission techniques works better in this respect and greatly reduces server pressure in a large-scale environment.

Rate adaption in Dynamic Adaptive Streaming over HTTP (DASH) is widely applied to adapt the transmission rate to varying network capacity. For rate adaption on variable bitrate (VBR) encoded video, it is still a challenge to properly identify and address the dynamics of bandwidth and segment bitrate. In this paper, the trend of client buffer level variation (TBLV) is analyzed to be a more effective metric for detecting the dynamics of bandwidth and segment bitrate compared to previous metrics. Then, a partial-linear trend prediction model is developed to accurately estimate TBLV. Finally, based on the prediction model, a novel simple rate adaption algorithm is designed to achieve efficient and smooth video quality level adjustment. Experimental results show that while maintaining similar average video quality, the proposed algorithm achieves up to 47.3% improvement in rate adaption smoothness compared to the existing work.

Although the emerging video coding standard High Efficiency Video Coding (HEVC) successfully doubles the compression efficiency of H.264/AVC, its growing computational complexity makes real-time decoding of high-definition HEVC videos a very challenging issue for the existing personal computers and mobile devices. In this paper, a systematical, efficient HEVC decoding solution on general processors is provided, consisting of structure-level, data-level, and task-level approaches. First, a redesigned overall structure of a HEVC decoder with data redundancy reduction mechanism is introduced, which cuts down basic data operation cost and achieves an average decoding speedup of 2.37 × compared to the HM 10.0 decoder. On this basis, novel single-instruction multiple-data (SIMD) algorithms such as low-complexity motion compensation, transpose-free transform, symmetric deblocking filter, and parallel-index sample adaptive offset are developed, which further parallelize the data operations of each decoding task and bring another 2.67 × decoding speedup. Finally, a frame-based task-level parallel framework is employed with a flexible entry scheme to efficiently support the simultaneous processing of multiple decoding tasks for different HEVC parallel strategies. The overall solution achieves decoding fps of 40-75 for 4k HEVC videos on the Intel i7-2600 3.4 GHz quad-core processor (4-thread decoding) and 35-55 for 720p videos on the ARM Cortex-A9 1.2 GHz duo-core processor (2-thread decoding). This proposal is the recommended cross-platform HEVC decoding solution of Intel, AMD, and Cisco, and has provided HEVC service to over 1500 million people in China via the Xunlei Kankan video client.

This paper provides a systematic rate-distortion (R-D) analysis of the dead-zone plus uniform threshold scalar quantization (DZ+UTSQ) with nearly uniform reconstruction quantization (NURQ) for generalized Gaussian distribution (GGD), which consists of two aspects: R-D performance analysis and R-D modeling. In R-D performance analysis, we first derive the preliminary constraint of optimum entropy-constrained DZ+UTSQ/NURQ for GGD, under which the property of the GGD distortion-rate (D-R) function is elucidated. Then for the GGD source of actual transform coefficients, the refined constraint and precise conditions of optimum DZ+UTSQ/NURQ are rigorously deduced in the real coding bit rate range, and efficient DZ+UTSQ/NURQ design criteria are proposed to reasonably simplify the utilization of effective quantizers in practice. In R-D modeling, inspired by R-D performance analysis, the D-R function is first developed, followed by the novel rate-quantization (R-Q) and distortion-quantization (D-Q) models derived using analytical and heuristic methods. The D-R, R-Q, and D-Q models form the source model describing the relationship between the rate, distortion, and quantization steps. One application of the proposed source model is the effective two-pass VBR coding algorithm design on an encoder of H.264/AVC reference software, which achieves constant video quality and desirable rate control accuracy.

In the first part of this paper, we derive a source model describing the relationship between the rate, distortion, and quantization steps of the dead-zone plus uniform threshold scalar quantizers with nearly uniform reconstruction quantizers for generalized Gaussian distribution. This source model consists of rate-quantization, distortion-quantization (D-Q), and distortion-rate (D-R) models. In this part, we first rigorously confirm the accuracy of the proposed source model by comparing the calculated results with the coding data of JM 16.0. Efficient parameter estimation strategies are then developed to better employ this source model in our two-pass rate control method for H.264 variable bit rate coding. Based on our D-Q and D-R models, the proposed method is of high stability, low complexity and is easy to implement. Extensive experiments demonstrate that the proposed method achieves: 1) average peak signal-to-noise ratio variance of only 0.0658 dB, compared to 1.8758 dB of JM 16.0's method, with an average rate control error of 1.95% and 2) significant improvement in smoothing the video quality compared with the latest two-pass rate control method.

This paper provides a systematical rate-distortion (R-D) analysis and modeling of generalized Gaussian distribution (GGD) under the dead-zone plus uniform threshold scalar quantization (DZ+UTSQ) and nearly-uniform reconstruction quantization (NURQ). In R-D analysis, we clearly explain the property of GGD source under efficient DZ+UTSQ/NURQ. On this basis, in R-D modeling, the heuristic D-R model is proposed.

In this paper, we first employ rate-distortion (R-D) modeling to derive the source model for H.264 video coding. This source model consists of the distortion-quantization (D-Q) model and distortion-rate (D-R) model, which describe the relationship between rate, distortion and quantization step for generalized Gaussian distribution (GGD) under the quantization scheme of H.264/AVC. The accuracy of the source model is confirmed by comparing the estimated results with the actual data of JM16.0. And the parameter selection criteria are developed to better employ the source model in our two-pass rate control algorithm for H.264 VBR coding. Based on the D-Q and D-R models, the proposed algorithm is of high stability, low complexity and is easy to implement. Extensive experiments covering different resolutions and target rates demonstrates: 1) about 96.7% reduction in PSNR variance compared to the method of JM16.0 with the average rate control error of 1.95%, and 2) significant improvement in smoothing the video quality compared with the latest two-pass rate control method.

High Efficient Video Coding (HEVC) is the next generation video coding standard in progress. Based on the traditional hybrid coding framework, HEVC implements enhanced tools to improve compression efficiency at the cost of far more computational payload than the capacity of real-time video applications. In this paper, we focus on the software implementation of a real-time HEVC decoder over modern Intel x86 processors. First, we identify the most time-consuming modules of HM 4.0 decoder, represented by motion compensation, adaptive loopfilter, deblocking filter and integer transform. Then the single-execution-multiple-data (SIMD) methods are proposed to optimize the computational performance of these modules. Experimental results show that the optimized decoder is more than 4 times faster than the HM 4.0 decoder, with decoding speed of over 40 frames per second for 1920×1080 resolution videos on Intel i5-2400 processor.

This demonstration illustrates an optimized HEVC decoder, which is compliant with the reference software HM 4.0. The optimized decoder is more than 4 times faster than the reference decoder, and can well meet the real-time decoding demands of the 1080p high-definition (HD) videos. In addition, based on the frame-level multi-thread framework, the optimized decoder can even achieve up to 13.2 times speedup with 4-thread parallel decoding over Intel i5-2400 processor.

This paper studies the rate-distortion (R-D) performance of entropy-constrained dead-zone plus uniform threshold scalar quantization (DZ+UTSQ) and nearly-uniform reconstruction quantization (NURQ) for generalized Gaussian distribution (GGD). We first derive the preliminary constraint of R-D optimized DZ+UTSQ/NURQ for GGD. Then for GGD source of actual DCT coefficients, the refined constraint and precise conditions of optimum DZ+UTSQ/NURQ are rigorously deduced in the real coding bit rate range. Based on above analysis, efficient DZ+UTSQ/NURQ design criteria are proposed to reasonably simplify the implementation of effective quantizer in practice.