Contemporary motion picture content, consisting of scenes with different amounts of visual complexity or camera noise, represents demanding input for video encoders operating in rate control (RC) modes. This paper presents improvements to the 2-pass RC method integrated into VVenC, an open VVC encoder implementation, outlined in previous publications. We specifically introduce three extensions to our RC solution: first, frame type adaptation operating near scene cuts, along with an associated simple detector; second, rate stabilization means to allow for more reliable lookahead based 2-pass RC operation in on-the-fly encoding applications; and third, a low-complexity approach for estimating the instantaneous intensity of camera noise or film grain to avoid large variations in bit consumption when encoding individual frames in the final RC pass. Experimental evaluation confirms that these extensions significantly improve both the objective (BD rate) and subjective (visual) RC performance of VVenC especially on challenging video content.

The latest international Versatile Video Coding (VVC) standard was finalized in July 2020 by the Joint Video Experts Team (JVET) from ITU-T and ISO/IEC. Compared to the High Efficiency Video Coding (HEVC) standard, VVC offers up to 50% of bitrate savings. However, the encoder runtime of the reference software increases tenfold compared to the HEVC equivalent. This paper shows that VVC coding tools which do not consume much of the reference encoder time can have prohibitive computational cost for practical encoders, such as VVenC. VVenC is an optimized open-source real-world implementation with high relevance for practical applications. After analyzing the encoding tools in VTM and VVenC, Symmetric Motion Vector Difference (SMVD) was identified as one of the tools whose efficiency-complexity trade-off may be unfavorable for practical applications. For that reason, two complexity reduction methods for the SMVD search are proposed in this paper. The experimental evaluation on VVenC encoder at an optimized operation point confirms the SMVD encoding runtime reduction by over 72%, with minimal impact on the encoding efficiency.

Two-pass rate control (RC) schemes have proven useful for generating low-bitrate video-on-demand or streaming catalogs. Visually optimized encoding particularly using latest-generation coding standards like Versatile Video Coding (VVC), however, is still a subject of intensive study. This paper describes the two-pass RC method integrated into version 1 of VVenC, an open VVC encoding software. The RC design is based on a novel two-step rate-quantization parameter (R-QP) model to derive the second-pass coding parameters, and it uses the low-complexity XPSNR visual distortion measure to provide numerically as well as visually stable, perceptually R-D optimized encoding results. Random-access evaluation experiments confirm the improved objective as well as subjective performance of our RC solution.

To reduce the residual energy of a video signal, motion compensated prediction with fractional-sample accuracy has been successfully employed in modern video coding technology. In contrast to the fixed quarter-sample motion vector resolution for the luma component in High Efficiency Video Coding standard, the current draft of a new Versatile Video Coding standard introduces a block-level adaptive motion vector resolution (AMVR) scheme. The AMVR allows coding of motion vector difference at different precisions. Nevertheless, the interpolation filters for each fractional sample position are fixed. In this paper, alternative half-luma-sample interpolation filters are proposed. Enabling the interpolation filter selection at a fine granularity allows to better adapt to the local image characteristics. Experimental results show that the proposed method can improve the average BD-rate of VTM 5.0 reference software by -0.40% for Random Access configuration and by -0.76% for Low Delay P configuration.

Versatile Video Coding (VVC) is a new international video coding standard to be finalized in July 2020. It is designed to provide around 50% bit-rate saving at the same subjective visual quality over its predecessor, High Efficiency Video Coding (H.265/HEVC). During the standard development, objective bit-rate savings of around 40% have been reported for the VVC reference software (VTM) compared to the HEVC reference software (HM). The unoptimized VTM encoder is around 9x, and the decoder around 2x, slower than HM. This paper discusses the VVC encoder complexity in terms of soft-ware runtime. The modular design of the standard allows a VVC encoder to trade off bit-rate savings and encoder runtime. Based on a detailed tradeoff analysis, results for different operating points are reported. Additionally, initial work on software and algorithm optimization is presented. With the optimized software algorithms, an operating point with an over 22x faster single-threaded encoder runtime than VTM can be achieved, i.e. around 2.5x faster than HM, while still providing more than 30% bit-rate savings over HM. Finally, our experiments demonstrate the flexibility of VVC and its potential for optimized soft-ware encoder implementations.

The flexible partitioning scheme and increased number of prediction modes in the High Efficiency Video Coding (HEVC) standard are largely responsible for both its high compression efficiency and computational complexity. Each frame in HEVC is partitioned in Coding Tree Units (CTUs) of fixed size, which are then recursively partitioned in Coding Units (CUs). In typical implementations, CUs in a CTU are visited from top to bottom at each level of recursion. In this paper, a different approach is used in which CUs in a CTU can be adaptively visited also in reverse order from bottom to top. Three novel algorithms to reduce complexity of HEVC depth selection, mode decision and inter-prediction are presented, based on this adaptive visiting order. Experimental results show that the proposed encoder achieves on average 38.2% speed-ups compared to fast reference HEVC implementations with pre-built speed-ups enabled, for very limited efficiency losses.

Many video coding standards, including the state-of-the-art High Efficiency Video Coding (HEVC) standard, employ Motion Estimation (ME) at sub-pel precision level to improve the prediction accuracy. However, interpolation and additional Motion Vector (MV) search associated with sub-pel ME result in increased computational complexity. In this paper, an algorithm for fast ME is proposed based on neighbouring sample cost similarity. The costs corresponding to the neighbouring MVs around the optimal MV at different precision levels are analyzed and used to interpolate an error surface. The estimated location of the surface minimum is then used to either skip the remaining ME process, or to limit the number of points to test. Experimental evaluation on high resolution sequences shows that the proposed method can speed up the ME module by 55.5% for 1.5% BD rate losses. Furthermore, the proposed fast quarter-pel ME reduces the quarter-pel ME time by 79.6% while achieving only 17.6% of theoretical encoding losses.

Rate control has been regarded as an indispensable video coding tool for virtually any application involving video transmission. With the advent of many flexible tools introduced in the current state-of-the-art High Efficiency Video Coding (HEVC) standard, previous Rate-Distortion (RD) models used for rate control become insufficiently accurate. To overcome this issue, a new RD model has been recently proposed based on a robust correspondence between the rate and Lagrange multiplier λ. However, existing methods based on this model tend to perform sub-optimally after the scene change. In this paper, a two-pass rate control method is proposed, targeting Ultra High Definition Television (UHDTV) applications. In the first pass, a fast encoder with a reduced set of coding tools is used to obtain the data used for rate allocation and model parameter initialisation utilised during the second pass. Multiple encoding steps required to derive this information are avoided with the proposed variable quantization parameter framework. Experimental evaluation showed that the proposed two-pass rate control method achieves on average 4.4% BD-rate loss, compared with variable bit-rate encoding. That significantly outperforms the state-of-the-art HEVC rate control method with an average BD-rate loss of 8.8%.

The flexible partitioning scheme and increased number of prediction modes in the high efficiency video coding (HEVC) standard are largely responsible for both its high compression efficiency and computational complexity. In typical HEVC encoder implementations, coding units (CUs) in a coding tree unit (CTU) are visited from top to bottom at each level of recursion to select the optimal coding configuration. In this paper, a novel approach is presented in which CUs in a CTU can be adaptively visited also ina reverse, bottom to top visiting order. This reverse CU (RCU) visiting order allows for different algorithmic optimizations for further complexity reduction of many HEVC encoding steps, especially under challenging conditions, such as highly textured or fast moving content. In particular, algorithms to reduce complexity of HEVC depth selection, mode decision, and inter-prediction are presented here based on the coding information obtained from higher depths when using the RCU visiting order. Experimental results show that enabling different stages of the proposed algorithm can achieve average speedups from 16.3% to 36.6% compared to fast reference HEVC implementation with pre-built speed-ups enabled (up to 51.2% in some cases), for 0.3% to 2.2% BD-rate penalty.

The recently ratified High Efficiency Video Coding (HEVC) standard is significantly outperforming previous video coding standards in terms of compression efficiency. However, this comes at the cost of very high computational complexity, which may limit its real-time usage, particularly when targeting Ultra High Definition (UHD) applications. In this paper, an analysis of HEVC coding on UHD content is presented, showing that on average more than 18% of the total encoding time is spent performing uni-directional Motion Estimation (ME) even when using fast algorithms such as Enhanced Predictive Zonal Search (EPZS). In order to speed up the ME process, a novel approach for fast inter prediction is proposed in this paper based on a Multiple Early Termination (MET) decision process. EPZS is only performed in blocks in which it is needed based on local features of the encoded content, or it is skipped otherwise. Experimental results show that the algorithm achieves on average 9.3% speed-ups over conventional HEVC, at the cost of very small BD-rate losses.

Typical H.265/High Efficiency Video Coding (HEVC) encoder implementations test a variety of prediction modes and select the optimal configuration for each block in terms of Rate-Distortion (RD) cost. A fast HEVC mode decision algorithm is proposed here referred to as Context Adaptive Mode Sorting (CAMS). The frequency of selection of modes and their RD costs are collected while encoding the training frames based on local parameters (the context). This information is then used to sort and restrict the prediction modes to test for each context, while the optimal mode found using CAMS on each CU is validated based on the RD cost distributions found during the training. Experimental results show that the method reduces total encoding time of fast HEVC implementations on average by 29.3%, at modest efficiency losses.

The High Efficiency Video Coding (HEVC) standard makes use of flexible partitioning to achieve high compression ratios. Each frame is divided in Coding Tree Units (CTUs) of fixed size, which are further partitioned into Coding Units (CUs) following a recursive quadtree structure. Typically, CUs at each level of recursion are tested to select the optimal coding configuration, hence this process is extremely demanding in terms of computational complexity. In this paper, a method to reduce complexity of HEVC quadtree configuration selection and mode decision is presented, based on a reverse bottom-to-top visiting order of CUs in the quadtree. By visiting smallest CUs first, information can be extracted to make decisions on larger CUs. The encoder adaptively selects whether a CTU is encoded using the reverse CU visiting, allowing for considerably faster encoding under all conditions. Experimental results show that the algorithm achieves on average 21% speedups over previous state-of-the-art fast HEVC algorithms, and up to 36% for some sequences, at very limited efficiency losses.

Most video coding standards, including the state-of-the-art High Efficiency Video Coding (HEVC), make use of sub-pixel Motion Estimation (ME) with Motion Vectors (MV) at fractional precisions to achieve high compression ratios. Unfortunately, sub-pixel ME comes at very high computational costs due to the interpolation step and additional motion searches. In this paper, a fast sub-pixel ME algorithm is proposed. The MV precision is adaptively selected on each block to skip the half or quarter precision steps when not needed. The algorithm bases the decision on local features, such as the behaviour of the residual error samples, and global features, such as the amount of edges in the pictures. Experimental results show that the method reduces total encoding time by up to 17.6% compared to conventional HEVC, at modest efficiency losses.

Ultra High Definition TV (UHDTV) services are being trialled while UHD streaming services have already seen commercial débuts. The amount of data associated with these new services is very high thus extremely efficient video compression tools are required for delivery to the end user. The recently published High Efficiency Video Coding (HEVC) standard promises a new level of compression efficiency, up to 50% better than its predecessor, Advanced Video Coding (AVC). The greater efficiency in HEVC is obtained at much greater computational cost compared to AVC. A practical encoder must optimise the choice of coding tools and devise strategies to reduce the complexity without affecting the compression efficiency. This paper describes the results of a study aimed at optimising HEVC encoding for UHDTV content. The study first reviews the available HEVC coding tools to identify the best configuration before developing three new algorithms to further reduce the computational cost. The proposed optimisations can provide an additional 11.5% encoder speed-up for an average 3.1% bitrate increase on top of the best encoder configuration.