This work investigates video encoding parallelization techniques based on the Versatile Video Coding (VVC) standard, using the open and optimized encoder software implementation VVenC. Modern multi-processor systems offer significant opportunities for accelerating video encoding. By employing a proposed combination of parallelization methods, the VVenC encoder achieves an acceleration factor of up to 22 compared to single-threaded mode on a 32-core system, with potential increases to 27× at higher bitrates. Building upon prior work on Inter-frame Parallelization (IFP), the study introduces frame region-based synchronization, enabling further acceleration of up to 10%. Beyond that, the study demonstrates extending frame parallelization beyond Group of Pictures (GOP) boundaries, which improves IFP speed up by 37% and 11% at high-definition (HD) and ultra-high-definition (UHD) resolutions, respectively. Additional combinations with other VVC parallelization tools, such as tiles and VVC Wavefront Parallel Processing (WPP), are also explored. The article provides a comprehensive analysis of parallelization challenges and highlights areas for further improvement.

The Versatile Video Coding (VVC) standard promises high compression efficiency for diverse content types. Based on VVenC, an open and optimized VVC software video encoder, this work presents an inter-frame parallelization (IFP) method designed to exploit the processing power of modern platforms featuring a high number of computing cores. Encoding an ultrahigh definition video on a 32-core machine with the VVenC's faster preset, the proposed method shows more than 20% increase in encoder speed while only a 1% decrease in compression efficiency compared to the default multi-threading mode. In comparison to single-threaded mode, it corresponds to a speedup factor of 18, up from 15x achievable with the previous parallelization scheme. Furthermore, the synergy of the developed inter-frame parallelization technique with other parallelization methods is explored, including tiles and VVC wavefront parallel processing (WPP). The combination of these approaches enables a notable speedup factor of 21, albeit with a trade-off in coding efficiency. With a focus on VVC, this research contributes to the ongoing discourse on video coding optimization, providing valuable insights into possible pitfalls and the potential gains achievable through efficient parallelization techniques on high-core platforms.

Rate control (RC), specifically two-pass, is the main operation mode in VVenC, an open and optimized Versatile Video Coding (VVC) encoder. VVC offers substantial bitrate savings over its predecessor, High Efficiency Video Coding (HEVC), at the price of increased complexity. This complexity increase is apparent in both encoding passes of VVenC. While the complexity redaction in the final pass has been discussed, this paper considers complexity reduction in the first pass, in addition to its already reduced search space. To reduce the overall runtime of a two-pass RC method, spatial and temporal sub-sampling of the first encoding pass is proposed. The experimental results show that the proposed first-pass sub-sampling in two-pass RC can speed up the encoding process of the default two-pass rate control algorithm in VVenC by 18%, with 0.48% loss in coding efficiency, when using the faster preset. Using temporal sub-sampling for the look-ahead, one-pass RC in VVenC can achieve time savings of 11% for bit-rate increases of 0.28%.

Rate control (RC) schemes allow audio and video encoders to produce bitstreams according to specific overall bitrate constraints. However, when no rate capping is enforced, the instantaneous bitrate may vary strongly and may exceed the target rate by an order of magnitude, potentially causing playback stutter especially in video streaming scenarios. This paper introduces a rate capping extension for the two RC modes in VVenC, an open Versatile Video Coding (VVC) compliant encoder implementation. After a revisit of VVenC’s two-pass RC approach, the algorithmic details of the rate capping model are described. The paper concludes with an objective evaluation of the performance of the RC extension in a random-access configuration.

The Versatile Video Coding (VVC) standard has been developed to meet the ever-increasing demand for higher compression of digital video data. Compared to its predecessor, the High-Efficiency Video Coding (HEVC) standard, VVC reduces the bitrate by around 50% for the same perceived quality. This increase in compression efficiency is associated with an increase in computational complexity, mainly on the encoder side. As an open and optimized VVC software encoder implementation, VVenC integrates algorithmic optimizations for each coding tool in VVC. This allows to define a set of five presets from faster to slower as Pareto-optimal tradeoffs between runtime and efficiency. On top, multithreading allows to reduce the runtime and preserves most of the compression efficiency of each preset. This paper presents and analyses the different multi-threading strategies in VVenC. Using a combination of pre-processing, picture-level and in-picture parallelization, VVenC can achieve a parallelization speedup with a factor of 4 for 4 threads while reducing the compression efficiency by only 0.4%. For higher thread numbers, i.e. 16, the speedup depends on the video resolution and used encoder preset, ranging from 6-9 for high definition to 10-12 for ultrahigh definition video with similar loss of compression efficiency. Using additional wavefront and tiles in-picture parallelization, higher speedups can be achieved at the costs of decreased coding efficiency.

Contemporary motion picture content, consisting of scenes with different amounts of visual complexity or camera noise, represents demanding input for video encoders operating in rate control (RC) modes. This paper presents improvements to the 2-pass RC method integrated into VVenC, an open VVC encoder implementation, outlined in previous publications. We specifically introduce three extensions to our RC solution: first, frame type adaptation operating near scene cuts, along with an associated simple detector; second, rate stabilization means to allow for more reliable lookahead based 2-pass RC operation in on-the-fly encoding applications; and third, a low-complexity approach for estimating the instantaneous intensity of camera noise or film grain to avoid large variations in bit consumption when encoding individual frames in the final RC pass. Experimental evaluation confirms that these extensions significantly improve both the objective (BD rate) and subjective (visual) RC performance of VVenC especially on challenging video content.

The Versatile Video Coding (VVC) standard was finalized in July 2020. It has since been studied in the literature with regards to both compression efficiency and complexity. Most of the experiments regarding VVC were performed based on the VTM reference software using its full configuration, usually compared to the test model of its predecessor, High Efficiency Video Coding (HEVC), HM. In this paper, we present a comprehensive and in-depth analysis of the VVC encoding complexity by using the fast and efficient open VVC encoder implementation VVenC. As a practical non-reference encoder, it offers alternative tradeoffs between compression efficiency and complexity. To find those, we approximate the Pareto set of the configuration space in an iterative process, encompassing search space extending encoding tools as well as search space pruning fast algorithms. While the VVenC Pareto set is derived based on HD and UHD video encoding results, it is validated and improved for alternative input material, namely for low-resolution and screen content video. Based on this optimization, five presets are defined constituting approximate Pareto-optimal encoding points for various content types as well as for multi-threaded operation. Without multi-threading, the presets span working points from faster at 4.7% HM runtime at 11% BD-rate gain through medium providing 38% BD-rate gain over HM at a quarter of the HM runtime, up to slower providing all of VTM’s gain at less than half of its runtime.

Two-pass rate control (RC) schemes have proven useful for generating low-bitrate video-on-demand or streaming catalogs. Visually optimized encoding particularly using latest-generation coding standards like Versatile Video Coding (VVC), however, is still a subject of intensive study. This paper describes the two-pass RC method integrated into version 1 of VVenC, an open VVC encoding software. The RC design is based on a novel two-step rate-quantization parameter (R-QP) model to derive the second-pass coding parameters, and it uses the low-complexity XPSNR visual distortion measure to provide numerically as well as visually stable, perceptually R-D optimized encoding results. Random-access evaluation experiments confirm the improved objective as well as subjective performance of our RC solution.

Versatile Video Coding (VVC) is a new international video coding standard to be finalized in July 2020. It is designed to provide around 50% bit-rate saving at the same subjective visual quality over its predecessor, High Efficiency Video Coding (H.265/HEVC). During the standard development, objective bit-rate savings of around 40% have been reported for the VVC reference software (VTM) compared to the HEVC reference software (HM). The unoptimized VTM encoder is around 9x, and the decoder around 2x, slower than HM. This paper discusses the VVC encoder complexity in terms of soft-ware runtime. The modular design of the standard allows a VVC encoder to trade off bit-rate savings and encoder runtime. Based on a detailed tradeoff analysis, results for different operating points are reported. Additionally, initial work on software and algorithm optimization is presented. With the optimized software algorithms, an operating point with an over 22x faster single-threaded encoder runtime than VTM can be achieved, i.e. around 2.5x faster than HM, while still providing more than 30% bit-rate savings over HM. Finally, our experiments demonstrate the flexibility of VVC and its potential for optimized soft-ware encoder implementations.

The high efficiency video coding (HEVC) standard shows enhanced video compression efficiency at the cost of high performance requirements. To address these requirements different approaches, like algorithmic optimization, parallelization and hardware acceleration can be used leading to a complex design space. In order to find an efficient solution, early design verification and performance evaluation is crucial. Hereby the prevailing methodology is the simulation of the complex HW/SW architecture. Targeting heterogeneous designs, different simulation models have different performance evaluation capabilities making a combined HW/SW co-analysis of the entire system a cumbersome task. To facilitate this co-analysis, we propose a non-intrusive instrumentation methodology for simulation models, which automatically adapts to the model under observation. With the help of this instrumentation methodology we perform the analysis and exploration of different design aspects of a SystemC-based heterogeneous multi-core model of an HEVC intra encoder. In the course of this HW/SW co-analysis various aspects of the parallelization and hardware acceleration of the video coding algorithms are presented and further improved. Due to its cycle accurate nature the developed model is well suited to facilitate various performance evaluations and to drive HW/SW co-optimizations of the explored system, as discussed in this paper.