This work investigates video encoding parallelization techniques based on the Versatile Video Coding (VVC) standard, using the open and optimized encoder software implementation VVenC. Modern multi-processor systems offer significant opportunities for accelerating video encoding. By employing a proposed combination of parallelization methods, the VVenC encoder achieves an acceleration factor of up to 22 compared to single-threaded mode on a 32-core system, with potential increases to 27× at higher bitrates. Building upon prior work on Inter-frame Parallelization (IFP), the study introduces frame region-based synchronization, enabling further acceleration of up to 10%. Beyond that, the study demonstrates extending frame parallelization beyond Group of Pictures (GOP) boundaries, which improves IFP speed up by 37% and 11% at high-definition (HD) and ultra-high-definition (UHD) resolutions, respectively. Additional combinations with other VVC parallelization tools, such as tiles and VVC Wavefront Parallel Processing (WPP), are also explored. The article provides a comprehensive analysis of parallelization challenges and highlights areas for further improvement.

VVenC, an open Versatile Video Coding (VVC) encoder, has recently been equipped with rate capping functionality in its two-pass rate control modes, providing constrained variable bitrate coding governed by target rate and maximum rate parameters. This paper reports on implementations and evaluation results of straightforward extensions to VVenC which enable the use of the maximum rate parameter also in the single-pass fixed-QP modes, controlled by a base quantization parameter (QP) instead of a target rate. The rate capping in the fixed-QP mode is achieved, with sufficient accuracy, by evaluating only already calculated pre-processing statistics, thereby avoiding increases in encoder runtime. This encoding mode, given that it supports visual quality optimizations such as XPSNR based block-wise perceptual QP adaptation, can be considered a rate capped constant-quality mode, which was missing in VVenC and which is an interesting configuration for video streaming.

The Versatile Video Coding (VVC) standard promises high compression efficiency for diverse content types. Based on VVenC, an open and optimized VVC software video encoder, this work presents an inter-frame parallelization (IFP) method designed to exploit the processing power of modern platforms featuring a high number of computing cores. Encoding an ultrahigh definition video on a 32-core machine with the VVenC's faster preset, the proposed method shows more than 20% increase in encoder speed while only a 1% decrease in compression efficiency compared to the default multi-threading mode. In comparison to single-threaded mode, it corresponds to a speedup factor of 18, up from 15x achievable with the previous parallelization scheme. Furthermore, the synergy of the developed inter-frame parallelization technique with other parallelization methods is explored, including tiles and VVC wavefront parallel processing (WPP). The combination of these approaches enables a notable speedup factor of 21, albeit with a trade-off in coding efficiency. With a focus on VVC, this research contributes to the ongoing discourse on video coding optimization, providing valuable insights into possible pitfalls and the potential gains achievable through efficient parallelization techniques on high-core platforms.

The Versatile Video Coding (VVC) standard has been developed to meet the ever-increasing demand for higher compression of digital video data. Compared to its predecessor, the High-Efficiency Video Coding (HEVC) standard, VVC reduces the bitrate by around 50% for the same perceived quality. This increase in compression efficiency is associated with an increase in computational complexity, mainly on the encoder side. As an open and optimized VVC software encoder implementation, VVenC integrates algorithmic optimizations for each coding tool in VVC. This allows to define a set of five presets from faster to slower as Pareto-optimal tradeoffs between runtime and efficiency. On top, multithreading allows to reduce the runtime and preserves most of the compression efficiency of each preset. This paper presents and analyses the different multi-threading strategies in VVenC. Using a combination of pre-processing, picture-level and in-picture parallelization, VVenC can achieve a parallelization speedup with a factor of 4 for 4 threads while reducing the compression efficiency by only 0.4%. For higher thread numbers, i.e. 16, the speedup depends on the video resolution and used encoder preset, ranging from 6-9 for high definition to 10-12 for ultrahigh definition video with similar loss of compression efficiency. Using additional wavefront and tiles in-picture parallelization, higher speedups can be achieved at the costs of decreased coding efficiency.

Contemporary motion picture content, consisting of scenes with different amounts of visual complexity or camera noise, represents demanding input for video encoders operating in rate control (RC) modes. This paper presents improvements to the 2-pass RC method integrated into VVenC, an open VVC encoder implementation, outlined in previous publications. We specifically introduce three extensions to our RC solution: first, frame type adaptation operating near scene cuts, along with an associated simple detector; second, rate stabilization means to allow for more reliable lookahead based 2-pass RC operation in on-the-fly encoding applications; and third, a low-complexity approach for estimating the instantaneous intensity of camera noise or film grain to avoid large variations in bit consumption when encoding individual frames in the final RC pass. Experimental evaluation confirms that these extensions significantly improve both the objective (BD rate) and subjective (visual) RC performance of VVenC especially on challenging video content.

Two-pass rate control (RC) schemes have proven useful for generating low-bitrate video-on-demand or streaming catalogs. Visually optimized encoding particularly using latest-generation coding standards like Versatile Video Coding (VVC), however, is still a subject of intensive study. This paper describes the two-pass RC method integrated into version 1 of VVenC, an open VVC encoding software. The RC design is based on a novel two-step rate-quantization parameter (R-QP) model to derive the second-pass coding parameters, and it uses the low-complexity XPSNR visual distortion measure to provide numerically as well as visually stable, perceptually R-D optimized encoding results. Random-access evaluation experiments confirm the improved objective as well as subjective performance of our RC solution.

Lossy compression is the main target of the upcoming video coding standard Versatile Video Coding (VVC). However, lossless coding is supported in VVC by utilizing a certain encoder configuration. Particularly, the Transform Skip Mode (TSM) is always selected at the block level to bypass the transform stage (together with a QP that results in the same output as input at the quantization stage). Consequently, the Intra Subpartition (ISP) coding mode cannot be used for lossless coding, considering that its combination with TSM is not supported in VVC because it does not provide a significant coding benefit for the lossy common test conditions. For this reason, it is proposed to enable such a combination for the benefit of lossless coding. Besides, the encoder search has been optimized to improve the trade-off between compression benefit and encoder run-time. Experimental results show a 0.71% coding gain with a corresponding encoder run-time of 111%.

Versatile Video Coding (VVC) is a new international video coding standard to be finalized in July 2020. It is designed to provide around 50% bit-rate saving at the same subjective visual quality over its predecessor, High Efficiency Video Coding (H.265/HEVC). During the standard development, objective bit-rate savings of around 40% have been reported for the VVC reference software (VTM) compared to the HEVC reference software (HM). The unoptimized VTM encoder is around 9x, and the decoder around 2x, slower than HM. This paper discusses the VVC encoder complexity in terms of soft-ware runtime. The modular design of the standard allows a VVC encoder to trade off bit-rate savings and encoder runtime. Based on a detailed tradeoff analysis, results for different operating points are reported. Additionally, initial work on software and algorithm optimization is presented. With the optimized software algorithms, an operating point with an over 22x faster single-threaded encoder runtime than VTM can be achieved, i.e. around 2.5x faster than HM, while still providing more than 30% bit-rate savings over HM. Finally, our experiments demonstrate the flexibility of VVC and its potential for optimized soft-ware encoder implementations.

The upcoming Versatile Video Coding (VVC) Standard includes various new intra prediction tools that were not present in its predecessor High Efficiency Video Coding (HEVC), such as the wide angle intra prediction, the position dependent prediction combination or the multiple reference line intra prediction. In order to improve the intra prediction coding efficiency, this paper proposes the usage of the Intra Subpartition (ISP) algorithm. ISP is an updated version of the Line-Based Intra Prediction (LIP) mode that improves the trade-off between coding gain and complexity of the original method. The basic principle of ISP consists in subdividing an intra-predicted block into 2 or 4 subpartitions of at least 16 samples according to the original block dimensions. The method has been implemented on top of the VVC Test Model 3.0 (VTM-3.0), with the result of obtaining a gain of 0.57% with encoding and decoding run-times of 112% and 102% respectively for the All Intra configuration and a gain of 0.29% with encoding and decoding run-times of 102% and 101% respectively for the Random Access configuration.

Block partitioning is the basis of every modern hybrid video coding standard. It specifies how the video pictures can be subdivided into blocks for prediction and residual coding. In H.265/HEVC, quad-tree partitioning is one of the key technologies allowing for flexible mode allocation and providing a substantial part of the gains over H.264/AVC. The current draft of the upcoming standard Versatile Video Coding (VVC) provides over 30% bit-rate savings over HEVC and almost one third of the gain is achieved by using a more flexible partitioning scheme than the quad-tree partitioning used in HEVC. In this paper, we describe a partitioning concept that generalizes many of the ideas developed during the exploration and early standardization phase of VVC. In fact, our method includes the VVC partitioning as well as many other state-of-the-art methods. The proposed method can be implemented in a fully configurable design. For instance, it can be configured to match the performance of VTM-1.0 at much faster runtime (69%) or it can be configured to obtain additional bit-rate savings of up to 3% by exploiting additional degrees of freedom.

In this paper we propose and evaluate a parallelization strategy for the emerging HEVC video coding standard. The proposed strategy is based on entropy slices which allows exploiting parallelism in the entropy decoding stage while maintaining high coding efficiency. Our approach requires to encode videos with one entropy slice per LCU row in order to decode multiple LCU rows in a wavefront parallel manner. Evaluations performed on a PC with 12 Intel Xeon cores running at 3.3 GHz show that it is possible to achieve real-time performance for 1920×1080p50 (53.1 fps) and 2560×1600 (29.5fps) video resolutions with speedups of 5.2× and 6.3× compared to sequential execution, respectively.

In this paper we present a new parallelization approach for HEVC decoding called Overlapped Wavefront (OWF). It is based on wavefront processing and improves its parallelization efficiency by allowing overlapped execution of consecutive pictures. Furthermore, in this strategy of the decoding steps are performed on a CTB basis rather than on a picture basis, which improves data locality. Our implementation achieves between 29.6%, 42.4%, and 66.6% higher frame rates compared to previous results and 11.3%, 21.0%, and 38.0% higher frame rates compared to Tiles, for 2160p, 1600p, and 1080p, respectively.

The state-of-the-art hybrid video coding standard H.264/AVC defines two entropy-coding schemes with different complexity-performance trade-offs. Supporting these two schemes within a single standard raises several problems ranging from higher efforts for product development to increased silicon costs for hardware implementations. To overcome these issues, this work proposes a unified and complexity-scalable entropy-coding framework that is based on PIPE/V2V. The proposed framework uses a single set of tools for all configurations and achieves the same complexity-performance trade-offs as the existing entropy-coding schemes through scalability.

In hybrid video coding, an entropy coding scheme transmits the quantized transform coefficients, resulting from block-based transformation and quantization of the difference between the prediction signal and the original signal, and additional side information. The state-of-the-art hybrid video coding standard H.264/AVC defines two different entropy coding schemes with different complexity-performance tradeoff. As a result, the support for two different entropy coding schemes has to be maintained and introduces several problems. To overcome these issues, a unified solution is proposed, which is based on the PIPE/V2V coding concept. It achieves the same complexity-performance trade-offs as the existing entropy coding schemes by scalability. The advantage of the proposed scheme over the existing concept is the usage of the same set of tools for all configurations. Simulation results and complexity analysis on hardware show the efficiency of the proposed scheme.

In this paper, we present a multipoint video conferencing system that adapts to heterogeneous members including mobiles. The system is built upon a low complexity scalable extension of our H.264 codec DAVC, and a congestion-aware dynamic adaptation layer. Our temporally scaled video codec DSVC has the same RD performance as the non-scaled version with comparable configuration. We achieve this by QP cascading, i.e., assigning gradually refining quantization parameters to the declining temporal layers. We present and analyse a mobile-compliant version of DSVC at reduced complexity that still admits comparable performance. Finally, we report on early work of dynamic layer tuning. Derived of delay variation measures, senders exploit scalable video layering to adapt the video transmission to varying network conditions. Initial results indicate that video performance remains close to optimal.

Block-based video-coding for 4:4:4 color sampling is extended by an adaptive color space transform. The presented technique enables an encoder to switch between several given color space representations in order to optimize rate-distortion performance. Simulations based on the current draft of the H.264/MPEG4-AVC 4:4:4 extensions demonstrate that our technique provides a rate-distortion performance equal or better than that obtained when using any of the individual color spaces only.