This paper presents a live demonstrator that is designed to visualize the encoding process of uvg266 VVC open-source intra encoder. The proposed visualizer provides real-time insights into the decision-making process of uvg266 by illustrating chosen block structures, modes, transforms, and parallelization strategies. The tool also facilitates interactive analysis through a user-friendly interface that allows users to dynamically adjust encoding tools and parameters on the fly. Researchers, developers, and students can use the tool to enhance their understanding of VVC coding flow and to support their optimization and educational efforts.

Versatile Video Coding (VVC/H.266) standard is the emerging successor to the widespread High Efficiency Video Coding (HEVC/H.265). This work introduces the latest version of our academic open-source VVC intra encoder called uvg266. It has been developed from our well-known Kvazaar HEVC encoder by introducing new VVC coding tools into carefully optimized and parallelized coding flow of Kvazaar. This paper outlines the design methodology and implementation aspects of all intra (AI) configuration of uvg266. The experimental results show that single-threaded uvg266 is more than twice as fast as the state-of-the-art VVenC encoder in all our test cases. In speed-optimized coding, the coding overhead of uvg266 is 21.7% but the gap narrows down to 2.6% in the rate distortion optimized coding case. uvg266 has almost linear speedup with core count up to 32 threads. The better scalability of uvg266 quadruples the speed over VVenC. Furthermore, single-threaded uvg266 is up to 380× as fast as VVC reference software VTM and the gap raises to over 11,000× with 32 threads. To the best of our knowledge, uvg266 is currently the fastest available open-source VVC intra software encoder.

Video-based Point Cloud Compression (V-PCC) is the latest MPEG standard for visual volumetric data coding. V-PCC leverages the coding efficiency of well-established 2D video codecs by creating patches that link sub-regions of a point cloud to their 2D projections. This paper provides an in-depth analysis of the V-PCC patch generation process, which is the most compute-intensive part of the V-PCC standard. To the best of our knowledge, this is the first work to explore the patch generation tools individually and evaluate their impact on coding efficiency and complexity with different coding parameters. The main purpose of this characterization is to offer key insights into the V-PCC encoder design process and thereby foster the development of practical V-PCC encoders.

Reducing the huge computational complexity of intra mode decision is the key to real-time Video Coding (VVC). This paper proposes a fast intra mode decision scheme that takes advantage of lightweight machine learning (ML) models to classify intra modes into fifteen clusters. The cluster is further refined using one of the three proposed strategies to select the most optimal mode. Our experimental results with the fastest configuration of the practical uvg266 encoder show that the proposed methods yield a competitive rate-distortion-complexity trade-off over a conventional rough mode decision (RMD). To the best of our knowledge, this is the first work to successfully reduce the complexity of RMD in a practical VVC encoder with the use of ML techniques.

Versatile Video Coding (VVC) provides new coding tools for more efficient intra prediction but with a substantial increase in computational complexity. This paper introduces vectorized kernels for 8-bit angular intra prediction and position dependent intra prediction combination (PDPC), which are carefully optimized for all block sizes and prediction modes of VVC. The proposed kernels streamline the filtering process and utilize optimized memory access patterns. Our standalone tests show that the proposed vectorization achieves speedups of 6.68× for luma and 4.40× for chroma predictions over scalar implementations. Integrating these kernels into the practical uvg266 VVC encoder provides speedups of 1.07× in the slowest configuration and 1.68× in the fastest configuration. The reported speedups are obtained without any coding overhead, so the proposed vectorization plays an integral role in pursuing real-time VVC coding with high coding efficiency.

High Efficiency Video Coding (HEVC) creates the conditions for economic video transmission and storage but making most of its compression potential calls for effective rate-distortion optimization (RDO) techniques in practical HEVC encoders. This paper explores the effectiveness of the following universally applicable RDO techniques: 1) rough mode decision for intra RDO candidate selection; 2) number of intra and inter RDO search candidates; and 3) accurate bit cost estimation in entropy coding. All these techniques are implemented into Kvazaar open-source HEVC encoder and altogether they improve the coding efficiency of Kvazaar veryslow preset by 5.7%, 4.0%, and 7.1% with PSNR, SSIM, and VMAF quality metrics, respectively. Even though the proposed techniques reduce the coding speed of Kvazaar to 0.64×, Kvazaar is still, on average, 2.16× as fast as the x265 encoder and attains 12.4%, 22.3%, and 4.7% better coding gain for the same PSNR, SSIM, and VMAF quality, respectively. These results let us conclude that Kvazaar is currently the leading practical open-source solution for high-quality HEVC encoding.

High Efficiency Video Coding (HEVC) sets the stage for economic video transmission and storage, but its inherent computational complexity calls for powerful implementations. This paper addresses the principal performance bottleneck of HEVC codecs by introducing AVX2-vectorized algorithms for HEVC interpolation filters. The proposed speed-up techniques include 1) a data permutation scheme for the horizontal interpolation stage; 2) a sliding window strategy for the vertical interpolation stage; 3) optimal usage of horizontal and vertical interpolation during fractional motion estimation; and 4) a lane-based approach to double the vector lengths from 128-bit legacy vector extensions to 256bits of AVX2. Our AVX2-optimized interpolation filters were benchmarked as part of the practical Kvazaar open-source HEVC encoder. On an Intel 8-core Xeon processor, they were shown to be 9.7 and 8.5 times as fast as scalar interpolation with the Kvazaar ultrafast and veryslow presets, respectively. In both cases, changing over from scalar to vectorized interpolation increases the coding speed of Kvazaar by more than 50%, which stresses the importance of interpolation optimizations in modern video encoders.

Versatile Video Coding (VVC/H.266) takes video coding to the next level by doubling the coding efficiency over its predecessors for the same subjective quality, but at the cost of immense coding complexity. Therefore, VVC calls for aggressively optimized codecs to make it feasible for live streaming media applications. This paper introduces the first public end-to-end (E2E) pipeline for live 4K30p VVC intra coding and streaming. The pipeline is made up of three open-source components: 1) uvg266 for VVC encoding; 2) uvgRTP for VVC streaming; and 3) OpenVVC for VVC decoding. The proposed setup is demonstrated with a proof-of-concept prototype that implements the encoder end on AMD ThreadRipper 2990WX and the decoder end on Nvidia Jetson AGX Orin. Our prototype is almost 34 000 times as fast as the corresponding E2E pipeline built around the VTM codec. Respectively, it achieves 3.3 times speedup without any significant coding overhead over the pipeline that utilizes the fastest possible configuration of the well-known VVenC/VVdeC codec. These results indicate that our prototype is currently the only viable open-source solution for live 4K VVC intra coding and streaming.

Versatile Video Coding (VVC) introduces dependent quantization (DQ) as a key coding tool, but it also has a high computational complexity. Therefore, optimizing DQ is crucial for practical VVC encoder implementations. In this paper, we propose optimization techniques for reducing the complexity of the DQ process, including efficient structure initialization, vectorized state update, rate distortion (RD) cost calculation, and last nonzero (LNZ) coefficient identification. The proposed optimizations are incorporated into the uvg266 practical VVC encoder. According to our evaluations, these optimization techniques accelerate the DQ process by over 1.7× and the entire VVC intra encoder by 1.4×, without any coding efficiency degradation. The proposed optimizations achieve more than twice the speedup over the existing techniques.

This paper summarizes the key parallelization aspects of the two most popular academic rate control (RC) algorithms for High Efficiency Video Coding (HEVC): 1) Lambda domain (LD) [1] and 2) R-Lambda model (R-LM) [2] that were originally designed for the HEVC test model (HM). In this work, these two sequential RC algorithms were ported to the practical Kvazaar open-source HEVC encoder [3] and adapted to work with wavefront parallel processing (WPP) and overlapping wavefront (OWF) parallelization techniques.

The agile and efficient development of modern video encoders calls for automated testing methodologies. This paper presents the first-of-its-kind open-source test automation framework called uvgVenctester (github.com/ultravideo/uvgVenctester) that is designed for comprehensive performance and conformance testing of video encoders with the desired set of test video sequences. Our framework comes with built-in support for the popular AVC, HEVC, VVC, VP9, and AV1 video coding formats and the state-of-the-art HM, Kvazaar, x265, VTM, VVenC, SVT-VP9, and SVT-AV1 video encoders. Furthermore, there are no technical limitations of adopting other formats or encoders. The developers can evaluate the encoder of interest under the three primary usage scenarios: 1) conformance testing of the encoded bitstream; 2) rate-distortion-complexity comparison with the other encoders; and 3) systematic exploration of encoding parameters. The framework provides commonly used analysis tools to quantify encoding quality, speed, and bitrate with versatile set of absolute and comparative results such as Bjøntegaard Delta (BD)-Rate for PSNR, SSIM, and VMAF quality metrics. The supported output formats include CSV, graph, and comparison table. They ensure that the results are available in human and machine-readable formats. To the best of our knowledge, the proposed framework is currently the most comprehensive and modular open-source software toolset for video encoder benchmarking.

This paper presents a demonstrator setup for real-time HEVC encoding with gaze-based region of interest (ROI) detection. This proof-of-concept system is built on Kvazaar open-source HEVC encoder and Pupil eye tracking glasses. The gaze data is used to extract the ROI from live video and the ROI is encoded with higher quality than non-ROI regions. This demonstration illustrates that performing HEVC encoding with non-uniform quality reduces bit rate by 40-90% and complexity by 10-35% over that of the conventional approaches with negligible to minor deterioration in subjective quality.

Eye tracking is nowadays the primary method for collecting training data for neural networks in the Human Visual System modelling. Our recommendation is to collect eye tracking data from videos with eye tracking glasses that are more affordable and applicable to diverse test conditions than conventionally used screen based eye trackers. Eye tracking glasses are prone to moving during the gaze data collection but our experiments show that the observed displacement error accumulates fairly linearly and can be compensated automatically by the proposed framework. This paper describes how our framework can be used in practice with videos up to 4K resolution. The proposed framework and the data collected during our sample experiment are made publicly available.