459

Data-driven optimization is employed to study alternative approaches [1] to the probability estimator of the the Enhanced Compression Model (ECM) (which includes additional coding tools on top of the Versatile Video Coding standard). In ECM, each context model uses a weighted sum of two hypotheses for probability estimation with different associated adaptation rates. Four alternative approaches are studied:

This paper presents a CNN to reduce the encoding time of a VVC-based intra-picture encoder. For encoding a 32 × 32 block, the CNN estimates two partitioning parameters that restrict the allowed coding block width and height. To estimate them such that the encoder skips testing inefficient partitioning modes, we train the CNN as follows: First, we generate training data by encoding sequences without the CNN. While encoding, we test all combinations of the two parameters for each 32 × 32 block and store the resulting Lagrangian rate-distortion-time (RDT) cost. We use the recorded cost to derive the loss function when training the CNN. Consequently, the CNN is trained such that it minimizes the Lagrangian RDT cost. Our CNN reduces the encoding time by 50% with a bit rate increase of 0.9%, which outperforms existing CNN-based approaches. Our generic training approach could also be applied for other encoder parameters.

In this paper we present a "State-Based Multi-Parameter Probability Estimation" (SBMP) for Context-Based Adaptive Binary Arithmetic Coding (CABAC) which employs a two hypotheses probability estimator based on exponentially weighted moving averages. It uses a logarithmic state representation and a single subsampled transition table with only 32 elements for the probability update. This reduces the memory requirements virtually without affecting the compression efficiency, compared to corresponding approaches that use a linear state representation and a computation-based probability update. The proposed scheme is based on simple operations like table look-ups and additions. Compared to the state-of-the-art probability estimator of the video compression standard H.265/HEVC, the compression efficiency is increased by up to 1 % Bjøntegaard-Delta bit rate (BD rate) when applied to draft 2 of the Versatile Video Coding (VVC) standard. Furthermore, SBMP was recently adopted to working draft 2 of the MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis.

The support for screen content coding has received more attention with the latest development in video compression, the upcoming Versatile Video Coding (VVC) standard. Among the dedicated screen content coding tools, the transform skip mode (TSM) represents a promising approach for improving the coding efficiency at a low impact on implementation complexity. In this work, we present a dedicated residual coding for transform blocks coded in TSM. Due to the lack of the energy compaction of the transform, the quantization indexes for blocks coded in TSM have different statistical properties, which can be exploited in the entropy coding. Our coding experiments with screen content sequences yielded bit-rate savings of 3.9% for intra-only coding and 2.8% for typical random access configurations.

Intra-picture prediction as in HEVC exploits the nearest reference line adjacent to the current coding unit (CU) for prediction of samples. If this reference line represents a discontinuity, the reference samples in this reference line can differ to a large extent from the original samples and may lead to a large prediction error. We propose a multiple reference lines (MRLs) coding to allow not only the nearest reference line 0 but also reference lines 1 and 3 to be candidates for angular intra prediction as shown in Fig. 1. To reduce the complexity arising from additional lines to be checked at encoder side, we further propose to restrict the MRL to angular most probable modes (MPMs) only. The MRL coding signals the reference line index before the intra prediction mode. This allows to not signal the MPM flag of the current CU and implicitly derive it as true when a non-zero reference line index is signaled. Experimental results are provided to evaluate the performance of the proposed MRL coding on top of the VVC test model VTM-2.0.1. 26 test sequences in different categories, including 4k, 1080p, 720p, WVGA, WQVGA resolutions and screen contents are tested. Two coding structures are evaluated, all intra (AI) and random access (RA). The objective coding efficiency is measured in terms of Bjøntegaard Delta (BD) rate (%) computed using four rate/PSNR points that were generated by using quantization parameters 22, 27, 32 and 37. Lower (negative) BD-rate implies better compression rate. Table 1 shows that the presented MRL provides 0.46% bitrate savings for an all-intra and 0.2% for a random-access configuration on average. Furthermore, it provides 1.45% bitrate reduction for screen content test sequences, which are representing an increasingly important video application. Because of a fairly good trade-off between coding efficiency and complexity, the proposed MRL coding mode with MPM restriction was adopted into the current VVC draft standard.

Versatile Video Coding (VVC) is going to be the successor of High Efficiency Video Coding (HEVC). The new standard (currently under development) requires the adoption of tools that provide a good trade-off between gain and implementation complexity. In previous work we proposed a Line-Based Intra Prediction (LIP) algorithm to improve the coding performance of the upcoming standard. However, that contribution did not sufficiently take into account the hardware-related complexity of the method. For this reason, this publication addresses these problems and proposes several modifications of the algorithm to solve them. Experimental results show an average gain of 1.0% and 0.5% for the All Intra (AI) and Random Access (RA) configurations, respectively.

We train a neural network to perform intra picture prediction for block based video coding. Our network has multiple prediction modes which co-adapt during training to minimize a loss function. By applying the l1-norm and a sigmoid-function to the prediction residual in the DCT domain, our loss function reflects properties of the residual quantization and coding stages present in the typical hybrid video coding architecture. We simplify the resulting predictors by pruning them in the frequency domain, thus greatly reducing the number of multiplications otherwise needed for the dense matrix-vector multiplications. Also, by quantizing the network weights and using fixed point arithmetic, we allow for a hardware friendly implementation. We demonstrate significant coding gains over state of the art intra prediction.

It is well known that input-invariant quantization in perceptual image or video coding often leads to visually suboptimal results and that quantization parameter adaptation (QPA) based on a model of the human visual system can improve subjective coding quality. This paper introduces a simple low-complexity QPA algorithm, controlled using a block-wise perceptually weighted distortion measure representing a generalization of the PSNR metric. The weighting scheme of this WPSNR metric is based on a psychovisual model. It directly leads to a perceptually adapted scaling of the block-wise Lagrange parameter used in the bit-allocation process in the encoder and, consequently, to a block-wise QPA. Unlike prior QPA approaches, the proposal avoids classifications of picture regions and easily extends from still-image or grayscale to video or chromatic coding. The WPSNR metric also uses fewer algorithmic operations than e. g. the multiscale structural similarity measure (MS-SSIM). Due to the results of two formal subjective tests indicating its visual benefit, the QPA proposal has been adopted into VTM, the currently developed Versatile Video Coding (VVC) reference software.

In state-of-the-art video coding, the prediction error signals are transmitted using transform coding, which consists of an orthogonal transform, scalar quantization, and entropy coding of the quantization indexes. We show that the coding efficiency of transform coding can be improved by replacing scalar quantization with trellis-coded quantization (TCQ) and using advanced entropy coding techniques for coding the quantization indexes. The proposed approach was implemented into the first test model (VTM-1) of the new standardization project Versatile Video Coding (VVC). Our coding experiments yielded average bit-rate savings of 4.9% for intra-only coding and 3.3% for typical random access configurations, where bit-rate savings of 3.5% (intra-only) and 2.4% (random access) can be attributed to the usage of TCQ. These coding gains are obtained at a 5-10% increase in encoder run time and without any change in decoder run time.

In a template matching (TM) intra method, the neighboring samples of the current block are regarded as a template. The decoder searches for the best template match in the current reconstructed picture using an error minimizing metric like sum of squared differences (SSD). The prediction signal is generated by copying the samples of the adjacent block to the selected template match. The increased decoder complexity from the search algorithm makes it less attractive for modern applications like video calling. In a previous publication [1], we presented a region-based template matching (RTM) approach for intra coding. Compared to the conventional TM methods which searches for the template match in a complete search window, RTM searches in a region of the search window. Thus, RTM offers a better trade-off between coding efficiency and decoder complexity. Nevertheless, the memory requirements and number of computations to be carried at the decoder are still high, making RTM difficult for hardware realization. This paper aims to address these issues.

One of the innovations in H.265/HEVC is the quad-tree partitioning framework. It allows flexible block subdivision and mode allocation across the encoded picture. The increased flexibility comes at a cost of vast search space expansion, making exhaustive search algorithms inapplicable. We propose a novel early termination condition to skip the exhaustive search of whole tree-branches in the well-established top-down encoding approach. The condition is based on a simple and intuitive split cost prediction. It can be parametrized to control the trade-off between the speed-up and caused BD-rate loss. Data driven parameter estimation and parameter number reduction is presented. For random-access encoding, the method can achieve an average speed-up of 30% with a BD-rate loss of 0.03%. At another trade-off point, speed-up is increased to over 40% for a BD-rate loss below 0.5%.

The paper describes a scalable video coding extension of the upcoming HEVC video coding standard for spatial and quality scalable coding. Besides coding tools known from scalable profiles of prior video coding standards, it includes new coding tools that further improve the enhancement layer coding efficiency. The effectiveness of the proposed scalable HEVC extension is demonstrated by comparing the coding efficiency to simulcast and single-layer coding for several test sequences and coding conditions.

Multi-hypothesis motion-compensated prediction extends traditional motion-compensated prediction used in video coding schemes. Known algorithms for block-based multi-hypothesis motion-compensated prediction are, for example, overlapped block motion compensation (OBMC) and bidirectionally predicted frames (B-frames). This paper presents a generalization of these algorithms in a rate-distortion framework. All blocks which are available for prediction are called hypotheses. Further, we explicitly distinguish between the search space and the superposition of hypotheses. Hypotheses are selected from a search space and their spatio-temporal positions are transmitted by means of spatio-temporal displacement codewords. Constant predictor coefficients are used to combine linearly hypotheses of a multi-hypothesis. The presented design algorithm provides an estimation criterion for optimal multi-hypotheses, a rule for optimal displacement codes, and a condition for optimal predictor coefficients. Statistically dependent hypotheses of a multi-hypothesis are determined by an iterative algorithm. Experimental results show that Increasing the number of hypotheses from 1 to 8 provides prediction gains up to 3 dB in prediction error.