This paper presents a CNN to reduce the encoding time of a VVC-based intra-picture encoder. For encoding a 32 × 32 block, the CNN estimates two partitioning parameters that restrict the allowed coding block width and height. To estimate them such that the encoder skips testing inefficient partitioning modes, we train the CNN as follows: First, we generate training data by encoding sequences without the CNN. While encoding, we test all combinations of the two parameters for each 32 × 32 block and store the resulting Lagrangian rate-distortion-time (RDT) cost. We use the recorded cost to derive the loss function when training the CNN. Consequently, the CNN is trained such that it minimizes the Lagrangian RDT cost. Our CNN reduces the encoding time by 50% with a bit rate increase of 0.9%, which outperforms existing CNN-based approaches. Our generic training approach could also be applied for other encoder parameters.

We train a neural network to perform intra picture prediction for block based video coding. Our network has multiple prediction modes which co-adapt during training to minimize a loss function. By applying the l1-norm and a sigmoid-function to the prediction residual in the DCT domain, our loss function reflects properties of the residual quantization and coding stages present in the typical hybrid video coding architecture. We simplify the resulting predictors by pruning them in the frequency domain, thus greatly reducing the number of multiplications otherwise needed for the dense matrix-vector multiplications. Also, by quantizing the network weights and using fixed point arithmetic, we allow for a hardware friendly implementation. We demonstrate significant coding gains over state of the art intra prediction.

In a template matching (TM) intra method, the neighboring samples of the current block are regarded as a template. The decoder searches for the best template match in the current reconstructed picture using an error minimizing metric like sum of squared differences (SSD). The prediction signal is generated by copying the samples of the adjacent block to the selected template match. The increased decoder complexity from the search algorithm makes it less attractive for modern applications like video calling. In a previous publication [1], we presented a region-based template matching (RTM) approach for intra coding. Compared to the conventional TM methods which searches for the template match in a complete search window, RTM searches in a region of the search window. Thus, RTM offers a better trade-off between coding efficiency and decoder complexity. Nevertheless, the memory requirements and number of computations to be carried at the decoder are still high, making RTM difficult for hardware realization. This paper aims to address these issues.

The paper describes a scalable video coding extension of the upcoming HEVC video coding standard for spatial and quality scalable coding. Besides coding tools known from scalable profiles of prior video coding standards, it includes new coding tools that further improve the enhancement layer coding efficiency. The effectiveness of the proposed scalable HEVC extension is demonstrated by comparing the coding efficiency to simulcast and single-layer coding for several test sequences and coding conditions.