This paper describes the background, design and application of a new SEI message – the Multiplane Image Information SEI message, which has recently been adopted into the Technology under Consideration (TuC) document of the JVET committee for potential inclusion in the VSEI standard (ITU-T H.274 and ISO/IEC 23002-7). The paper also provides preliminary compression experiment results and analysis on the implications of the coding efficiency and functionality of the different packing options supported in the SEI message.

Intra template matching prediction (Intra TMP) is a promising intra prediction tool which generates the prediction block by copying from a reconstructed block of the current frame. The position of the reconstructed block is derived by template matching at both encoder and decoder. Intra TMP has been adopted in enhanced compression model (ECM) for both screen content and natural content due to its outstanding trade-off between coding efficiency and complexity. This paper describes an advanced intra TMP algorithm with fusion techniques to further improve the coding efficiency. The proposed intra TMP fusion scheme includes the following aspects: 1) extended template matching search range, 2) improved search procedure with multiple candidates, 3) adaptive fusion method with template updating, and 4) support fractional-pel precision in intra TMP. Experimental results show that the proposed intra TMP fusion scheme provides 0.76% average luma Bjøntegaard delta rate (BD-rate) reduction with negligible runtime increase over ECM-8.0 in all-intra configuration.

In this paper, a novel low complexity residual block fusion (RBF) based split luma chroma architecture is proposed to improve coding efficiency of neural network-based in-loop filter in video compression. The residual block in this architecture consists of a 1x1 convolution layer with wide activation and a regular 3x3 convolutional layer decomposed into 1x1 pointwise convolutions and 1x3/3x1 separable convolutions via Canonical Polyadic (CP) decomposition to reduce complexity. By adjusting the location of the skip connection in each residual block, the fusion of adjacent 1x1 pointwise convolutions is performed. The RBF backbone consists of a new wide activation that directly starts with PReLU and is followed by a 1x1 convolution, while the 1x1 layers after CP decomposition are fully fused. This new fusion design reduces the complexity from 17.05 kMac/Pixel to 16.56 kMac/Pixel and the number of convolutional layers by 13%. The experimental results show that new RBF architecture’s BDRate is {-0.11%, -0.31%, -0.33%} under All Intra (AI) and {-0.14%, 0.66%, 1.56%} under Random Access (RA) compared to existing residual block design, while the BD-Rate of the proposed RBF loop filer compared to VTM anchor is {-4.77%, -9.14%, -9.13%} under AI and {-5.46%, -9.31%, -9.20%} under RA. The actual decoding time is reduced by around 5% after residual block fusion. The BD-Rate and kMac/Pixel plot also shows superior trade-off between complexity and coding gain compared to state-of-the-art filters.

In this paper, a novel low complexity convolutional neural network with fused CP decomposition is proposed for in-loop filtering in video coding. Based on the baseline model in JVET-X0140, the regular 3x3 convolutional layers are replaced by pointwise convolutions and separable convolutions via CP decomposition. We further propose to fuse the 1x1 pointwise convolutional layers among the decomposed layers with their adjacent regular 1x1 convolutional layers, resulting in one single 1x1 convolutional layer. The two procedures reduce the model complexity from 33.6 KMAC/Pixel to 16.265 KMAC/Pixel. Experimental results show that the model has 4.45% BD-Rate luma gain over VTM NNVC-2.0. It demonstrates the (0.56%, -0.63%, -1.89%) loss of (Y, U, V) for RA and (0.51%, 0.21%, 0.39%) for AI, while the CPU decoding time is reduced by 19% for RA and 24% for AI, proving the great ability of the fused CP decomposition model to reduce complexity while maintaining good trade-off. The BD-Rate and KMAC/Pixel plot also shows the superior trade-off between complexity and coding gain compared to state-of the-art filters.

This paper describes a new video coding tool in the Versatile Video Coding standard (VVC) named as luma mapping with chroma scaling (LMCS). Experimental compression performance results for LMCS and non-normative examples for deriving LMCS parameter values are also provided. LMCS has two main components: 1) a process for mapping input luma code values to a new set of code values for use inside the coding loop; and 2) a luma-dependent process for scaling chroma residue values. The first process, luma mapping, aims at improving the coding efficiency for standard and high dynamic range video signals by making better use of the range of luma code values allowed at a specified bit depth. The second process, chroma scaling, manages relative compression efficiency for the luma and chroma components of the video signal. The luma mapping process of LMCS is applied at the pixel sample level, and is implemented using a piecewise linear model. The chroma scaling process is applied at the chroma block level, and is implemented using a scaling factor derived from reconstructed neighboring luma samples of the chroma block.