We introduce Region-Adaptive Learned Hierarchical Encoding (RALHE) for 3D Gaussian Splatting (3DGS) data. While 3DGS has recently become popular for novel view synthesis, the size of trained models limits its deployment in bandwidth-constrained applications such as volumetric media streaming. To address this, we propose a learned hierarchical latent representation that builds upon the principles of “overfitted” learned image compression (e.g., Cool-Chic and C3) to efficiently encode 3DGS attributes. Unlike images, 3DGS data have irregular spatial distributions of Gaussians (geometry) and consist of multiple attributes (signals) defined on the irregular geometry. Our codec is designed to account for these differences between images and 3DGS. Specifically, we leverage the octree structure of the voxelized 3DGS geometry to obtain a hierarchical multi-resolution representation. Our approach overfits latents to each Gaussian attribute under a global rate constraint. These latents are decoded independently through a lightweight decoder network. To estimate the bitrate during training, we employ an autoregressive probability model that leverages octree-derived contexts from the 3D point structure. The multi-resolution latents, decoder, and autoregressive entropy coding networks are jointly optimized for each Gaussian attribute. Experiments on 3DGS models from the Synthetic-NeRF dataset demonstrate that the proposed RALHE compression framework achieves a rendering PSNR gain of up to 2 dB at low bitrates ($\leq 1 \text{MB}$) compared to the baseline 3DGS compression methods.

This paper describes the background, design and application of a new SEI message – the Multiplane Image Information SEI message, which has recently been adopted into the Technology under Consideration (TuC) document of the JVET committee for potential inclusion in the VSEI standard (ITU-T H.274 and ISO/IEC 23002-7). The paper also provides preliminary compression experiment results and analysis on the implications of the coding efficiency and functionality of the different packing options supported in the SEI message.

Intra template matching prediction (Intra TMP) is a promising intra prediction tool which generates the prediction block by copying from a reconstructed block of the current frame. The position of the reconstructed block is derived by template matching at both encoder and decoder. Intra TMP has been adopted in enhanced compression model (ECM) for both screen content and natural content due to its outstanding trade-off between coding efficiency and complexity. This paper describes an advanced intra TMP algorithm with fusion techniques to further improve the coding efficiency. The proposed intra TMP fusion scheme includes the following aspects: 1) extended template matching search range, 2) improved search procedure with multiple candidates, 3) adaptive fusion method with template updating, and 4) support fractional-pel precision in intra TMP. Experimental results show that the proposed intra TMP fusion scheme provides 0.76% average luma Bjøntegaard delta rate (BD-rate) reduction with negligible runtime increase over ECM-8.0 in all-intra configuration.

In this paper, a novel low complexity residual block fusion (RBF) based split luma chroma architecture is proposed to improve coding efficiency of neural network-based in-loop filter in video compression. The residual block in this architecture consists of a 1x1 convolution layer with wide activation and a regular 3x3 convolutional layer decomposed into 1x1 pointwise convolutions and 1x3/3x1 separable convolutions via Canonical Polyadic (CP) decomposition to reduce complexity. By adjusting the location of the skip connection in each residual block, the fusion of adjacent 1x1 pointwise convolutions is performed. The RBF backbone consists of a new wide activation that directly starts with PReLU and is followed by a 1x1 convolution, while the 1x1 layers after CP decomposition are fully fused. This new fusion design reduces the complexity from 17.05 kMac/Pixel to 16.56 kMac/Pixel and the number of convolutional layers by 13%. The experimental results show that new RBF architecture’s BDRate is {-0.11%, -0.31%, -0.33%} under All Intra (AI) and {-0.14%, 0.66%, 1.56%} under Random Access (RA) compared to existing residual block design, while the BD-Rate of the proposed RBF loop filer compared to VTM anchor is {-4.77%, -9.14%, -9.13%} under AI and {-5.46%, -9.31%, -9.20%} under RA. The actual decoding time is reduced by around 5% after residual block fusion. The BD-Rate and kMac/Pixel plot also shows superior trade-off between complexity and coding gain compared to state-of-the-art filters.

In this paper, a novel low complexity convolutional neural network with fused CP decomposition is proposed for in-loop filtering in video coding. Based on the baseline model in JVET-X0140, the regular 3x3 convolutional layers are replaced by pointwise convolutions and separable convolutions via CP decomposition. We further propose to fuse the 1x1 pointwise convolutional layers among the decomposed layers with their adjacent regular 1x1 convolutional layers, resulting in one single 1x1 convolutional layer. The two procedures reduce the model complexity from 33.6 KMAC/Pixel to 16.265 KMAC/Pixel. Experimental results show that the model has 4.45% BD-Rate luma gain over VTM NNVC-2.0. It demonstrates the (0.56%, -0.63%, -1.89%) loss of (Y, U, V) for RA and (0.51%, 0.21%, 0.39%) for AI, while the CPU decoding time is reduced by 19% for RA and 24% for AI, proving the great ability of the fused CP decomposition model to reduce complexity while maintaining good trade-off. The BD-Rate and KMAC/Pixel plot also shows the superior trade-off between complexity and coding gain compared to state-of the-art filters.

This paper describes a new video coding tool in the Versatile Video Coding standard (VVC) named as luma mapping with chroma scaling (LMCS). Experimental compression performance results for LMCS and non-normative examples for deriving LMCS parameter values are also provided. LMCS has two main components: 1) a process for mapping input luma code values to a new set of code values for use inside the coding loop; and 2) a luma-dependent process for scaling chroma residue values. The first process, luma mapping, aims at improving the coding efficiency for standard and high dynamic range video signals by making better use of the range of luma code values allowed at a specified bit depth. The second process, chroma scaling, manages relative compression efficiency for the luma and chroma components of the video signal. The luma mapping process of LMCS is applied at the pixel sample level, and is implemented using a piecewise linear model. The chroma scaling process is applied at the chroma block level, and is implemented using a scaling factor derived from reconstructed neighboring luma samples of the chroma block.

This paper describes a video coding scheme submitted in response to the joint call for proposals (CfP) on video compression for capability beyond HEVC issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11(MPEG) in October 2017. It includes video coding techniques for the standard dynamic range (SDR) and high dynamic range (HDR) categories. Design of the core SDR codec in the response is based on the joint exploration model (JEM) reference software developed by the joint video exploration team (JVET). Some of key coding tools in the JEM are significantly simplified to reduce both average and worst-case complexity for hardware design with negligible coding performance loss. Furthermore, two additional coding technologies, namely multi-type tree (MTT) and decoder-side intra mode derivation (DIMD), are used to further improve coding efficiency. For the HDR category, besides the tools used in SDR category, two additional coding tools: an in-loop reshaper and a luma-based QP prediction method are used to further improve HDR coding efficiency. Simulation results demonstrate the high coding efficiency achieved by the proposed video codec at the expense of moderate coding complexity over HEVC. For random access configuration, it achieves average bit rate savings of 35.7% and 4.00% over the HM and JEM anchors with decoding time of 263% and 33%, respectively, for the SDR sequences. For the HDR sequences, the proposed in-loop reshaper is configured to maximize HDR objective metrics, it achieves average bit rate savings of 31.3% and 4.6% over the HM and JEM for wPSNRY metrics for the HDR PQ content.

ICTCP is a colour difference signal representation recently standardized in Rec. ITU-R BT.2100. One design criteria of ICTCP for HDR/WCG imagery is to address issues in Non-Constant Luminance (NCL) Y'CbCr with HDR/WCG signals. In this paper, the superior performance of ICTCP in preserving constant luminance over the NCL Y'CbCr is analyzed in detail. The fundamental limitations of Y'CbCr luma adjustment algorithm which aims at fixing chroma subsampling artifact of Y'CbCr is discussed as well. This paper also provides guidance on signaling of ICTCP and feasible encoder settings with HEVC Main 10 codec. Experimental results suggest that for HDR content with real BT. 2100 (BT. 2020) colour gamut, to perform chroma subsampling and compression in ICTCP have significant value.

The paper presents the joint proposal by Arris, Dolby and InterDigital as a response to the Call-for-Evidence of the High Dynamic Range and Wide Color Gamut (HDR/WCG) video compression in MPEG. The joint proposal introduces a set of new HDR coding technologies, including the IPT-PQ color space, the adaptive reshaping process, the color enhancement filters, and the adaptive transfer function. These new coding technologies are applied to the decoded output of an HEVC decoder. Hence, no changes to the lower level logics of the HEVC decoder are required to implement the proposal. Formal subjective tests conducted by MPEG confirmed that the proposal could achieve significant subjective quality improvements over the HEVC Main10 anchors at similar bit rates for HDR/WCG video content.