Intra prediction is a vital tool in video coding that eliminates the spatial redundancy within a frame to enhance compression efficiency. Conventional intra prediction methods employ multiple directional prediction modes to describe textures in local areas. Recently, research on neural network-based intra prediction has achieved great success. The block-context pairs are divided into multiple clusters according to a predefined relationship, and a corresponding network is trained and applied for each cluster. However, the networks in these methods adopt fixed parameters to predict diverse image blocks, making it hard to cope with various textures in natural images. Inspired by recent works on parameter prediction, in this paper, we propose a meta-network-based intra prediction method, called MetaIP, that dynamically customizes the network parameters for each block sample in a given cluster. MetaIP consists of a meta-subnetwork and a prediction subnetwork. For an image block, the meta-subnetwork takes its neighboring reference pixels and some auxiliary information (e.g., quantization parameter) as inputs to generate customized parameters first. Then, the prediction subnetwork uses the customized parameters to infer the predicted block. MetaIP can generate multiple sets of network parameters corresponding to multiple modes for an image block. The optimal mode is determined by the rate-distortion optimization. MetaIP is integrated into VVC to assist or replace the directional prediction modes to evaluate its performance. The experimental results demonstrate that MetaIP with four prediction modes achieves an average of 3.84% and 1.96% bitrate saving for the luma component over VTM-17.0 when assisting or replacing VVC intra modes, respectively.

The sphere-to-plane projection of 360-degree video introduces substantial stretched redundant data, which is discarded when reprojected to the 3D sphere for display. Consequently, encoding and transmitting such redundant data is unnecessary. Highly redundant blocks can be referred to as all-zero blocks (AZBs). Detecting these AZBs in advance can reduce computational and transmission resource consumption. However, this cannot be achieved by existing AZB detection techniques due to the unawareness of the stretching redundancy. In this paper, we first derive a latitude-adaptive redundancy detection (LARD) approach to adaptively detect coefficients carrying redundancy in transformed blocks by modeling the dependency between valid frequency range and the stretching degree based on spectrum analysis. Utilizing LARD, a latitude-redundancy-aware AZB detection scheme tailored for fast 360-degree video coding (LRAS) is proposed to accelerate the encoding process. LRAS consists of three sequential stages: latitude-adaptive AZB (L-AZB) detection, latitude-adaptive genuine-AZB (LG-AZB) detection and latitude-adaptive pseudo-AZB (LP-AZB) detection. Specifically, L-AZB refers to the AZB introduced by projection. LARD is used to detect L-AZB directly. LG-AZB refers to the AZB after hard-decision quantization and zeroing redundant coefficients. A novel latitude-adaptive sum of absolute difference estimation model is built to derive the threshold for LG-AZB detection. LP-AZB refers to the AZB in terms of rate-distortion optimization considering redundancy. A latitude-adaptive rate-distortion model is established for LP-AZB detection. Experimental results show that LRAS can achieve an average total encoding time reduction of 25.85% and 20.38% under low-delay and random access configurations compared to the original HEVC encoder, with only 0.16% and 0.13% BDBR increases and 0.01dB BDPSNR loss, respectively. The transform and quantization time savings are 60.13% and 59.94% on average.

Deep learning has shown its great potential in video coding. Lots of research has been done regarding deep tools used within traditional schemes or end-to-end deep codec. In this paper, we propose multiple novel deep tools focusing on intra prediction, motion compensation refinement, virtual reference frame, and post-processing and form a deep learning-assisted video compression framework. For intra prediction, a novel data clustering-driven neural network (DCDNN), which could learn deep features of the clustered data, is proposed to assist intra angular modes. For inter prediction, a neural network-based motion compensation refinement (NNMCR) algorithm is applied to enhance the motion compensation. Meanwhile, a V-RF network is employed to generate a more reliable reference frame that can characterize complex motions in natural video signals. For post-processing, a quality adaptive neural network-based in-loop filter (QANNLF) is designed to utilize Quality Parameter (QP) information in the encoder to adjust filter strength for reconstructed videos with various qualities. Our method can obtain on average 14.83% BD-rate saving for the luma component under the all intra configuration and 12.16% under the random access configuration compared with HEVC reference software 16.9.