The Video Multi-method Assessment Fusion (VMAF) has demonstrated a better correlation with Human Visual System than the conventional objective metrics, and has gradually gained adoption in the industry that needs to monitor the visual quality of compressed videos. However, due to its machine learning nature, it can not be expressed through a simple and explicit mathematical formula, which makes it difficult to incorporate VMAF into the rate-distortion optimization framework in video compression. In this work, we propose a new perspective that decomposes the VMAF as a superposition of spatial and temporal factors. The spatial factor, which also directly applies to image quality evaluation, is approximated by a saliency map. It in conjunction with the temporal factor approximated by the motion quantities allows a simple analytical formula that translates the mean squared distortion at each pixel to its impact to the overall VMAF metric. The proposed hypothesis is embedded into the rate-distortion optimization framework, and is experimentally shown to provide considerable coding gains in VMAF for both image and video compression.

Modern video codecs explore the temporal and spatial correlations of video signal to achieve the goal of compression. The noise in video signal corrupts such temporal and spatial correlations and thus is difficult to compress. Denoising of video signal is a potential solution to this problem. Despite the significant progress in video denoising in recent years, there is few research exploring the feasibility of denoising for video compression. In this work, we demonstrate that video denoising is able to significantly reduce bit rates while maintaining the subjective and objective quality when appropriately incorporated into the hierarchical coding structure of video coding. We present a temporal filtering algorithm for denoising and apply it to AV1 for lossy video compression. We obtain a significant compression efficiency improvement over videos of different resolutions, types, and noise.

Modern video codec uses arithmetic coding for entropy coding. The arithmetic coding asymptotically achieves the entropy bound provided the true probability distribution. Hence the compression efficiency heavily relies on the ability to capture the time-variant probability model in video signals. Variants of first-order linear probability model update schemes have been used in recent generation video codecs. Built on top of those, a multimodal estimation scheme that forms a higher order probability model update has been proposed in this work. We experimentally demonstrate its coding efficiency.

The AV1 codec recently released by the Alliance of Open Media provides nearly 30% BDrate reduction over its predecessor VP9. It substantially extends the available coding block sizes and supports a wide range of prediction modes. There are also a large variety of transform kernel types and sizes. The combination provides an extremely wide range of flexible coding options. To translate such flexibility into compression efficiency, the encoder needs to conduct an extensive search over the space of coding modes. Optimization of the encoder complexity and compression efficiency trade-off is critical to productionizing AV1. Many research efforts have been devoted to devising feature space based pruning methods ranging from decision rules based on some simple observations to more complex neural network models. A multi-pass coding mode search framework is proposed in this work to provide a structural approach to reduce the search volume. It decomposes the original high dimensional space search into cascaded stages of lower dimensional space searches. To retain a near optimal search result, the scheme departs from conventional dimension reduction approach in which one retains a single winner at each stage, and uses that winner for the next stage (dimension). Instead, this framework retains a subset of the states that are the most likely winners at each stage, which are then fed into the next stage to find the next subset of winners. The subset size at each stage is determined by the likelihood that the optimal route will be captured in the current stage. Changing this likelihood parameter tunes the encoder for speed and compression performance trade-off. This framework can integrate with most existing feature based methods at its various stages. The framework provides 60% encoding time reduction at the expense of 0.6% compression loss in libaom AV1 encoder.

The hierarchical coding structure that supports bi-directional motion compensated prediction is commonly used for video compression efficiency. Conventional approach directly seeks the reference pixel block from each individual reference frame and use it or its linear combinations for prediction. It largely ignores the motion information between these reference frames. To fully utilize all the information from the bi-directional reference frames, this work builds a per-pixel motion field that connects the two-sided reference frames using optical flow estimation. A reference frame is then interpolated at the current frame location. This collocated reference frame effectively accounts for the true motion trajectories in the video signal including both translational and the more complex non-translational motion models, which are beyond the capability of the conventional block-based motion compensated prediction. The scheme is experimentally shown to provide substantial compression performance gains. A number of optimization designs are proposed to make the codec complexity feasible while largely maintaining the coding performance.

This paper proposes a multi-layer multi-reference prediction framework for effective video compression. Current AOM/AV1 baseline uses three reference frames for the inter prediction of each video frame. This paper first presents a new coding tool that extends the total number of reference frames in both forward and backward prediction directions. A multi-layer framework is then described, which suggests the encoder design and places different reference frames within one Golden Frame (GF) group to different layers. The multi-layer framework leverages the existing coding tools in the AV1 baseline, including the tool of "show_existing_frame" and the reference frame buffer update module of a wide flexibility. The use of extended ALTREF_FRAMEs is proposed, and multiple ALTREF_FRAME candidates are selected and widely spaced within one GF group. ALTREF_FRAME is a constructed, no-show reference obtained through temporal filtering of a look-ahead frame. In the multi-layer structure, one reference frame may serve different roles for the encoding of different frames through the virtual index manipulation. The experimental results have been collected over several video test sets of various resolutions and characteristics both texture- and motion-wise, which demonstrate that the proposed approach achieves a consistent coding gain compared to the AV1 baseline. For instance, using PSNR as the distortion metric, an average bitrate saving of 5.57+% in BDRate is obtained for the CIF-level resolution set, some of which has a gain of up to 13+%, and 4.47% on average for the VGA-level resolution set, some of which up to 18+%.