As the latest international video coding standard finalized in July 2020, the Versatile Video Coding (VVC) standard achieves around 50% bitrate savings at the same video quality over its predecessor, the High Efficiency Video Coding (HEVC) standard. The improvement of coding efficiency brings great value with the rapid growth of video applications. However, increased coding efficiency also comes at the cost of increased encoding complexity, which makes it rather difficult to apply VVC to real-time video services using general-purpose processors. In this paper, an optimized VVC software encoder implementation towards real-time applications, named Ali266, is introduced, including coding efficiency optimization and computational complexity reduction algorithms in Ali266. Our experiments demonstrate that Ali266 with the “faster” preset enables real-time encoding for 1080p video-on-demand (VOD) content, while providing more than 40% bitrate savings over x265 at the medium preset with the same objective video quality.

To achieve higher coding efficiency, the latest Versatile Video Coding (VVC) standard adopts a series of new intra coding techniques, including the quadtree plus multi-type tree (QTMT), intra sub-partitions (ISP) and intra block copy (IBC). However, this makes the intra coding more complicated, as VVC needs to traverse all prediction modes and partition types of QTMT to find the optimal combination. In this paper, we propose a fast algorithm for VVC from two aspects of mode selection and prediction terminating to reduce coding complexity. For the mode selection, adaptive mode pruning (AMP) is proposed to remove non-promising modes. First, since the newly introduced modes (IBC and ISP) are not effective for all blocks, learning-based classifiers are designed to remove them intelligently. Second, for normal modes, an ensemble decision strategy is proposed to sort the candidate modes and increase the probability of being the optimal mode for the first few candidates; thus, we can remove redundant candidates more efficiently. In terms of prediction terminating, we find that different optimal modes of current depth level lead to different termination probabilities of remaining intra predictions. Therefore, mode-dependent termination (MDT) is proposed to select an appropriate model through the optimal mode and terminate unnecessary intra predictions of remaining depth levels. The proposed algorithm is implemented on VVC test model, and simulation results show that it can achieve 51%$\sim$53% time savings with only 0.93%$\sim$1.08% BDBR increases.

For more efficient inter prediction in quality scalable high efficiency video coding (SHVC), a learning-based framework for generating a virtual reference frame (VRF) is proposed in this letter. In our method, reconstructed base layer (BL) and enhancement layer (EL) frames are employed to make the generated VRF and current EL frame as same as possible. To this end, this letter proposes a novel VRF generation convolutional neural network (VRFCNN) to jointly handle enhancement of corresponding BL frame and compensation of previous EL frame. Specifically, the VRFCNN consists of BL enhancement, EL compensation and feature fusion subnets. Previous EL frames are firstly compensated with the learned coarse flow between two adjacent BL frames and then adopted to provide interlayer information for corresponding BL enhancement. The learned finer flow between the EL and enhanced BL features is adopted to provide temporal information for previous EL compensation. For efficiently handling slow- and fast-motion videos, the enhanced BL and compensated EL features are fused to generate a VRF. Experimental results show that VRFCNN averagely achieves 11.8% BD-rate reduction under low delay P configuration, which outperforms other methods. The code of our VRFCNN approach is available at https://github.com/dq0309/VRFCNN.

Quadtree with nested multi-type tree (QTMT) partition structure is an efficient improvement in versatile video coding (VVC) over the quadtree (QT) structure in the advanced high-efficiency video coding (HEVC) standard. With the exception of the recursive QT partition structure, recursive multi-type tree partition is applied to each leaf node, which generates more flexible block sizes. Besides, intra prediction modes are extended from 35 to 67 so as to satisfy various texture patterns. These newly developed techniques achieve high coding efficiency but also result in very high computational complexity. To tackle this problem, we propose a fast intra-coding algorithm consisting of low-complexity coding tree units (CTU) structure decision and fast intra mode decision in this paper. The contributions of the proposed algorithm lie in the following aspects: 1) the new block size and coding mode distribution features are first explored for a reasonable fast coding scheme; 2) a novel fast QTMT partition decision framework is developed, which can determine the partition decision on both QT and multi-type tree with a novel cascade decision structure; and 3) fast intra mode decision with gradient descent search is introduced, while the best initial search point and search step are also investigated in this paper. The simulation results show that the complexity reduction of the proposed algorithm is up to 70% compared to VVC reference software (VTM), and averagely 63% encoding time saving is achieved with 1.93% BDBR increasing. Such results demonstrate that our method yields a superior performance in terms of computational complexity and compression quality compared to the state-of-the-art methods.