The Versatile Video Coding (VVC) standard introduces a block partitioning structure known as quadtree plus nested multi-type tree (QTMTT), which allows more flexible block partitioning compared to its predecessors, like High Efficiency Video Coding (HEVC). Meanwhile, the partition search (PS) process, which is to find out the best partitioning structure for optimizing the rate-distortion cost, becomes far more complicated for VVC than for HEVC. Also, the PS process in VVC reference software (VTM) is not friendly to hardware implementation. We propose a partition map prediction method for fast block partitioning in VVC intra-frame encoding. The proposed method may replace PS totally or be combined with PS partially, thereby achieving adjustable acceleration of the VTM intra-frame encoding. Different from the previous methods for fast block partitioning, we propose to represent a QTMTT-based block partitioning structure by a partition map, which consists of a quadtree (QT) depth map, several multi-type tree (MTT) depth maps, and several MTT direction maps. We then propose to predict the optimal partition map from the pixels through a convolutional neural network (CNN). We propose a CNN structure, known as Down-Up-CNN, for the partition map prediction, where the CNN structure emulates the recursive nature of the PS process. Moreover, we design a post-processing algorithm to adjust the network output partition map, so as to obtain a standard-compliant block partitioning structure. The post-processing algorithm may produce a partial partition tree as well; then based on the partial partition tree, the PS process is performed to obtain the full tree. Experimental results show that the proposed method achieves 1.61× to 8.64× encoding acceleration for the VTM-10.0 intra-frame encoder, with the ratio depending on how much PS is performed. Especially, when achieving 3.89× encoding acceleration, the compression efficiency loss is 2.77% in BD-rate, which is a better tradeoff than the previous methods.

High Efficiency Video Coding (HEVC) achieves significant improvement in compression efficiency by introducing quadtree-based block partition. However, in the HEVC reference software–HM, the optimal partition is found by a recursive rate-distortion optimization (RDO) process, which is computationally expensive and not friendly to hardware implementation. We propose a fast block partitioning algorithm using convolutional neural network (CNN) based depth map prediction for HEVC intra coding. We use the depth map to represent the block partition of a coding tree unit (CTU). Then, we design a CNN to predict the depth map for a CTU, and we construct a large-scale dataset to train the CNN. Through the depth map prediction, we obtain a block partitioning structure for the entire CTU, and then we could directly compress each coding unit, getting rid of the recursive RDO process for partitioning. Experimental results show that our proposed method reduces 65.55% encoding time of HM at the cost of 2.02% Bjøntegaard Delta rate (BD-rate) increase on the common test sequences. For 4K sequences, our method achieves 76.97% time saving with 2.89% BD-rate increase.