Video is often compressed by standard compression algorithms to facilitate storage and transmission. Compressed video will produce artifacts that affect the video quality. How to improve the quality of compressed video during video post-processing has become an important topic in the multimedia field. This paper proposes a Spatio-Temporal Information Fusion Network for quality enhancement of compressed video, as shown in Fig. 1. The algorithm comprises two parts. In the first part, we use 3D convolution to build a U-shaped network to model the temporal dynamics between input frames. We concatenate features with the same spatial resolution from shallow layers to deep layers by skip-connecting merging channels, which helps the local information of shallow- generated features to reach the output. In the second part, we designed a quality enhancement module to fully mine the spatio-temporal information extracted in the first part, cut the feature map on the time dimension t, and then extract the feature map separately on the spatial dimension and refine the feature. The network is trained in an end-to-end manner, and the data sets are selected from the database Xiph (Xiph.org) and VQEG. We use the H.265/HEVC reference software HM16.5 to compress the video to evaluate the performance of the model under different compression levels. The experimental results show that the average PSNR of 18 HEVC standard sequences is improved by 0.88 dB, 0.86 dB, 0.81 dB and 0.72 dB when the quantization parameters(QP) are equal to 37, 32, 27 and 22, respectively, and the number of parameters are only 0.66 million.

Existing block-based encoding frameworks often use inaccurate quantification and motion compensation techniques, which result in many compression artifacts due to the loss of high-frequency information. In particular, the blurring of content edges and significant compression distortion can negatively impact the subjective video quality given limited coding resources. Hence, there is an urgent need to build a quality enhancement method for improving the quality of the compressed video at the receiving end given the same coding resources.

How to obtain higher quality reconstructed video within limited coding resources is a research focus for video coding. First, the fluctuating quality, missing pixel and position fluctuation characteristics of the compressed video are found in this paper based on the block-based coding frameworks. Then, in order to reduce the degradation of compressed video quality caused by the above three characteristics, an non-aligned 3D-CNNs for compressed video quality enhancement called "3D-CVQE" is proposed, which can preserve and utilize limited input information in both temporal and spatial domains effectively. Finally, the experiments results validate the effectiveness and generalization ability of the proposed 3D-CVQE approach in the quality enhancement of compressed video.

View synthesis optimization (VSO) introduces heavy computational complexity caused by the VSO-based iterative search of all possible quad-tree partitions. To reduce the complexity, this paper proposes a convolutional neural network (CNN) scheme based on layer-classification for fast depth intra coding. First, a layer-classification model based on texture smoothness is proposed to determine the smoothest depth map. Then, a CNN network incorporating SENet (CNN-SENet) structure is designed and trained. Finally, the layer-classification model and the CNN-SENet network are combined to predict the coding unit (CU) partition of all coding units (CUs) for depth map at a specific view.

3D-High Efficiency Video Coding (3D-HEVC) is a video compression standard developed for multi-view video plus depth map coding based on the latest HEVC coding standard. We propose an eXtreme Gradient Boosting (XGBoost) system based fast coding unit (CU) level decision for depth maps, which is used to solve the problem of high coding complexity caused by the addition of depth maps and new coding tools in 3D-HEVC. We explore the application of data mining and machine learning in video coding by using texture feature attributes that are highly correlated with CU size. The algorithm is mainly divided into three parts as shown in Figure 1. The algorithm comprises two parts: Models training and fast CU segmentation decision. In the first part, we use data mining and machine learning to construct the decision models by using the texture information of the depth maps as the feature attribute vectors and whether the current CU continues to be divided into sub-CUs as class labels. In the second part, feature attributes were extracted from the coding process, and the trained models were used to decide if the CU continues to partition. Experimental results demonstrated that proposed algorithm yields average 43.52% encoding time reduction with 0.12% BD-rate decrease on V/T and 0.37% increase on S/T under the all intra configuration, compared with the reference software HTM-16.0. In addition, compared with related work, the proposed method achieves different degrees of improvement in coding performance.

As an upgrade for H.264/AVC, high efficiency video coding (HEVC) achieves 50% bitrate reduction under the equivalent visual quality. However, high computational complexity increases dramatically for adopting up to 35 intra prediction modes. To deal with this issue, we explore spatial-temporal correlation between PUs to narrow rough mode decision (RMD) candidate list and rate distortion optimization (RDO) candidate list respectively. A fast PU intra mode decision scheme is proposed for HEVC fast intra encoding. For RMD, the proposed scheme early determines the impossible range of the PU optimal mode in line with spatial statistical analysis theory. It would narrow RMD candidate list and speed up RMD process. First, 4 subsets (Sb1, Sb2, Sb3 and Sb4) are defined in mode set with 33 directional prediction modes as shown in Figure 1. And then, on the basis of spatial statistical analysis theory, the impossible range of the parent PU optimal mode would be judged by its known optimal mode. Last, low probability subsets of current PU are removed, and candidate subsets are determined for current PU. Where ModeParBest and ModeCurBest are the parent PU optimal mode and the current PU optimal mode respectively. For RDO, the proposed scheme tries to combine temporal correlation with intra coding, which adds co-located optimal mode of the previous frame into the RDO list of the current PU. At the same time, the modes in RDO list are decreased focusing on most time-consuming PUs (4 4 PUs, 8 8 PUs) from 8 candidate modes to 3 candidate modes. Experimental results demonstrate that the proposed scheme yields average 29% encoding time reduction with average 1.19% BDBR gain and 0.06dB BDPSNR loss compared with HM16.9. Further, the proposed method implements fast CU encoding without additional computation during the encoding process.

Quadtree brings extremely high computational complexity in high efficiency video coding (HEVC). Innovative works for improving quadtree structures to further reduce encoding time are stated in this paper. A novel quadtree probability mechanism is proposed for HEVC fast coding unit (CU) encoding. Firstly, this paper makes an in-depth study of the relationship among CU distribution, quantization parameter (QP) and video content change. Secondly, a CU quadtree probability model is proposed for modeling and predicting CU partition based on the group of picture (GOP). Eventually, a CU quadtree probability update is proposed, aiming to address probabilistic model distortion problems caused by video content change. Experimental results have shown that the proposed CU quadtree probability mechanism significantly outperforms HEVC by considerably reducing encoding time by 27% for lossy coding and 42% for (visually) lossless coding, without compromising rate-distortion (RD) performance.