Among the recent deep image compression frameworks, transform coding together with a context-adaptive entropy model is the most representative approach to achieve the best coding performance. For entropy model, 2D mask convolution is widely utilized to capture the spatial context, which omits the correlations along channel dimension. To complement to the spatial context, a cross channel context model is proposed. For transform, if given more network layers to improve its representation ability, how to allocate these network layers in forward and inverse transform is investigated. After analyzing the scheme of deep image compression connected with loop filter, we find this investigation can be regarded as a more generalized loop filter. The proposed cross channel context model and generalized loop filter (CCCMGLF) are integrated into the deep image compression framework and jointly optimized to improve the coding performance. Experimental results demonstrate that, using PSNR as distortion metric, the proposed CCCMGLF outperforms VTM-11.0 by 1.20%, 10.82% and 5.38% in terms of BD-rate reductions for Y, U and V components, respectively, for the Kodak dataset. For the JVET CTC sequences, the proposed method outperforms VTM-11.0 by 1.44% for Y but has a coding performance loss of 24.74% and 11.91% for U and V, respectively. Over the baseline deep compression framework, the proposed method provides 7.80%, 12.66% and 11.15% performance improvement for Y, U, and V, respectively, for the Kodak dataset; 9.10%, 12.27%, and 12.68% performance improvement for Y, U and V, respectively, for the JVET CTC sequences. The proposed approaches are applicable in both image compression and intra coding in video compression.

As the latest video coding standard, Versatile Video Coding (VVC) achieves up to 40% Bjontegaard delta bit-rate (BD-rate) reduction compared with High Efficiency Video Coding (HEVC). Recently, Convolutional Neural Network (CNN) has attracted tremendous attention and shows great potential in video coding. In this paper, we design a Multi-Density Convolutional Neural Network (MDCNN) as an integrated in-loop filter to improve the quality of the reconstructed frames. The core of our approach is the multi-density block (MDB), which contains two branches: (a) the basic branch maintaining full resolution for capturing spatially-precise representations, (b) the density branch for learning rich spatial correlation with larger receptive field through down-sampling and upsampling. The feature maps of two branches will be fused into one stream repeatedly. Benefiting from this architecture, spatially-precise representations and density correlations in larger receptive field are utilized to improve the model performance and promote the model's robustness to different input resolutions. Experimental results show that, in terms of BD-rate savings for the (Y, U, V) components compared to the state-of-the-art VVC standard, the proposed MDCNN filter can achieve (5.06%, 13.86%, 13.76%) and (4.36%, 10.85%, 10.91 %) coding gain for the Random Access (RA) configuration and the All Intra (AI) configuration, respectively.

We propose a convolutional neural network-based coefficients prediction (CNNCP) method for intra-predicted residues in the High Efficiency Video Coding (HEVC) standard. In HEVC, discrete cosine transform (DCT) or discrete sine transform (DST) is adopted to convert the intra-predicted residues in the spatial domain into coefficients in the frequency domain. Each coefficient is scalar quantized and entropy coded into the bitstream. As DCT or DST is non-optimal linear transform, there still exist linear and non-linear correlations among different coefficients after the transform. In addition, there exist coefficients' correlations between current block and neighboring blocks, as these correlations cannot be completely exploited in the intra prediction. We thus propose to perform coefficients prediction to further reduce the redundancy among coefficients. The coefficients prediction is achieved using trained convolutional neural networks (CNNs), as CNNs can build complex relationship between input and output by training with a lot of data. In addition, a flag that signals whether to perform coefficients prediction or not at the coding unit level is transmitted to decoder. The proposed CNNCP method is implemented upon the HEVC reference software. Experimental results show that the proposed method achieves on average 1.8%, 4.1%, and 4.5% BD-rate reduction ratios in Y, U, V, respectively, compared with the HEVC baseline in all-intra configuration. In particular, the average BD-rate reduction ratios for 4K test sequences are 2.9%, 6.5%, and 6.6%.