Intra block copy (IBC) has been verified as an effective tool for screen content coding. By redefining the reference area for IBC, we show that the coding efficiency can be much improved without increase the on-chip memory requirement in VVC.

Versatile Video Coding (VVC) achieves promising compression performances compared with High Efficiency Video Coding (HEVC) while sacrificing the encoding speed. This paper focuses on predicting the partitioning structures with convolutional neural networks to speedup the VVC encoder. Specifically, we formulate the partitioning prediction problem into two alternatives: implicit partitioning prediction based on the split type of subblock boundaries and explicit partitioning prediction from the ensemble partitioning space. Then, we address both formulations using convolutional neural networks.

In this paper, we concentrate on the super-resolution (SR) of compressed screen content video, in an effort to address the real-world challenges by considering the underlying characteristics of screen content. Firstly, we propose a new dataset for the SR of screen content video with different distortion levels. Meanwhile, we design an efficient SR structure that could capture the characteristics of compressed screen content video and manipulate the inner-connections in consecutive compressed low-resolution frames, facilitating the high-quality recovery of the high-resolution counter-part. Moreover, we design a new loss function for network training to better remedy the compression distortion and perceptual distortion. Experimental results demonstrate the effectiveness and superiority of the proposed method.

In this paper, we focus on computationally modeling of the local texture correlations, in an effort to better explore the coding modes with higher priorities in the rate-distortion optimized intra coding. In particular, strong correlations and continuities of local texture with neighboring blocks have been revealed in our analysis, and empirical justifications provide us inspirations on the joint optimization of rate-distortion-complexity when angular modes become finer to adapt the local textures. We examine the philosophy with extensive experiments conducted for refining the intra full-RD list. The results show that better coding performance with on average 0.72% and 3.00% BD-Rate savings for the natural scene and screen content sequences can be achieved in AVS3 test model HPM-5.0 under all intra configuration, with negligible encoding and decoding time variations.

Palette mode is a powerful tool for screen content coding in the upcoming versatile video coding (VVC) standard. In the palette mode, escape samples are employed to handle an outlier case. In this paper, a quantization parameter (QP) dependent fixed-length binarization is proposed for escape samples coding to 1) simplify the design and 2) improve coding efficiency. The length is calculated according to the QP for the current block and correspondingly, the dequantization process can be also implemented by only left shifting. The proposed method is evaluated with VVC reference software VTM-6.0 on typical sequences containing "text and graphics with motion". Experimental results report that the proposed scheme can achieve up to 2.0% BD-rate savings compared to VTM-6.0 with a much simpler design.

Screen content coding plays an important role in many applications. To meet the growing demands of screen content coding, the emerging AV1 video codec incorporates several coding tools, which are specially designed for screen content utilizing its distinctive characteristics. Among these tools, the intra block copy utilizes the characteristic that repeating patterns frequently occur in screen content. This paper presents the technology of intra block copy in AV1. In particular, to efficiently search the predictor in the reconstructed regions of the current picture, AV1 uses the hash matching method at the encoder side. For the generation of hash table, a bottom-to-up manner is adopted to reduce the redundant computation and then decrease the encoding time. In addition, several constraints are involved to facilitate hardware design. Experimental results demonstrate that the intra block copy in AV1 can bring 27.1% bitrate saving for screen content. When compared with the non hash-based intra block copy, the hash-based method achieves 12.2% bitrate saving.

Traditional intra prediction schemes usually only use the nearest adjacent reference line to generate the prediction. Although the nearest reference line generally has the strongest statistical correlation with current block, the farther non-adjacent reference lines can still provide potential better prediction in some cases. Thus, in this paper, not only the nearest reference line but also the farther reference lines are utilized to help intra prediction. When using the farther reference lines, an additional residue compensation procedure is introduced to further refine the prediction. In particular, this paper designs three solutions to meet different complexity requirements. They are multiple line-based intra prediction (MLIP), fast search for multiple line-based intra prediction (FS-MLIP), and dual line-based intra prediction (DLIP). Experimental results verify the effectiveness of the proposed methods. When compared with HM-16.9, the proposed MLIP achieves 2.4% bit saving on average with the encoding time increasing about 362%. The FS-MLIP achieves 2.0% bit saving on average with the encoding time increasing about 114%. The DLIP achieves 0.9% bit saving on average with the encoding time increasing about only 15%.

Fractional-pel motion compensation is very good at improving video coding efficiency, especially for camera-captured content. But for screen content, which is obtained from a computer desktop, motion vectors with integer-precision may be enough to represent the motion in different pictures. Using fractional-pel motion compensation for such content is a waste of bits. Thus, adaptive motion compensation precision is helpful for improving coding efficiency, especially for screen content coding. Usually, to select suitable motion compensation precision, multi-pass encoding is introduced, which significantly increases the encoding time. This paper presents a fast encoding algorithm for adaptive motion compensation precision used in screen content coding by hash-based block matching. With the proposed method, multi-pass encoding is avoided and most of the benefits brought by adaptive motion compensation precision are preserved. The experimental results show that with the proposed method, up to 7.7% bit saving is obtained without a significant impact on encoding time.

Motion estimation plays an important role in video coding standards, such as H.264/AVC and HEVC. In this paper, we propose a multi-stage hash based motion estimation algorithm for HEVC, which enables hash based motion estimation for natural videos. In the proposed method, the prediction blocks significantly different from the current prediction unit will be eliminated in the motion estimation process. Locality sensitive hashing functions are used to measure the difference between the input block and predicted blocks. The proposed algorithm is implemented into the HM 12.0 software, and the simulation results show that the complexity of motion estimation is significantly reduced with negligible coding performance loss.

Screen contents with complex structure contain random combination of texts, graphics and camera-captured images, which makes them difficult to be compressed efficiently by traditional video codecs. In this paper, we propose a 2-D dictionary based scheme to exploit the repeated patterns on screen content. In the proposed scheme, the current block is predicted from the reconstructed region using a hash-based block searching scheme. A hierarchical two-level hash based searching scheme is designed to find the best matching block for each block. The first-level hash function is used to search the blocks similar to the current block in the constructed 2-D dictionary. The second-level hash function is used to update the 2-D dictionary, which filters out the identical blocks from the blocks found using the first-level hash function. The proposed scheme is incorporated into HEVC framework as an additional mode. Experimental results show that the proposed scheme achieves significantly coding performance improvements on screen contents compared with HEVC.

In lifting-based directional wavelet transforms, different subsampling patterns may show significant difference for directional signals in image coding. This paper investigates the influence of subsampling in directional wavelet transform. We show that the best subsampling depends on the direction and the directionality strength of the signal. To improve the coding performance, we further propose a subsampling-adaptive directional wavelet transform, which can use different subsampling patterns adaptively and according to the local characteristics of the image. To handle the boundary transition when subsampling changes, a phase completion process is applied to ensure that wavelet transform with various subsampling can be performed without introducing boundary effects and performance loss. Experimental results show that the proposed transform can achieve significant coding gain in image coding compared to other existing directional wavelet transforms.

Visual summarization is an attractive new scheme to summarize web pages, which can help achieve a more friendly user experience in search and re-finding tasks by allowing users quickly get the idea of what the web page is about and helping users recall the visited web page. In this paper, we perform a careful study on the recently proposed visual summarization approaches, including the thumbnail of the web page snapshot, the internal image in the web page which is representative of the content in the page, and the visual snippet which is a synthesized image based on the internal image, the title, and the logo found in the web page. Moreover, since the internal image based summarization approach hardly works when the representative internal images are unavailable, we propose a new strategy, which retrieves the representative image from the external to summarize the web page. The experimental results suggest that the various summarization approaches have respective advantages on different types of web pages. While internal images and thumbnails can provide a reliable summarization on web pages with dominant images and web pages with simple structure respectively, the external images are regarded as a useful information to complement the internal images and are demonstrated very useful in helping users understanding new web pages. The visual snippet performs well on the re-finding tasks since it incorporates the title and logo which are advantageous on identifying the visited web pages.

Summary form only given. This paper studies how to better exploit intra correlation to compress images. In general, edge and texture areas of images exhibit strong anisotropic property. The correlation among samples is determined by not only their distance but also the link orientation. Traditional transforms are not efficient on handling this anisotropic correlation. Therefore, in this paper we propose a directional filtering transform (dFT, in order to distinguish from the common usage on DFT) to exploit local anisotropic correlation among samples. Similar to directional prediction in H.264 intra-frame coding, but it adopts the hierarchal structure to decrease the distance between samples to be predicted and that are used for prediction. From another viewpoint, the dFT prediction resembles the directional wavelet transform without update, which can take both intra-block and inter-block correlations into account.

This paper presents a scheme to design directional lapped transforms. Lapped transforms can be factorized into lifting steps. By introducing directional operator into each lifting step, the directional lapped transform is constructed. The directional lapped transform proposed not only preserves the advantages of lapped transforms, it also can represent directional signals more efficiently. An image coding scheme using the directional lapped transform is also described. Compared to the state-of-the-art image coding using lapped transform, HD photo, the proposed scheme shows more than 20 dB's gain for artificial images with strong directional correlations. And for natural images, up to 1.5 dB's gain can also be observed.