String prediction (SP) is a highly efficient screen content coding technique adopted into international and China video coding standards. However, SP requires a high number of SRAM fetches to decode and output a block for display, leading to low throughput (T). Low T results in a high decoder and SRAM clock frequency to output the required number of display pixels, which is determined by the specific display resolution and frame rate. To achieve hardware-decoder-friendly high throughput SP (HTSP), this paper exploits specific SRAM fetch rate constraints for five SRAM-cell sizes commonly used in hardware decoder designs. Additionally, the optimal reference string selection process is formulated as a multi-constraint rate-distortion optimization (MCRDO) problem and a novel reference string searching method is presented. HTSP boosts throughput by up to 4 times compared to the state-of-the- art SP, with only a negligible impact on coding efficiency.

Screen Content Coding (SCC) is an indispensable tool for enabling distributed collaboration, such as video conferencing. Encoders in the latest video coding standards, particularly for SCC scenarios, employ a wider variety of partitioning tree splitting types, recursively traversing all branches, as well as a larger number of coding modes and submodes, to achieve higher coding efficiency compared to encoders in previous standards. This process leads to very high coding complexity, as each tree leaf node, called a coding unit (CU), for every partitioning size and location in the picture is repeatedly visited and evaluated multiple times during the optimal partitioning search. Additionally, each CU visit involves evaluating a vast number of coding options and their combinations to identify the best one. The complexity is further exacerbated in SCC due to the addition of many new CU coding modes and options. To significantly reduce SCC complexity without coding efficiency loss, this article proposes a new technique, Accelerated Revisit CU-coding (ARC), along with an SCC search space analysis for in-depth operation-level and run/platform-independent assessment of SCC complexity. ARC exploits the correlation between the first visit and subsequent revisits of a CU with the same location and size. By fully leveraging the correlation and information from the first visit, ARC significantly accelerates revisit CU-coding while maintaining the same high coding efficiency. ARC is implemented in HPM, the AVS3 reference software. Experiments demonstrate that ARC reduces encoding runtime by 29.74%, 47.78%, and 54.25% for 1,920 × 1,080 FHD, 4K UHD, and 8K UHD test sequences, respectively, in All Intra configuration, without coding efficiency loss. These runtime reductions align with corresponding search space reductions of 30.91%, 49.67%, and 54.41%, as obtained from the search space analysis.

String prediction (SP) is a very efficient screen content coding (SCC) tool which has been adopted in the third generation of Audio Video Standard (AVS3). It is observed that two special types of strings occur frequently. To further improve the coding efficiency for SCC on top of the original SP, a new variation of SP named Equal-value-string and Copy-above-string based SP (ECSP) is proposed. An ECSP coding unit uses only three types of strings: Equal-value-string, Copy-above-string, and Unpredictable-pixel-string. Compared with the AVS3 reference software HPM9.0 with ECSP disabled, using AVS3 SCC Common Test Condition and YUV 4:2:0 test sequences, the proposed technique achieves an average Y BD-rate reduction of5.54and3.01%for All Intra and Low Delay configurations, respectively, with low additional encoding and decoding complexity. The proposed ECSP has been adopted in the AVS3 standard.

An efficient SCC tool named Intra String Copy (ISC) has been proposed and adopted in AVS3 recently. ISC has two CU-level sub-modes: FPSP (fully-matching-string and partially-matching-string based string prediction) sub-mode and EUSP (equal-value-string, unit-basis-vector-string and unmatched-pixel-string based string prediction) sub-mode. Compared with the latest AVS3 reference software HPM with SCC tools disabled, using AVS3 SCC Common Test Condition and YUV test sequences in text and graphics with motion (TGM) and mixed content (MC) categories, the proposed tool achieves an average Y BD-rate reduction of 57.7%/39.5% and 77.2%/57.9% for TGM and MC in All Intra (AI)/Low Delay B(LDB) configurations, respectively, with low additional encoding complexity and almost the same decoding complexity.

Screen content coding (SCC) is increasingly used in mobile devices, and ultralow coding complexity is required for low power consumption. This article proposes an ultralow complexity string matching approach to SCC. The proposed approach has two essential features: 1) allowing only two types of most effective reference strings that have two most frequently occurring values of offset (i.e., displacement vector) named dual unity offset and 2) fully optimized coding of string matching parameters for maximum coding efficiency. Since at most only two offset values and corresponding reference string positions are allowed for any current string being coded, the optimal reference string searching process is extremely simple and only needs to select the best string from at most two candidates. Moreover, in the proposed approach, the value ranges of string matching parameters, i.e., the string offset vector and string length are very limited, resulting in few bits to code the two parameters and high coding efficiency. Compared with the AVS3 reference software HPM7.0 with IBC disabled, using AVS3 SCC common test condition and YUV test sequences in text and graphics with motion category, the proposed technique achieves Y average BD-rate reduction of 10.8% and 5.8% for all intra (AI) and low-delay B (LDB) configurations, respectively, at ultralow encoding and decoding complexity.

In the past, string prediction (also known as string matching) was applied only to RGB and YUV 4:4:4 format screen content coding. This paper proposes a string prediction approach to 4:2:0 format screen content coding implemented in the third generation of Audio Video Standard (AVS3) in China. String prediction is applied to both YUV CU and Y CU. To further improve the coding performance, several improved technicals of string prediction are presented, including a mixed string searching strategy for finding the optimal reference string, a joint picture-level, CU-level, and pixel-level early termination strategy to reduce coding complexity, and two effective coding methods for string prediction parameters. For low-complexity hardware implementation of string prediction decoder, the memory access bandwidth is reduced by introducing string constraints. Meanwhile, string prediction reuses the reference pixel buffer of intra block copy (IBC). Compared with the newest AVS3 reference software HPM7.0 with string prediction disabled, the proposed string prediction approach achieves up to 18.48% Y BD-rate reduction. Using AVS3 Screen Content Coding (SCC) Common Test Condition and YUV test sequences in Text and Graphics with Motion category, the proposed technique achieves an average Y BD-rate reduction of 10.33%, 8.47%, 6.91% for All Intra (AI), Random Access (RA) and Low Delay (LD) configurations, respectively, with low additional encoding and decoding complexity. The proposed string prediction approach has been adopted in the newest AVS3 reference software HPM7.0.

Alpha channel is being applied in an increasing number of mobile web applications on mobile devices that require ultra-low power consumption in all cases including compute-intensive video encoding and decoding. Thus, we propose an ultra-low coding complexity and high efficiency alpha channel lossless coding approach. A novel coding framework and four new coding schemes are proposed for alpha channel coding. The framework fuses a string matching technique and a proposed prediction coding scheme named bit-depth preserving prediction (BDPP) together to reduce the correlations within and between repeated identical patterns and neighboring pixels. To achieve a good tradeoff between complexity and efficiency, either the unmatchable bytes are coded directly or the BDPP residuals of unmatchable bytes are coded by a proposed bytewise entropy coding scheme named 0.5-1-2byte-size-code. The other string matching parameters are coded by another proposed bytewise entropy coding scheme named byte-size multi-variable-length-code. To speed up the string-matching search, we apply a fast string search scheme that combines special position search and hash-based search. For the selected typical 236 alpha test images, compared with x265 in the fastest configuration and lossless mode, the proposed lossless approach achieves 14.33% less total compressed bytes with only 2.75% encoding and 1.83% decoding runtime. The proposed approach also outperforms the conventional lossless coding techniques such as LZ4HC, ZLIB, and PNG.

This paper proposes a universal string matching (USM) approach to screen content coding (SCC). USM uses a primary reference buffer and a secondary reference buffer for string matching and includes three modes: general string (GS) mode, constrained string 1 (CS1) mode, and constrained string 2 (CS2) mode. The CS1 mode and CS2 mode are constrained cases of the GS mode. Due to the diversity of the screen content, each of the three modes plays an indispensable role in coding some types of screen content.When using USM to code a coding unit (CU), one of the three modes is selected to code the CU. Compared with high-efficiency video coding (HEVC) SCC reference software HM-16.6 + SCM-5.2 of full frame search range for intrablock copy, USM achieves an average Y BD-rate of -28.4% for five text and graphics with motion (TGM) sequences from the audio video coding standard SCC common test condition (CTC) test suite and -5.8% for eight TGM test sequences from the HEVC SCC CTC test suite in all intraconfigurations, with a nearly 10% decrease in encoding runtime and almost the same decoding runtime.

This paper proposes a pseudo 2D string matching (P2SM) technique for high efficiency screen content coding (SCC). The technique uses a primary reference buffer (PRB) and a secondary reference buffer (SRB) for string matching and string copying. In the encoder, optimal reference string searching is performed in both PRB and SRB, and either a PRB or an SRB string is selected as an optimal reference string on a string-by-string basis. If no reference string of at least one pixel is founded for a current pixel, then the current pixel is coded as an unmatched pixel. Compared with HM-16.4${+}$SCM-4.0 reference software, the proposed P2SM technique achieves up to 37.7% Y BD-rate reduction for a screen snapshot of a spreadsheet. On average, using HEVC SCC common test condition and YUV test sequences in text and graphics with motion category, the proposed technique achieves Y BD-rate reduction of 7.7%, 5.0%, 2.6% for all intra (AI), random access (RA) and low-delay B (LB) configurations, respectively in lossy coding with both intra block copy (IBC) and P2SM having the same 4 coding tree units (CTUs) searching range, and bit-rate saving of 6.0%, 3.9%, 3.1% for AI, RA, LB configurations, respectively in lossless coding with IBC having full frame searching range while P2SM having only 2 CTUs searching range, at very low additional encoding and decoding complexity.

In this paper, we present an arbitrary shape matching (ASM) coding technique for screen contents. In ASM coder, a Coding Unit (CU) is broken into multiple pixel sample strings. Each string called matched string in the CU has a matching string in the previously coded and reconstructed pixel buffer. Then the distance (either 1D or 2D) between the matching string and matched string and the length of the string are entropy-coded into the bitstream buffer. If the length is zero (no matching is found), then the original pixel sample is entropy-coded into the bitstream buffer. Experiments show that for some types of screen contents, ASM can significantly improve coding performance.

This study proposes a unified LZ and hybrid coding (ULHC) method for compound image and video compression of visually lossless quality and high compression ratio. The method is macroblock‐based for ultra‐low coding latency and compatibility with the conventional and most popular hybrid video coding standards such as H.264 and MPEG‐2. First, each macroblock is coded by two tools: (i) gzip, a popular lossless LZ coding tool, modified to be macroblock‐oriented and to be seamlessly unifiable with a lossy hybrid coding tool; (ii) H.264, an advanced lossy hybrid coding tool. Then rate‐distortion optimisation is used to select either the modified gzip or H.264 as the final coding. To seamlessly unify these two coding tools for maximum quality and high compression ratio in ULHC, the modified gzip uses the most recent reconstructed and specially serialised macroblock data as the dictionary. Experimental results show that for images and videos composed of natural or synthesised picture, text and graphics, the proposed method provides higher peak signal‐to‐noise ratio and better subjective quality than H.264 at the same bitrate, and also achieves much higher compression ratio than gzip without any visual quality loss. In fact, ULHC can achieve partial‐lossless and partial‐near‐lossless coding with a high compression ratio.

Computer screens contain discontinuous-tone content and continuous-tone content. Thus, the most effective way for screen content coding (SCC) is to use two essentially different coders: a dictionary-entropy coder and a traditional hybrid coder. Although screen content is originally in a full-chroma (e.g., YUV444) format, the current method of compression is to first subsample chroma of pictures and then compress pictures using a chroma-subsampled (e.g., YUV420) coder. Using two chroma-subsampled coders cannot achieve high-quality SCC, but using two full-chroma coders is overkill and inefficient for SCC. To solve the dilemma, this paper proposes a mixed chroma sampling-rate approach for SCC. An original full-chroma input macroblock (coding unit) or its prediction residual is chroma-subsampled. One full-chroma base coder and one chroma-subsampled base coder are used simultaneously to code the original and the chroma-subsampled macroblock, respectively. The coder minimizing rate-distortion (R-D) is selected as the final coder for the macroblock. The two base coders are coherently unified and optimized to get the best overall coding performance and share coding components and resources as much as possible. The approach achieves very high visual quality with minimal computing complexity increment for SCC, and has better R-D performance than two full-chroma coders approach, especially in low bitrate.

This paper proposes a computing paradigm where many users share a host platform consisting of one or more multicore CPU(s) and GPU(s). Each user is connected to the host by a link transferring primarily compressed compound video of screen, mouse and keyboard data. All computation and processing tasks including but not limited to 2D/3D graphics and multimedia processing are done on the host using a traditional or existing architecture. Since all computing and storage resources are centralized on the "cloudlet" (host) and each user site has only a high-resolution monitor, a mouse, and a keyboard with some optional I/O devices, it is a highly energy-efficient, secure, reliable, and low total-ownership-cost paradigm. The compound video link between the host and the user is partial-lossless (partially lossless and partially near-lossless), and has ultra-low latency, so the proposed paradigm still has no-compromised 2D/3D graphics and multimedia performance demanded by the mass market.