String prediction (SP) is a highly efficient screen content coding technique adopted into international and China video coding standards. However, SP requires a high number of SRAM fetches to decode and output a block for display, leading to low throughput (T). Low T results in a high decoder and SRAM clock frequency to output the required number of display pixels, which is determined by the specific display resolution and frame rate. To achieve hardware-decoder-friendly high throughput SP (HTSP), this paper exploits specific SRAM fetch rate constraints for five SRAM-cell sizes commonly used in hardware decoder designs. Additionally, the optimal reference string selection process is formulated as a multi-constraint rate-distortion optimization (MCRDO) problem and a novel reference string searching method is presented. HTSP boosts throughput by up to 4 times compared to the state-of-the- art SP, with only a negligible impact on coding efficiency.

Screen Content Coding (SCC) is an indispensable tool for enabling distributed collaboration, such as video conferencing. Encoders in the latest video coding standards, particularly for SCC scenarios, employ a wider variety of partitioning tree splitting types, recursively traversing all branches, as well as a larger number of coding modes and submodes, to achieve higher coding efficiency compared to encoders in previous standards. This process leads to very high coding complexity, as each tree leaf node, called a coding unit (CU), for every partitioning size and location in the picture is repeatedly visited and evaluated multiple times during the optimal partitioning search. Additionally, each CU visit involves evaluating a vast number of coding options and their combinations to identify the best one. The complexity is further exacerbated in SCC due to the addition of many new CU coding modes and options. To significantly reduce SCC complexity without coding efficiency loss, this article proposes a new technique, Accelerated Revisit CU-coding (ARC), along with an SCC search space analysis for in-depth operation-level and run/platform-independent assessment of SCC complexity. ARC exploits the correlation between the first visit and subsequent revisits of a CU with the same location and size. By fully leveraging the correlation and information from the first visit, ARC significantly accelerates revisit CU-coding while maintaining the same high coding efficiency. ARC is implemented in HPM, the AVS3 reference software. Experiments demonstrate that ARC reduces encoding runtime by 29.74%, 47.78%, and 54.25% for 1,920 × 1,080 FHD, 4K UHD, and 8K UHD test sequences, respectively, in All Intra configuration, without coding efficiency loss. These runtime reductions align with corresponding search space reductions of 30.91%, 49.67%, and 54.41%, as obtained from the search space analysis.

Owing to the fast developments of economics, a lot of devices and objects have been connected and have formed the Internet of Things (IoT). Visual sensors have been applied in vehicle navigation, traffic situational awareness, and traffic safety management. However, the particles in the air degrade the imaging quality, which affects the performance of vehicle navigation, traffic situational awareness, and traffic safety management. Deep-learning-based dehazing methods were proposed to address this issue. However, these methods are trained with simulated hazy images and cannot generalize to natural haze images well. To address the domain shift problem, some methods resort to zero-shot learning or domain adaption to boost the generalization of the model on natural haze images. However, the relevance between dehazed results and clean images is ignored by zero-shot dehazing methods. Domain-adaption-based dehazing methods ignore the relationship between the dehazed results and the hazy images. To overcome these issues, a generative adversarial and self-supervised dehazing network is introduced to boost the dehazing performance on real haze images. First, generative adversarial is employed to construct the relevance between dehazed results and haze-free images, which can boost the natural appearance of dehazed results. Second, self-supervised learning is employed to construct the relevance between the dehazed results and hazy images, which can restrict the solution space of dehazing. To show the effectiveness of the proposed model, we conduct extensive experiments on real and simulated haze images. Compared with state-of-the-art methods, the proposed model achieves state-of-the-art dehazing performance.

String prediction (SP) is a very efficient screen content coding (SCC) tool which has been adopted in the third generation of Audio Video Standard (AVS3). It is observed that two special types of strings occur frequently. To further improve the coding efficiency for SCC on top of the original SP, a new variation of SP named Equal-value-string and Copy-above-string based SP (ECSP) is proposed. An ECSP coding unit uses only three types of strings: Equal-value-string, Copy-above-string, and Unpredictable-pixel-string. Compared with the AVS3 reference software HPM9.0 with ECSP disabled, using AVS3 SCC Common Test Condition and YUV 4:2:0 test sequences, the proposed technique achieves an average Y BD-rate reduction of5.54and3.01%for All Intra and Low Delay configurations, respectively, with low additional encoding and decoding complexity. The proposed ECSP has been adopted in the AVS3 standard.

An efficient SCC tool named Intra String Copy (ISC) has been proposed and adopted in AVS3 recently. ISC has two CU-level sub-modes: FPSP (fully-matching-string and partially-matching-string based string prediction) sub-mode and EUSP (equal-value-string, unit-basis-vector-string and unmatched-pixel-string based string prediction) sub-mode. Compared with the latest AVS3 reference software HPM with SCC tools disabled, using AVS3 SCC Common Test Condition and YUV test sequences in text and graphics with motion (TGM) and mixed content (MC) categories, the proposed tool achieves an average Y BD-rate reduction of 57.7%/39.5% and 77.2%/57.9% for TGM and MC in All Intra (AI)/Low Delay B(LDB) configurations, respectively, with low additional encoding complexity and almost the same decoding complexity.

Screen content coding (SCC) is increasingly used in mobile devices, and ultralow coding complexity is required for low power consumption. This article proposes an ultralow complexity string matching approach to SCC. The proposed approach has two essential features: 1) allowing only two types of most effective reference strings that have two most frequently occurring values of offset (i.e., displacement vector) named dual unity offset and 2) fully optimized coding of string matching parameters for maximum coding efficiency. Since at most only two offset values and corresponding reference string positions are allowed for any current string being coded, the optimal reference string searching process is extremely simple and only needs to select the best string from at most two candidates. Moreover, in the proposed approach, the value ranges of string matching parameters, i.e., the string offset vector and string length are very limited, resulting in few bits to code the two parameters and high coding efficiency. Compared with the AVS3 reference software HPM7.0 with IBC disabled, using AVS3 SCC common test condition and YUV test sequences in text and graphics with motion category, the proposed technique achieves Y average BD-rate reduction of 10.8% and 5.8% for all intra (AI) and low-delay B (LDB) configurations, respectively, at ultralow encoding and decoding complexity.

We consider visual tracking in numerous applications of computer vision and seek to achieve optimal tracking accuracy and robustness based on various evaluation criteria for applications in intelligent monitoring during disaster recovery activities. We propose a novel framework to integrate a Kalman filter (KF) with spatial-temporal regularized correlation filters (STRCF) for visual tracking to overcome the instability problem due to large-scale application variation. To solve the problem of target loss caused by sudden acceleration and steering, we present a stride length control method to limit the maximum amplitude of the output state of the framework, which provides a reasonable constraint based on the laws of motion of objects in real-world scenarios. Moreover, we analyze the attributes influencing the performance of the proposed framework in large-scale experiments. The experimental results illustrate that the proposed framework outperforms STRCF on OTB-2013, OTB-2015 and Temple-Color datasets for some specific attributes and achieves optimal visual tracking for computer vision. Compared with STRCF, our framework achieves AUC gains of 2.8%, 2%, 1.8%, 1.3%, and 2.4% for the background clutter, illumination variation, occlusion, out-of-plane rotation, and out-of-view attributes on the OTB-2015 datasets, respectively. For sporting events, our framework presents much better performance and greater robustness than its competitors.

In the past, string prediction (also known as string matching) was applied only to RGB and YUV 4:4:4 format screen content coding. This paper proposes a string prediction approach to 4:2:0 format screen content coding implemented in the third generation of Audio Video Standard (AVS3) in China. String prediction is applied to both YUV CU and Y CU. To further improve the coding performance, several improved technicals of string prediction are presented, including a mixed string searching strategy for finding the optimal reference string, a joint picture-level, CU-level, and pixel-level early termination strategy to reduce coding complexity, and two effective coding methods for string prediction parameters. For low-complexity hardware implementation of string prediction decoder, the memory access bandwidth is reduced by introducing string constraints. Meanwhile, string prediction reuses the reference pixel buffer of intra block copy (IBC). Compared with the newest AVS3 reference software HPM7.0 with string prediction disabled, the proposed string prediction approach achieves up to 18.48% Y BD-rate reduction. Using AVS3 Screen Content Coding (SCC) Common Test Condition and YUV test sequences in Text and Graphics with Motion category, the proposed technique achieves an average Y BD-rate reduction of 10.33%, 8.47%, 6.91% for All Intra (AI), Random Access (RA) and Low Delay (LD) configurations, respectively, with low additional encoding and decoding complexity. The proposed string prediction approach has been adopted in the newest AVS3 reference software HPM7.0.

Alpha channel is being applied in an increasing number of mobile web applications on mobile devices that require ultra-low power consumption in all cases including compute-intensive video encoding and decoding. Thus, we propose an ultra-low coding complexity and high efficiency alpha channel lossless coding approach. A novel coding framework and four new coding schemes are proposed for alpha channel coding. The framework fuses a string matching technique and a proposed prediction coding scheme named bit-depth preserving prediction (BDPP) together to reduce the correlations within and between repeated identical patterns and neighboring pixels. To achieve a good tradeoff between complexity and efficiency, either the unmatchable bytes are coded directly or the BDPP residuals of unmatchable bytes are coded by a proposed bytewise entropy coding scheme named 0.5-1-2byte-size-code. The other string matching parameters are coded by another proposed bytewise entropy coding scheme named byte-size multi-variable-length-code. To speed up the string-matching search, we apply a fast string search scheme that combines special position search and hash-based search. For the selected typical 236 alpha test images, compared with x265 in the fastest configuration and lossless mode, the proposed lossless approach achieves 14.33% less total compressed bytes with only 2.75% encoding and 1.83% decoding runtime. The proposed approach also outperforms the conventional lossless coding techniques such as LZ4HC, ZLIB, and PNG.

This paper proposes a universal string matching (USM) approach to screen content coding (SCC). USM uses a primary reference buffer and a secondary reference buffer for string matching and includes three modes: general string (GS) mode, constrained string 1 (CS1) mode, and constrained string 2 (CS2) mode. The CS1 mode and CS2 mode are constrained cases of the GS mode. Due to the diversity of the screen content, each of the three modes plays an indispensable role in coding some types of screen content.When using USM to code a coding unit (CU), one of the three modes is selected to code the CU. Compared with high-efficiency video coding (HEVC) SCC reference software HM-16.6 + SCM-5.2 of full frame search range for intrablock copy, USM achieves an average Y BD-rate of -28.4% for five text and graphics with motion (TGM) sequences from the audio video coding standard SCC common test condition (CTC) test suite and -5.8% for eight TGM test sequences from the HEVC SCC CTC test suite in all intraconfigurations, with a nearly 10% decrease in encoding runtime and almost the same decoding runtime.

This paper proposes a pseudo 2D string matching (P2SM) technique for high efficiency screen content coding (SCC). The technique uses a primary reference buffer (PRB) and a secondary reference buffer (SRB) for string matching and string copying. In the encoder, optimal reference string searching is performed in both PRB and SRB, and either a PRB or an SRB string is selected as an optimal reference string on a string-by-string basis. If no reference string of at least one pixel is founded for a current pixel, then the current pixel is coded as an unmatched pixel. Compared with HM-16.4${+}$SCM-4.0 reference software, the proposed P2SM technique achieves up to 37.7% Y BD-rate reduction for a screen snapshot of a spreadsheet. On average, using HEVC SCC common test condition and YUV test sequences in text and graphics with motion category, the proposed technique achieves Y BD-rate reduction of 7.7%, 5.0%, 2.6% for all intra (AI), random access (RA) and low-delay B (LB) configurations, respectively in lossy coding with both intra block copy (IBC) and P2SM having the same 4 coding tree units (CTUs) searching range, and bit-rate saving of 6.0%, 3.9%, 3.1% for AI, RA, LB configurations, respectively in lossless coding with IBC having full frame searching range while P2SM having only 2 CTUs searching range, at very low additional encoding and decoding complexity.