Text screen content images (TSCIs) have been extensively applied in multimedia applications. When a TSCI is compressed by an encoder, the reconstructed image often exhibits changes of text stroke structures, leading to a novel and intriguing distortion, namely character transformation artifact (CTA). Specifically, CTA makes the original characters being transformed into other characters with varying degrees, causing inaccurate or even misleading perception of text semantic information, thereby affecting the quality of TSCIs. This paper systematically focuses on the quantitative representation of CTA for the first time, taking the advanced Versatile Video Coding (H.266/VVC) standard and English text as examples. First, the perceptual forms of CTA and the variation of CTA perceptual degree (CTA-PD) with the quantization parameter (QP) of H.266/VVC are explored. Second, a method for predicting the CTA-PD, referred to as P-CTA-PD, is formulated by extracting and quantifying the stroke features of distorted characters. Finally, P-CTA-PD is further applied to form a TSCI quality assessment method, namely CTA-based image quality assessment (CTA-IQA). Experimental results demonstrate that P-CTA-PD can achieve high accuracy in predicting perceptual degree of CTA, with an accuracy of 95.64%. Meanwhile, CTA-IQA scores show a stronger correlation with the mean opinion score compared to state-of-the-art quality assessment methods. This paper lays the groundwork for CTA, providing fundamental support for future exploration on CTA for other video coding standards, text types, and application scenarios.

With the rapid development of multimedia applications such as online education, remote conferences, and telemedicine, an emerging type of image known as the text screen content image (TSCI) has gained widespread utilization. Distinguishing from natural images captured by cameras, TSCI is generally generated or rendered by computers and exhibits significant differences in content characteristics. Notably, TSCI primarily comprises text, which is a symbol system uniquely defined by humans with specific semantics. As an important carrier for transmitting semantic information, the quality of text in TSCI significantly affects the subjective perception experience of multimedia system users. Just noticeable difference (JND) is a widely studied image quality measure that is theoretically closest to human perception. However, the traditional JND (T-JND) experiments fail to distinguish text from other image contents, ignoring the significant impact of text semantic readability on image quality. This paper, for the first time, focuses on the impact of text semantics on the quality of TSCI, and JND experiments for TSCIs compressed by the state-of-the-art versatile video coding (VVC) standard are explored and discussed. Specifically, a matching TSCI dataset is first established. Using the dataset, image subjective observation experiments are further designed and carried out to construct the traditional JND (T-JND) experiment as well as the semantic aware JND (S-JND) experiment. By comparing the experimental results, crucial conclusions are reached, including the fact that the S-JND experiment provides a more precise description of the TSCI quality compared to the T-JND experiment. These conclusions have important guiding significance for the subsequent development of efficient JND models suitable for TSCIs compressed by VVC.

Surveillance videos have been widely used in criminal investigation, judicial evidence collection, and other fields. The authenticity of surveillance video is a key prerequisite for whether the surveillance video is used legally. Video recompression forensics (VRF) is a widely recognized technique for video authenticity authentication. And basic bitstream feature (BBF) extraction is the primary step in VRF, which significantly affects the accuracy of VRF. The current VRF methods have not systematically focused on the different impact of different types of BBF on VRF. Most VRF methods directly extract the BBF from video bitstream to construct the advanced bitstream features, which implies that all BBF have an equal impact on the accuracy of VRF. This operation is obviously unreasonable. This paper focuses on VVC encoded surveillance videos, and an experimental scheme is first designed to evaluate the effectiveness of BBF. Based on the experimental data, the impact of different types of BBF on the accuracy of VRF is further analyzed and evaluated, and some important conclusions are drawn. These conclusions have important guiding significance for the subsequent development of efficient VRF methods suitable for surveillance videos using VVC.

Video surveillance and its applications have become increasingly ubiquitous in modern daily life. In video surveillance system, video coding as a critical enabling technology determines the effective transmission and storage of surveillance videos. In order to meet the real-time or time-critical transmission requirements of video surveillance systems, the low-delay (LD) configuration of the advanced high efficiency video coding (HEVC) standard is usually used to encode surveillance videos. The coding efficiency of the LD configuration is closely related to the quantization parameter (QP) cascading technique which selects or determines the QPs for encoding. However, the quantization parameter cascading (QPC) technique currently adopted for the LD configuration in HEVC test model (i.e., HM) is not optimized since it has not taken full account of the reference dependency in coding. In this paper, an efficient QPC technique for surveillance video coding, referred to as QPC-SV, is proposed, considering all inter reference frames under the LD configuration. Experimental results demonstrate the efficacy of the proposed QPC-SV. Compared with the default configuration of QPC in the HM, the QPC-SV achieves significant rate-distortion performance gain with average BD-rates of -9.35% and -9.76% for the LDP and LDB configurations, respectively.

Rate control is a key technique for video communication systems. The aim of rate control is to transmit the best possible quality video sequences under various restrictions, such as channel bandwidth, buffer capacity, maximum time delay allowed for a given service, and so on. The λ domain rate control technique (λ-RC) has been integrated into the latest High Efficiency Video Coding standard (H.265/HEVC) test model, due to its accurate bit estimation and high rate-distortion performance. However, it is found that the λ-RC is not the optimal choice under the random-access configuration. When the random-access configuration is used, pictures are organized into temporal layers, where pictures in different layers are of different importance in terms of prediction. In this paper, a picture level lambda domain rate control technique for the randomaccess configuration in H.265/HEVC is proposed. The influence of temporal layers is effectively considered in the proposed algorithm referred to as TL-λ-PRC. Experimental results verify that the proposed TL-λ-PRC is efficient in coding performance and accurate in bit estimation. Compared with the λ-RC with the fixed ration bit allocation which has been implemented in the test model of H.265/HEVC (HM 14.0), TL-λ-PRC achieves an average reduction of 4.10% and 3.49% for slow motion and fast motion sequences, respectively, in BD-rate (Bjøntegaard-Delta bitrate) with more accurate bit estimation. The performance of different algorithms in terms of algorithm complexity and quality fluctuation are also carefully analyzed in this contribution.

Currently, as one of the most important video coding structures, the low-delay (LD) configuration has been widely adopted in the video communication systems, such as video surveillance and video conferencing systems. The rate-distortion optimization (RDO) is also a crucial technique for improving the performance of video coding. The RDO performance of H.265/HEVC with the LD configuration is significantly affected by the temporal dependence. This paper examines the dependence of rate and distortion among temporal layers in H.265/HEVC with the LD configuration. An efficient RDO method is then proposed based on temporal dependence, where the optimized Lagrangian multipliers are estimated by adaptively selecting the scaling factors according to the video motion characteristics pertaining to different temporal layers. The experimental results show that compared with the RDO method adopted in the H.265/HEVC test model (i.e., HM), the proposed RDO method can achieve the average BD-rate reductions of 6.4% and 5.8% for the LD P and LD B configurations, respectively.

In high-efficiency coding (HEVC), the random-access structure (RAS) is employed due to its high coding efficiency and “random-access” performance. Pictures in RAS are assigned to different temporal layers. And how to select the QP for each temporal layer in quantization parameter cascading (QPC) technique is critical for improving the coding efficiency of RAS. In order to further improve the coding efficiency of RAS, a QPC technique considering the video content characteristics (denoted as VC-QPC for short) is proposed. In VC-QPC, motion and texture complexities of a video are used for predicting the optimal quantization parameter values of temporal layers. Compared with the method in the test model of HEVC, i.e., HM14.0, the BD-rate of proposed VC-QPC is -5.60% for RAS.

The random-access configuration is employed in H.265/HEVC video coding to make inter-frame prediction more efficient. The coding efficiency under the random-access configuration is closely related to the quantization parameter cascading (QPC) technique which determines the QP for encoding pictures in different temporal layers. The present QPC technique for the random-access configuration in H.265/HEVC is not optimized. In this paper, first, a rate-distortion-optimization-based technique for QPC, referred to as RDO-QPC, is proposed for the random-access configuration in H.265/HEVC. Based on the results from RDO-QPC, a simplified QPC technique, referred to as SRDO-QPC, is proposed. The experimental results verify the efficiency of the proposed two QPC techniques. Compared with the original configuration of QPC in H.265/HEVC, RDO-QPC achieves an average gain of 0.19 dB in APSNR and an average reduction of 4.87% in ABR, while SRDO-QPC achieves an average gain of 0.17 dB in APSNR and an average reduction of 4.32% in ABR. These two QPC techniques can both be used in practical situations, meeting different requirements of computational complexity.

In H.265/HEVC, the random-access prediction structure helps to increase the coding efficiency and provides the temporal scalability. However, the quantization parameter cascading (QPC) strategy being used is not optimized in terms of the rate-distortion performance. This paper analyzes the rate and distortion dependency between temporal layers, and proposes a PSNR metric-based distortion model and a piecewise rate model to optimize the QPC. Experimental results demonstrate that the proposed algorithm achieves an average APSNR gain of 0.12dB with the maximum APSNR gain being 0.37dB. Meanwhile, an up to 8.7% BD-rate reduction with the average BD-rate reduction being 3.3% can be obtained.

A subjective assessment method is described to determine picture quality levels in the supra-threshold region for processed images, with reference to their original counterparts, based on just-noticeable difference (JND) detection experiment. It has been found that the range of JND levels is dependent on picture contents and can be predicted as a function of texture masking factor computed in the pixel domain. The experimental data obtained also reveal that relationship of JND levels in the supra-threshold region and the MSE (mean squared error) can be approximated by a linear function whose slope is modeled as a function of edge and texture contrast masking factors. The model is devised to predict JND levels which provide subjective picture quality rating discernible by human viewers and can be used for visual quality regulated image/video coding, as well as evaluating the capacity of existing objective metrics in predicting picture quality and/or distortion relative to JND based quality/distortion rating categories.

SUMMARY This paper presents an efficient method for bitstream extraction for scalable video based on simulated annealing. Following the same spirit as annealing, the proposed method searches for the optimized combination of quality layers for extraction through slowly reducing the simulated temperature according to the characteristics of frames in different temporal levels. Experimental results show that the proposed method provides an optimized performance, which is significantly higher than that of the basic extraction method. When compared with the quality layer‐based extraction method in the reference software model of H.264/SVC (Joint Scalable Video Model), the proposed method can achieve a similar rate‐distortion performance with a significantly reduced computational complexity. Furthermore, the proposed method can obtain a more smoothed video quality, which is always preferable by the end user. Copyright © 2011 John Wiley & Sons, Ltd.