Recent studies on video rate control (RC) have introduced accurate and high performance methods but have not explored the optimal RC solution. The optimal solution is crucial for improving RC methods and providing labels for supervised learning. To find approximate optimal RC solutions within a limited time, we are the first to propose a reinforcement learning based search method for finding approximate optimal RC solutions within a limited time. Specifically, the RC problem for a video is first modeled as a Markov decision process (MDP). Then, with the MDP model, we develop a search method based on the deep Q-network method, which consists of exploration and exploitation steps. During exploration, an agent is created, consisting of two multilayer perceptrons and a replay memory, and trained within the MDP to estimate the value function, while superior RC solutions are recorded throughout the training process. After training, RC solution is estimated by the trained agent using the value function and a greedy strategy during the exploitation step. Finally, the approximate optimal RC solution is determined based on the RC solutions from both two steps. In addition, the time complexity of proposed method is controllable, specifically,$O(m n)$where$m$denotes the number of training epoch. Experimental results show that the bit-rate error and compression quality of the solutions found by proposed method approach the optimal solutions, with differences of only less than 0.005% and 0.399%, respectively, and are achieved in a significantly shorter time compared to the brute force search.

In recent years, rate control (RC) for neural video coding (NVC) has become an active research area. However, existing RC methods in NVC neglect the actual rate-distortion (R-D) characteristics and lack dedicated optimization strategies for intra and inter modes, leading to significant bit rate errors. To address these issues, we propose a high accuracy RC method for NVC based onR-Dmodeling, which integrates intra frame RC, inter frame RC and bit allocation. Specifically, the rate-quantization parameter (R-Q) model andR-Dmodel are established for both intra frame and inter frame in NVC. To derive the model parameters, intra frame parameters are estimated using high dimensional features, while inter frame parameters are derived using gradient descent based model update methods. Based on the proposedR-Qmodel, intra frame and inter frame RC methods are proposed to determine the quantization parameters (QP). Meanwhile, a bit allocation method is developed based on the derivedR-Dmodels to allocate bits for the intra frame and inter frame. Extensive experiments demonstrate that, benefiting from the accurateR-Qmodels derived by the proposed approach, highly accurate RC is achieved with only 0.56% average bit rate error. Compared with other methods, the proposed method reduces the average bit rate error by more than 4.18%, and achieves over 8.94% Bjøntegaard Delta Rate savings.

Since the introduction of hierarchical picture prediction structure in the advanced video coding (AVC), the hierarchical coding structure (HCS) has been widely adopted and continuously improved in video coding standards. Correspondingly, the HCS-based bit allocation methods in rate control have also emerged endlessly. Considering that pictures in higher temporal levels (TLs) of HCS usually refer to pictures in lower TLs, most methods tend to allocate more bits to pictures in lower TLs. However, these methods do not fully consider the correlation of picture quality in different TLs, which leads to the bit allocation waste and the coding performance degradation. To address this issue, we propose a fast bit allocation refinement method that can adapt to different video rate control approaches. Fig. 1 shows the overall framework of the proposed method. In general, our method is to appropriately adjust the bit allocation of pictures in lower TLs according to the relationship between the quality of pictures in different TLs. Specifically, based on the hyperbolic rate-distortion (RD) model and initial allocated bits, the quality of picture in higher TLs is first predicted and then used to estimate the quality of picture in lower TLs. Subsequently, the bits of picture in lower TLs are derived using estimated quality and its RD model. Finally, the final allocated bits of picture in lower TLs are adjusted by comparing the estimated and initial allocated bits. Experimental results show that our method can improve the coding performance of different rate control methods without introducing latency and encoding complexity.

In recent years, user-generated content (UGC) videos have become the mainstream of internet videos, which are characterized by their rich content, complicated temporal changes and multiple distortions. However, existing rate control (RC) methods do not consider the above unique characteristics, leading to severe bit-rate errors and coding performance degradation. To address these issues, we propose a content-adaptive RC method for UGC videos, where accurate RC coding parameters are derived by our proposed rate-distortion (RD) model derivations for different types of pictures and a novel bit allocation refinement module. Specifically, the RD models of intra pictures are derived by established SVR-based predictors using some features designed for diverse content, such as texture complexity and regularity. Considering the complex temporal variation, single-reference inter pictures are firstly classified into three categories (i.e., low, regular and high correlation) by a SVM-based classifier using correlation-based features. Training data of the classifier are labeled by introducing a series of classification metrics. Then, RD model is derived by established predictors accordingly for each type of inter pictures. In addition, the RD model of multiple-reference inter pictures is derived by using a updated RD model selection based on content similarity. Based on derived RD models, allocated bits are refined to reduce bit waste. Experimental results show that compared with the default RC method in versatile video coding (VVC), our method can effectively save BD-Rate and reduce bit-rate errors for UGC videos. In particular, 1.99% BD-Rate saving and 0.18% bit-rate error reduction can be achieved under the random access (RA) configuration, and 0.45% BD-Rate saving under the low-delay B (LDB) configuration.

Existing quantization parameter (QP) adjustment methods in video coding often focus solely on coding efficiency and ignore the impact of bit-rate fluctuations on video transmission and bandwidth waste. This is mainly because intra pictures, in a hierarchical coding structure, are allocated smaller QP and thus consume more bits. To address this issue, we propose an adaptive block-level QP adjustment method. Specifically, intra picture importance (IPI) is first introduced to evaluate the adjustability of intra picture QP. For intra pictures whose QP can be adjusted, we further propose block importance (BI) to determine their optimal block-level QP adjustment. Experimental results show that our proposed method reduce the bit-rate fluctuations while basically maintaining the coding performance. Notably, significant improvements can be observed in high-resolution videos, with a reduction of approximately 11% in bit-rate fluctuations.

Lossy compression introduces complex compression artifacts such as the blocking, ringing and blurring artifacts, making decoded videos unpleasant for human visual system. In this paper, we propose a coding prior based high efficiency restoration algorithm to remove these compression artifacts. To improve the quality of the restored compressed videos, we take full advantage of side information from coding streams as coding prior which is ignored or not fully exploited by most existing post-processing methods. In particular, the unfiltered frames and the prediction frames are derived from coding streams which are utilized as coding priors and a high efficiency neural network is designed for these information to improve the overall quality of restored videos. Extensive experimental results on HEVC coding streams demonstrate that our proposed method can significantly improve both the objective and subjective quality of compressed videos.