Recent studies on video rate control (RC) have introduced accurate and high performance methods but have not explored the optimal RC solution. The optimal solution is crucial for improving RC methods and providing labels for supervised learning. To find approximate optimal RC solutions within a limited time, we are the first to propose a reinforcement learning based search method for finding approximate optimal RC solutions within a limited time. Specifically, the RC problem for a video is first modeled as a Markov decision process (MDP). Then, with the MDP model, we develop a search method based on the deep Q-network method, which consists of exploration and exploitation steps. During exploration, an agent is created, consisting of two multilayer perceptrons and a replay memory, and trained within the MDP to estimate the value function, while superior RC solutions are recorded throughout the training process. After training, RC solution is estimated by the trained agent using the value function and a greedy strategy during the exploitation step. Finally, the approximate optimal RC solution is determined based on the RC solutions from both two steps. In addition, the time complexity of proposed method is controllable, specifically,$O(m n)$where$m$denotes the number of training epoch. Experimental results show that the bit-rate error and compression quality of the solutions found by proposed method approach the optimal solutions, with differences of only less than 0.005% and 0.399%, respectively, and are achieved in a significantly shorter time compared to the brute force search.

Since the introduction of hierarchical picture prediction structure in the advanced video coding (AVC), the hierarchical coding structure (HCS) has been widely adopted and continuously improved in video coding standards. Correspondingly, the HCS-based bit allocation methods in rate control have also emerged endlessly. Considering that pictures in higher temporal levels (TLs) of HCS usually refer to pictures in lower TLs, most methods tend to allocate more bits to pictures in lower TLs. However, these methods do not fully consider the correlation of picture quality in different TLs, which leads to the bit allocation waste and the coding performance degradation. To address this issue, we propose a fast bit allocation refinement method that can adapt to different video rate control approaches. Fig. 1 shows the overall framework of the proposed method. In general, our method is to appropriately adjust the bit allocation of pictures in lower TLs according to the relationship between the quality of pictures in different TLs. Specifically, based on the hyperbolic rate-distortion (RD) model and initial allocated bits, the quality of picture in higher TLs is first predicted and then used to estimate the quality of picture in lower TLs. Subsequently, the bits of picture in lower TLs are derived using estimated quality and its RD model. Finally, the final allocated bits of picture in lower TLs are adjusted by comparing the estimated and initial allocated bits. Experimental results show that our method can improve the coding performance of different rate control methods without introducing latency and encoding complexity.

Cloud gaming is an advanced form of Internet service that necessitates local terminals to decode within limited resources and time latency. Super-Resolution (SR) techniques are often employed on these terminals as an efficient way to reduce the required bit-rate bandwidth for cloud gaming. However, insufficient attention has been paid to SR of compressed game video content. Most SR networks amplify block artifacts and ringing effects in decoded frames while ignoring edge details of game content, leading to unsatisfactory reconstruction results. In this paper, we propose a novel lightweight network called Coding Prior-Guided Super-Resolution (CPGSR) to address the SR challenges in compressed game video content. First, we design a Compressed Domain Guided Block (CDGB) to extract features of different depths from coding priors, which are subsequently integrated with features from the U-net backbone. Then, a series of re-parameterization blocks are utilized for reconstruction. Ultimately, inspired by the quantization in video coding, we propose a partitioned focal frequency loss to effectively guide the model's focus on preserving high-frequency information. Extensive experiments demonstrate the advancement of our approach.

Navigation within Cave Automatic Virtual Environment (CAVE) systems often faces challenges due to limited physical space and the necessity for seamless user interaction. Traditional solutions typically rely on multi-view tracking systems or constrained locomotion techniques, which can interrupt immersion and hinder usability. In this paper, we introduce MoRLACS, a novel locomotion approach for CAVE systems that leverages a single RGBD camera. This hybrid framework integrates small-scale physical walking with controller-based large-scale exploration through a tailored guidance method. By accurately tracking the user's head position in the real world and synchronizing it with the virtual camera, MoRLACS enables natural walking within confined CAVE spaces and supports extended interaction in larger virtual environments. Preliminary user experiments demonstrate the approach's effectiveness, revealing improvements in usability and a heightened sense of presence. These findings underscore the potential of MoRLACS to enrich user experiences in immersive CAVE settings and offer valuable design insights for integrating 3D sensor data into multimedia interaction frameworks.

Recent years witnessed the development in MPEG point cloud compression (PCC). However, the exploration of inter-frame coding may be impeded due to the lack of dynamic point clouds (point cloud sequences). To promote the development of PCC technology, we propose Dynamic3D , a dynamic 3D point cloud dataset with high-quality real-captured 3D persons and objects. There are several appealing properties: 1) Dynamic scenes: It contains five sequences and each sequence comprises 600 frames with temporal variation; 2) Complex content: instead of a single person or object in the existing dataset from MPEG, our established dataset contains multiple persons or both person and objects; 3) Realistic capture: the color industrial cameras and infrared cameras are used for data acquisition. This dataset provides the vast exploration space for PCC, especially the elimination of temporal redundancy. Extensive simulations are conducted on this dataset by using the reference software of MPEG G-PCC and V-PCC, i.e., (GeS-TM and TMC2), delivering observations, analysis and opportunities for the future research of PCC.