In this paper, we propose a learning-based video compression which can perform continuously variable bitrate coding. The proposed method generates feature transformation parameters through a conditional network according to the input spatial quality map. These parameters are then used to adaptively transform the intermediate features of the encoder, decoder, and spatiotemporal entropy model in the codec, thus enabling variable bitrate coding. Additionally, to improve the compression efficiency of the codec, we propose incorporating the quality map of the preceding frame into the hyperprior encoder and leveraging the temporal prior encoder. A multi-stage training strategy is employed to jointly train the codec with a multi-frame rate-distortion loss function. The experimental results demonstrate that the proposed method can achieve continuously variable bitrate adaptation while maintaining rate-distortion performance comparable to the fixed bitrate model. Furthermore, the proposed method also supports ROI-based compression.

Cloud virtual reality (Cloud VR) services usually introduce high latency in rendering and streaming, resulting in a mismatch between the visual and vestibular systems, causing user sickness and dizziness during the service. To solve this problem, asynchronous rendering technology is usually used to provide smooth viewing. The asynchronous solution, on the other hand, will introduce another “black edge” (BE) artifact in the service, which frequently appears at the viewport's boundary with a black area when users turn their heads. This unwanted BE artifact also has an impact on the user's quality of experience (QoE). In this paper, we investigated the impact of the BE artifact on the user's QoE of the field of view (FOV) Cloud VR gaming services. The appearance of the BE artifact during the playing period was regarded as a series of BE events and the impact of BE artifact on the users’ QoE was evaluated by accumulating the influence of the BE events during the whole playing period. More specifically, the user's QoE affected by a single BE event was first evaluated by combining the area ratio and duration of the BE artifact. Then, the QoE affected by multiple BE events was analyzed, where the cumulative influence of the previous BE events on the user's current QoE was evaluated. Finally, a unified event-based evaluation model was proposed to predict the user's time-varying QoE at any point in time. Experimental results showed that the proposed model performed exceptionally well in predicting the impact of BE artifact on the user's QoE.

Quality control is an important topic in many application scenarios such as medical image compression. To achieve quality control in image compression network, in this paper, we propose a quality controllable image compression network, Quality Controllable Variational Autoencoder (QCVAE). QC-VAE consists of the Quality-Feature-Level (QFL) model we proposed and the Hyperprior Continuously Variable Rate (HCVR) image compression network which can adapt to multiple target qualities with only one single model. Note that even if the target quality is the same, different images should be quantized with different levels. With the help of the QFL model, we can obtain the estimated corresponding quantization level of the input image under the target quality, the selected level will be used to control the quantization loss. By this means, the QC-VAE can adapt to the target quality with high accuracy. Experimental results have shown that compared with the HCVR model, the proposed QC-VAE achieves accurate quality control without rate-distortion (RD) performance loss, indicating its superiority.

In this paper, we propose a rate controllable image compression framework, Rate Controllable Variational Autoencoder (RC-VAE), based on the Rate-Feature-Level (RFL) model established through our exploration on the correlation among target rates, image features and quantization levels. Considering that, when meeting the same target rate, different images should be quantized in different levels, we focus on jointly utilizing the target rate and the extracted features of the image to predict the corresponding quantization level and propose the RFL model. Combining the proposed RFL model with a Hyperprior Continuously Variable Rate (HCVR) image compression network, we further propose the RC-VAE. By controlling information loss in quantization process, the RC-VAE can work at the target rate. Experimental results have demonstrated that one single RC-VAE model can adapt to multiple target rates with higher rate control accuracy and better R-D performance compared with the state-of-the-art rate controllable Image compression networks.

When user turns head in an interactive Cloud VR service, such as the Cloud VR games, the black edge artifact often appears at the boundary of the viewport due to the inevitable latency, which significantly affects the user experience. In this paper, we focused on the interactive Cloud VR gaming services and investigated the influence of the black edge artifact on the user experience. The appearance of the black edge artifact during the playing period was regarded as a series of black edge events. The user experience on a single black edge event was first evaluated combining the influential factors of the area ratio and duration of the black edge artifact. Then, a pooling strategy was proposed to evaluate the user’s overall experience. Experimental results showed that the proposed model can accurately predict the user experience that influenced by the black edge artifact for the Cloud VR services. This work can be served as a guideline for service providers and network operators to improve their services.