Providing high video quality under the lowest possible bitrate constraint is one of the critical challenges in video coding technology. Inspired by the continuous development of motion imitation [1], the model-based video coding method is derived from extracting a series of features or parameters representing the person's motion and reconstructing each frame by motion imitation model at the decoder. Thus, we propose a Semantic Neural Rendering-based Video Coding framework (SNRVC) to transmit video at ultra-low bitrate while maintaining high subjective quality. At the encoder, we first extract the motion parameters with specific semantic meanings from each frame and then compress the first frame and the parameters of the subsequent frames by truncating to different decimals and differential pulse code modulation coding. Finally, the decoded image and parameters are fed into the motion imitator [2] to obtain each reconstructed frame consistent with the movements of the original frame. Our SNRVC can achieve better visual quality than traditional and model-based methods [3] at the ultra-low bitrate below 0.01 bpp.

Image compression approaches based on deep learning have achieved remarkable success. Existing studies mainly focus on human vision and machine analysis tasks taking reconstructed images as input. However, those methods need images to be decoded before performing downstream visual tasks, which motivates us to explore how to directly conduct visual analysis using the compressed data without decoding. The overview of our proposed model is shown as Fig. 1(a). Specifically, a task-agnostic learning-based compression model is proposed, which effectively supports various compressed domain-based analytical tasks meanwhile reserves outstanding re-constructed perceptual quality compared with traditional and learning-based codecs. To obtain the extremely compacted data representation with essential semantic infor-mation, we take the help of the generative model on decoder part. Then, we propose a multi-task learning model which can directly obtain semantic information from the compressed visual data. The pipeline of the proposed model is detailedly illus-trated in Fig. 1(b). In addition, joint optimization strategy is adopted to achieve the best balance point among compression efficiency, reconstructed image quality, and the downstream visual tasks' performance. Experimental results verify that our proposed compressed domain-based multi-task analysis model outperforms the reconstructed image-based method on transmission efficiency, saving more than ten times of bit-rate consumption while preserving comparable visual analysis precision (i.e., classification and segmentation tasks) when compared with RGB image input models, which is evaluated on the CelebA-HO dataset.