Deep video compression has made significant progress in recent years, achieving rate-distortion performance that surpasses that of traditional video compression methods. However, rate control schemes tailored for deep video compression have not been well studied. In this paper, we propose a neural network-based$\lambda$-domain rate control scheme for deep video compression, which determines the coding parameter$\lambda$for each to-be-coded frame based on the rate-distortion-$\lambda\ (\mathrm{R}-\mathrm{D}-\lambda)$relationships directly learned from uncompressed frames, achieving high rate control accuracy efficiently without the need for pre-encoding. Moreover, this content-aware scheme is able to mitigate inter-frame quality fluctuations and adapt to abrupt changes in video content. Specifically, we introduce two neural network-based predictors to estimate the relationship between bitrate and$\lambda$, as well as the relationship between distortion and$\lambda$for each frame. Then we determine the coding parameter$\lambda$for each frame to achieve the target bitrate. Experimental results demonstrate that our approach achieves high rate control accuracy at the mini-GOP level with low time overhead and mitigates inter-frame quality fluctuations across video content of varying resolutions.

Despite the unprecedented compression efficiency achieved by deep learned image compression (LIC), existing methods usually approximate the desired bitrate by adjusting a single quality factor for a given input image, which may compromise the rate control results. Considering the Rate-Distortion ( R − D ) characteristics of different spatial content, this work introduces the block-level rate control specific for LIC.

A Transformer-based Image Compression (TIC) approach is developed which reuses the canonical variational autoencoder (VAE) architecture with paired main and hyper encoder-decoders [1], as shown in Fig. 1a. Both main and hyper encoders are comprised of a sequence of neural transformation units (NTUs) to analyse and aggregate important information for more compact representation of input image, while the decoders mirror the encoder-side operations to generate pixel-domain im-age reconstruction from the compressed bitstream. Each NTU is consist of a Swin Transformer Block (STB) [2] and a convolutional layer (Conv) to best embed both long-range and short-range information; In the meantime, a causal attention module (CAM) is devised for adaptive context modeling of latent features to utilize both hyper and autoregressive priors. The TIC rivals with state-of-the-art approaches including deep convolutional neural networks (CNNs) based learnt image coding (LIC) methods and handcrafted rules-based intra profile of recently-approved Versatile Video Coding (VVC) standard, and requires much less model parameters, e.g., up to 45% reduction to leading-performance LIC.