This fact sheet describes our proposed method for the video track of Challenge on Learned Image Compression (CLIC) 2024. Our scheme follows the typical hybrid coding framework with advanced techniques in motion estimation, context mining, and spatial-temporal super-resolution to enhance rate-distortion performance, particularly at low bitrates.

We introduce an enhanced spatial perception module, as shown in Fig. 1 , pre-trained on multiple image quality assessment datasets, and a lightweight temporal fusion module to address the no-reference visual quality assessment (NR-VQA) task. This model implements Swin Transformer V2 [1] as a local-level spatial feature extractor and fuses these multi-scale features to enhance the quality-aware information. Furthermore, a temporal transformer is utilized for spatiotemporal feature fusion. To accommodate compressed videos of varying bitrates, we incorporate a coarse-to-fine contrastive strategy, that is, the group contrast loss is used for coarse discrimination of different bitrates, and the rank loss is used at a fine-grained level to enrich the model’s capability to discriminate different quality level.

During Coefficient thresholding (CT), the last several nonzero DCT coefficients after quantization are dropped if better Rate-Distortion (RD) performance can be achieved. Since image data can be reconstructed by incomplete frequency information with some prior knowledge of image property (e.g., luminance continuity), the quality degradation caused by CT can be sometimes alleviated by some prior knowledge based image restoration. Consider an 8×8 image block I that is represented by 64 transform coefficients of the prediction residual, C1~64=DCT(I-Ipred). When CT is performed, the last few nonzero coefficients of C1~64are dropped as long as the RD cost can be reduced. Then the reconstructed block Ireccan be calculated by Irec1=IDCT(C1~k)+Ipred, where k denotes the index of last nonzero coefficient of the remaining. With some certain image restoration technique, the lost information during CT can be partially recovered, Irec2=RESTORE(DCT(C1~k) )+Ipred). We employ Bilateral Filter (BF) for the image restoration after CT. This CT/BF approach is implemented as a candidate mode in addition to the traditional IDCT mode, and RD optimization is utilized for the mode selection. The CT/BF mode is enabled for the blocks with texture and edges, which saves considerable computational complexity. Moreover, to save the overhead of the mode flags, the mode information are transmitted covertly in terms of the parity of the number of nonzero coefficients like watermarks. Experiments show that the codec with CT/BF improves the quality of decoded video by up to 0.54 dB compared to H.264 high profile.