In this paper, we propose a novel implicit neural representation (INR)-based image compression paradigm that generate image-specific latents from a multi-scale Gaussian noise tensor which is deterministically generated using a shared random seed. As shown in Fig. 1, inspired by the normalization strategy in E2E codecs where latents are transformed into Gaussian noise to demonstrate the removal of spatial redundancy, we explore the inverse direction: generating Gaussian latents directly from Gaussian noise. Specifically, given a noise input$z$, we introduce a Gaussian Parameter Prediction (GPP) module that estimates the Gaussian distribution parameters, enabling one-shot latent generation via the reparameterization trick. The predicted latent is then transformed into the reconstructed image through a synthesis network.

Recently, implicit neural representation (INR)-based image compression has attracted growing attention due to its low decoding complexity. However, most existing methods require multiple encoding processes to generate multiple bitstreams and reconstructions at different quality levels, leading to high storage and computational overhead. A more practical solution is progressive compression, which supports multiple qualities within a single bitstream. In this paper, we propose PINIC (Progressive Implicit Neural Image Compression), a DWT-guided framework that integrates progressive functionality into INR-based compression for the first time. Specifically, DWT energy is applied to adaptively allocate the energy of latent channels and latent will be entropy coded in the order of channel energy. Moreover, we adopt reparameterization to decouple network structures between training and inference, retaining the expressive power of complex architectures while reducing parameter count at deployment. Extensive experiments demonstrate that PINIC achieves state-of-the-art rate-distortion performance on the Kodak and CLIC2020P datasets, while simultaneously supporting progressive compression.

Screen content has emerged as a prominent medium in our increasingly connected world. However, compressed screen content images often suffer from unpleasant artifacts, significantly obstructing the comprehension of text and graphic regions. In this paper, we introduce a quality enhancement framework specifically designed for compressed screen content images. We first propose a dataset for enhancing the quality of screen content images affected by various levels of compression distortion, using state-of-the-art Versatile Video Coding with screen content coding techniques enabled. Given the unique characteristics of screen content images, our enhancement framework incorporates B-spline representation to mitigate the quality degradation caused by compression. Additionally, we focus on recovering distorted text by detecting text regions within the degraded image and generating a pristine textual map to guide the recovery process. Experimental results demonstrate that our proposed method effectively enhances the quality of reconstructed screen content images across different compression distortion levels, leading to the quantitative and qualitative improvement.

Recent exploration efforts in JVET (Joint Video Experts Team of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29) has achieved progresses on neural network-based video coding (NNVC)11NNVC is also the name of the reference software for evaluating neural network-based video coding technologies in JVET. The project locates at https://vcgit.hhi.fraunhofer.de/jvet-ahg-nnvc/VVCSoftware_VTM. Latest version of NNVC features two normative deep tools, i.e., neural network-based intra prediction and neural network-based in-loop filtering. Specifically, the neural network-based filtering in NNVC supports three operating points, known as VLOP (very low-complexity operating point), LOP (low-complexity operating point), and HOP (high-complexity operating point). LOP filter receives more attention among these three due to its favorable performance-complexity trade-off. In this paper, we introduce CCLOP, a cross-component enhanced LOP filter. CCLOP builds upon LOP filter in NNVC but incorporates deep luma features for chroma filtering. We conduct extensive experiments to verify the effectiveness of CCLOP. Compared with NNVC-10, the latest reference software of NNVC, CCLOP achieves {-0.13%, −2.27%, −3.11%}, {-0.18%, −2.07%, −3.21%}, and {-0.03%, −1.81%, −2.51%} BD-rate changes on average for {Y, Cb, Cr} under random-access, low-delay, and all-intra configurations respectively, while maintaining the same complexity as existing LOP filter ([email protected] kMAC/pixel, [email protected] kMAC/pixel).

Neural compression methods often rely on highly expressive models to fit large datasets, resulting in significant decoding complexity. Overfitted codecs have been proposed as an alternative to reduce decoding complexity. However, these approaches typically lack flexibility in encoding configurations. To address this, we introduce CD, a neural video compression method that employs picture-wise overfitting. CD is built upon the Cool-chic video framework [1], but incorporates Decoupled representations for motion and residue. Additionally, we propose an effective training strategy for CD to further enhance its performance.

In this paper, a hybrid video compression framework is proposed that serves as a demonstrative showcase of deep learning-based approaches extending beyond the confines of traditional coding methodologies. The proposed hybrid framework is founded upon the Enhanced Compression Model (ECM), which is a further enhancement of the Versatile Video Coding (VVC) standard. We have augmented the latest ECM reference software with well-designed coding techniques, including block partitioning, deep learning-based loop filter, and the activation of block importance mapping (BIM) which was integrated but previously inactive within ECM, further enhancing coding performance. We evaluate the coding performance of the proposed framework with extensive experiments on the JVET dataset compared with ECM10.0 and VTM-11.0. Due to the testing environment and the coding complexity of the ECM, we did not conduct testing on Class A. The QPs are set as 22, 27, 32, 37, and 42. Compared with ECM-10.0, our method achieves 6.26%, 13.33%, and 12.33% BD-rate savings for the Y, U, and V components under random access (RA) configuration. The traditional hybrid coding framework combined with the three coding tools can further improve compression efficiency and has great potential for performance improvement.

Resampling-based coding, i.e. down-sampling before encoding and up-sampling after decoding, has been recognized to be an effective tool for compressing high-resolution videos at low bitrates. The newest video coding standard, Versatile Video Coding (VVC), supports resampling-based coding via a mechanism named Reference Picture Resampling (RPR), where the spatial resolution can be changed without inserting an intra frame. Intuitively, it is not wise to utilize a single resolution throughout the whole video, because frames with different contents may prefer different coding resolutions. In this paper, we propose a smart reference picture resampling approach, namely smart-RPR, where the coding-resolution of a frame is determined based on the property of the frame without multiple-pass encoding. Specifically, we first down- and up-sample a frame without considering compression and compare the up-sampled frame with the original frame to obtain the resampling distortion, which is then compared with a threshold to decide whether to code the frame in a resampling way. Then, we build up an exponential model to approximate the optimal threshold. In addition, we also study how to derive the coding parameters of the down-sampled frame to achieve better performance. Simulation results on the VTM-12.0 show that the proposed method could achieve 2.72%, 5.29%, and 10.82% BD-rate reductions for Y, Cb, and Cr components, respectively, with lower encoding and decoding complexity.

Existing deep learning based approaches for coding artifact removal mainly focus on the relationship between reconstruction and original signals. Although the CNN-based in-loop filtering reveals significant potential, the filtering procedure is separated from the rate distortion optimization (RDO) in the block-based hybrid framework. In this paper, we propose a joint rate distortion optimization scheme [1] with CNN-based in-loop filter for the Versatile Video Coding (VVC), with the goal of excavating the potential of in-loop filtering. In particular, a joint rate distortion optimization is presented, to consider the CNN-based in-loop filtering in the coding unit. More specifically, the CNN-based in-loop filter is involved in the partitioning mode selection, and meanwhile the prediction mode determination process remains unchanged. The proposed joint optimization is only applied to the CUs, whose height and width are no larger than 64. To keep the encoding complexity as low as possible, a light-weight version of CNN-based model is provided to facilitate the encoding optimization. As shown in Table 1, the proposed method brings 9.61% and 12.81% BD-Rate gains for luma component under all intra and random access configurations, respectively, on VTM-11.0-nnvc platform.

Versatile Video Coding (VVC) achieves promising compression performances compared with High Efficiency Video Coding (HEVC) while sacrificing the encoding speed. This paper focuses on predicting the partitioning structures with convolutional neural networks to speedup the VVC encoder. Specifically, we formulate the partitioning prediction problem into two alternatives: implicit partitioning prediction based on the split type of subblock boundaries and explicit partitioning prediction from the ensemble partitioning space. Then, we address both formulations using convolutional neural networks.

In this paper, based on the Hessian approximation, an incremental pruning method is proposed to compress the deep neural network. The proposed method starts from the idea of using the Hessian to measure the "importance" of each weight in a deep neural network, and it mainly has the following key contributions. First, we propose to use the second moment in Adam optimizer as a measure of the "importance" of each weight to avoid calculating the Hessian matrix. Second, an incremental method is proposed to prune the neural network step by step. The incremental method can adjust the remaining non-zero weights of the whole network after each pruning to help boost the performance of the pruned network. Last but not least, the proposed method applies an automatically-generated global threshold for all the weights among all the layers, which achieves the inter-layer bit allocation automatically. Such a method can improve performance and save the complexity of adjusting the pruning threshold layer by layer. We perform a number of experiments on MNIST and ImageNet using commonly used neural networks such as AlexNet and VGG16 to show the benefits of the proposed algorithm. The experimental results show that the proposed algorithm is able to compress the network significantly with almost no loss of accuracy, which demonstrates the effectiveness of the proposed algorithm.