In recent years, image compression based on diffusion models has achieved excellent perceptual reconstruction. However, their models typically support only fixed bitrates, which results in significant training costs and memory demands. In this paper, we propose a quantization-controllable diffusion-based image compression framework, which extends diffusion-based extreme compression to variable-rate scenarios by modulating quantization through a single control parameter. Furthermore, we introduce the Quantization Regulator Modulation RefineNet (QRMR), which dynamically modulates the diffusion decoding process according to different quantization levels, allowing the diffusion-based decoder to be quantization-aware and thereby significantly improving the rate-distortion performance. Experiments demonstrate that our method achieves good fidelity and perceptual quality in reconstructed images at extremely low bitrates.

Image codecs are typically optimized to trade-off between bitrate and distortion metrics. At low bitrates, they often result in compression artefacts. To solve this problem, we leverage the ability of diffusion models to produce high-quality images and design a scalable lossy image compression framework for ultra-low bitrates, as shown in Fig. 1. On the encoding side, we extract multiple conditional information from the image and perform scalable encoding on them. When the bandwidth is limited, only the text is encoded. We use Prompt Inversion (PI) to extract text and apply lossless compression using Lempel-Ziv (LZ) coding from the zlib library. As the bandwidth gradually increases, sketch and Spatial color palette can be encoded to supplement the conditional information. We use the edge prediction model PiDiNet to extract the sketch of the image. We further perform scalable palette compression to provide color information at different bitrates. We downsample the image by 64x using bicubic interpolation and upsample it to 1/32, 1/16, 1/4, and 1/2 of the original size with nearest-neighbor to obtain the spatial color palette at different bitrates. We use standard learned nonlinear transform codes (NTC) to compress the sketch and spatial color palette. On the decoding side, We use the T2I-Adapter [1] as our decoder. The text is processed through the CLIP text encoder to generate conditional text embeddings, while the sketch and spatial color palette are fed through adapters to obtain features at different scales. The decoding is performed in a scalable manner, where the transmitted information is processed as described above and input into the diffusion model to guide the image generation. Compared to JPEG and the latest diffusion-based methods (PIC and PICS [2]), our method achieves better perceptual quality, as shown in Table 1. It's important to note that t denotes using text as the sole condition, ts indicates using both text and sketch, and tsc1 to tsc4 refer to using text, sketch, and color palette at different bitrates as conditions. Experimental results show that as the number of conditional information increases and the accuracy of color information improves, the perceptual quality of the images improves.

Summary form only given. Traditional image coding standards (such as JPEG and JPEG2000) make the decoded image suffer from many blocking artifacts or noises since the use of big quantization steps. To overcome this problem, we proposed an end-to-end compression framework based on two CNNs, as shown in Figure 1, which produce a compact representation for encoding using a third party coding standard and reconstruct the decoded image, respectively. To make two CNNs effectively collaborate, we develop a unified end-to-end learning framework to simultaneously learn CrCNN and ReCNN such that the compact representation obtained by CrCNN preserves the structural information of the image, which facilitates to accurately reconstruct the decoded image using ReCNN and also makes the proposed compression framework compatible with existing image coding standards.