Trit-plane coding enables deep progressive image compression, but it cannot use autoregressive context models. In this paper, we propose the context-based trit-plane coding (CTC) algorithm to achieve progressive compression more compactly. First, we develop the context-based rate reduction module to estimate trit probabilities of latent elements accurately and thus encode the trit-planes compactly. Second, we develop the context-based distortion reduction module to refine partial latent tensors from the trit-planes and improve the reconstructed image quality. Third, we propose a retraining scheme for the decoder to attain better rate-distortion tradeoffs. Extensive experiments show that CTC outperforms the baseline trit-plane codec significantly, e.g. by -14.84% in BD-rate on the Kodak loss less dataset, while increasing the time complexity only marginally. The source codes are available at https://github.com/seungminjeon-github/CTC.

We propose the deep progressive image compression using trit-planes (DPICT) algorithm, which is the first learning-based codec supporting fine granular scalability (FGS). First, we transform an image into a latent tensor using an analysis network. Then, we represent the latent tensor in ternary digits (trits) and encode it into a compressed bitstream trit-plane by trit-plane in the decreasing order of significance. Moreover, within each trit-plane, we sort the trits according to their rate-distortion priorities and transmit more important information first. Since the compression network is less optimized for the cases of using fewer tritplanes, we develop a postprocessing network for refining reconstructed images at low rates. Experimental results show that DPICT outperforms conventional progressive codecs significantly, while enabling FGS transmission. Codes are available at https://github.com/jaehanlee-mcl/DPICT.

Adaptive streaming service nowadays, became an essential key technology for delivering videos over internet protocol network. Due to limitations and fluctuations in internet bandwidth, scaler has become essential for streaming service. Recently, learning-based scalers have greatly improved performance compared to conventional methods in visual quality. However, performance is not guaranteed for compatibility of conventional scaler and real-time processing is difficult due to high complexity. In this paper, we present a low complexity scaler based on convolutional neural networks called Video Scale Network (VSN). The proposed method has a simple structure with real-time processing and a loss function compatible of conventional scaler. Furthermore, we propose a learning method to improve performance using only a single network. Experimental results on the AOM test sequences reveal improved performance by using the proposed method compared to the conventional method and real-time processing in the decoder of AV1 codec is also able to be done.

The goal of downscaling an image is to reduce its resolution to a lower resolution, while maintaining the visual characteristics of the original image. Recent learning-based algorithms have shown great improvement over conventional methods in preserving high-frequency information. However, they continue to suffer from various artifacts due to the lack of true high-low resolution image pairs in their training datasets. In this paper, an unsupervised image downscaler that preserves the high frequency content of the original image based on an autoencoder is presented. Specifically, the image downscaler is obtained by extracting the decoder of the developed autoencoder. Furthermore, we propose a preprocessing step that enables image downscaler to any arbitrary scales. Experimental results on five benchmark datasets reveal the qualitative and quantitative superiority of the proposed method at various scales compared to other methods.

Conventional video quality assessment methods are either full-, reduced-, or no-reference methods that need to access decoded videos. Hence, to calculate quality of decoded video in video coding regarding an image/video quality metric, complete encoding and decoding have to executed, which is computationally expensive. To address this problem, we propose to estimate quality of decoded videos from the original video only (i.e., only-reference) using convolutional neural network, as if the original video is encoded using a range of quantization parameter. The proposed network is shallow and can be trained to estimate various video quality metrics. Furthermore, among potential rate control applications using the proposed network, we demonstrate achieving a targeted decoded-video quality by selecting a proper quantization parameter before actually encoding.