This paper presents an intra coding tool, named Merge mode for Template-based Intra Mode Derivation (TIMD). TIMD-Merge has been adopted in the 15thversion of the Enhanced Compression Model (ECM) software that explores video coding technologies beyond Versatile Video Coding (VVC) standard. This proposed tool operates on top of the regular TIMD mode that applies a template-based search on a causal adjacent template at top and at left of the current block, in order to find the best Intra Prediction Modes (IPMs) that matches the template. The proposed TIMD-Merge in this paper addresses a shortcoming in the regular TIMD method where due to texture discrepancy, the adjacent template information around the block is not reliable. To do so, the proposed TIMD-Merge constructs a list of all TIMD-coded blocks in relatively larger template area than the template of the regular TIMD, which also includes non-adjacent neighboring blocks. This list, called the merge list, is then sorted on the template to give one best set of TIMD modes. The use of TIMD-Merge mode is signalled at the block level and the implementation in the ECM-14.0 demonstrates -0.08% performance improvement in terms of luma BDR gain, with negligible encoding and decoding runtime increase of 100.6% and 100.2%, respectively.

This paper presents a method allowing learned video encoders to apply arbitrary latent refinement strategies to serve as RateDistortion Optimization (RDO) at the time of encoding. To do so, a latent domain search is applied on an initial latent representation of the video signal. This search is implemented as a set of iterations, each of which performs a gradient descent with back-propagation of error defined by a Lagrangian RD cost. This cost function is intentionally chosen to be the same as the cost function that was used during the end-to-end model training, except that instead of updating model weights, each iteration fine-tunes the latent representation itself. Moreover, a temporal look-ahead is integrated in the cost function of I and P frames to take into account the cascade effect of their latent fine-tuning on subsequent frames in the Group of Pictures (GOP). The experiments show that the proposed latent space RDO method can improve by 11.6% and 9.4% in terms of BD-BR coding efficiency in Random-Access (RA) and All-Intra (AI) configurations, when applied on top a high-performance opensource end-to-end codec.

The integration of advanced video codecs into the streaming pipeline is growing in response to the increasing demand for high quality video content. However, the significant computational demand for advanced codecs like Versatile Video Coding (VVC) poses challenges for service providers, including longer encoding time and higher encoding cost. This challenge becomes even more pronounced in streaming, as the same content needs to be encoded at multiple bitrates (also known as representations) to accommodate different network conditions. To accelerate the encoding process of multiple representations of the same content in VVC, we employ the encoding map of a single representation, known as the reference representation, and utilize its partitioning structure to accelerate the encoding of the remaining representations, referred to as dependent representations. To ensure compatibility with parallel processing, we designate the lowest bitrate representation as the reference representation. The experimental results indicate a substantial improvement in the encoding time for the dependent representations, achieving an average reduction of 40%, while maintaining a minimal average quality drop of only 0.43 in Video Multi-method Assessment Fusion (VMAF). This improvement is observed when utilizing Versatile Video Encoder (VVenC), an open and optimized VVC encoder implementation.

The promising compression efficiency improvement of Versatile Video Coding (VVC) compared to High Efficiency Video Coding (HEVC) [1] comes at the cost of a non-negligible encoder-side complexity. The largely increased complexity overhead is a possible obstacle towards its industrial implementation. Many papers have proposed acceleration methods for VVC. Still, a better understanding of VVC complexity, especially related to new partitions and coding tools, is desirable to help the design of new and better acceleration methods. For this purpose, statistical analyses have been conducted, with a focus on Coding Unit (CU) sizes and inter coding modes.

Modern video compression solutions rely on intra-prediction to exploit spatial redundancy within a frame. In this paper, a new intra-prediction scheme is presented whereby one or more prediction directions are inferred at the decoder side by means of a texture analysis process. The inferred directions are then combined by means of prediction fusion, to compute the final predicted samples. No further information is required in the bitstream, considerably limiting the amount of bits necessary to be transmitted in order to derive the prediction block at the decoder. Experiments in the context of the state-of-the-art Versatile Video Coding standard show the proposed method can improve coding efficiency by -0.63% BD-rates on average, and up to -1.04% on larger resolution sequences, with limited impact on the decoder complexity.

A transform coefficient coding scheme is proposed for 4 × 4 blocks in Versatile Video Coding (VVC), targeting screen content applications. The proposed algorithm, called Unary Bitplane Coding (UBC), uses unary codes of the coefficient amplitudes and represents each block by their bitplanes. This representation allows exploiting further contextual information for source separation during the entropy coding. Experiments in the Joint Exploration test Model (JEM) show that replacing the existing transform coding with UBC only for 4 × 4 blocks brings on average 2.8% and 3.4% BD-R gain in the random access and all intra modes, respectively.

An intra coding algorithm with layer separation is proposed. This algorithm is designed on top of an adopted tool in VVC, called Block DPCM (BDPCM), and benefits from texture information in a neighborhood to derive intensity levels of background and foreground layers. This information is used to reduce large rate of residual in case of incorrect layer prediction by BDPCM. For this purpose, three inter-layer transition states are defined that are either implicitly or explicitly conveyed to the decoder. Once a transition is signaled, the decoder corrects the prediction value using the derived layer information. Experiments on screen contents show a BD-rate gain of about 10% percent over VVC Test Model (VTM) and 1% over the regular BDPCM, with the cost of computational complexity.

The standardization of the next generation video codec, called Versatile Video Coding (VVC), has been progressing fast since 2018. At the same video quality, the VVC Test Model (VTM) currently provides about 27% bitrate saving compared to the preceding High Efficiency Video Coding (HEVC) standard. The ultimate objective is to reach about 50% improvement by the end of 2020. This paper presents an intra coding algorithm for VTM that skips the mode signaling at the encoder side and leaves it to be derived at the decoder side. To this end, a Histogram of Gradient (HoG) is computed from the texture of the previously reconstructed pixels and processed to derive the intra mode. Moreover, the proposed method is tuned to minimize its side-effects on the existing intra coding tools. Experimental results, on top of VTM-3.0, show that the proposed algorithm brings -0.22% BD-rate gain, with a negligible complexity.

An objective image quality assessment tool is proposed to measure image enhancement quality with emphasis on contrast. In the proposed tool, which is based on maximizing contrast with minimum artefact (MCMA), local and global properties of an image are measured through pixel‐wise and histogram‐wise features, respectively. To this aim, three sub‐measures are introduced, each of which able to detect one contrast‐related quality aspect: (i) low dynamic range of image; (ii) histogram shape preservation during image enhancement process; and (iii) local pixel diversity. These sub‐measures are combined through a subjective test to adapt them to the mean opinion scores (MOSs) of a diverse set of training contrast‐enhanced images. A regression algorithm performs the adaptation by fitting the three sub‐measures to the MOS values and finding an optimal linear combination by maximizing the Pearson correlation. In order to evaluate the performance of the MCMA algorithm, another independent, subsequent, subjective test was performed on a set of images enhanced by various known contrast enhancement algorithms to obtain MOS values and to compare them with the output of the proposed MCMA method. The experimental results show that MCMA has the highest correlation to the MOS when compared to the existing tested contrast measurement tools.

A novel intra prediction algorithm is proposed to improve the coding performance of screen content for the emerging Versatile Video Coding (VVC) standard. The algorithm, called in-loop residual coding with scalar quantization, employs in-block pixels as reference rather than the regular out-block ones. To this end, an additional in-loop residual signal is used to partially reconstruct the block at the pixel level, during the prediction. The proposed algorithm is essentially designed to target high detail textures, where deep block partitioning structure is required. Therefore, it is implemented to operate on 4× 4 blocks only, where further block split is not allowed and the standard algorithm is still unable to properly predict the texture. Experiments in the Joint Exploration Model (JEM) reference software show that the proposed algorithm brings a Bjontegaard Delta (BD)-rate gain of 13% on synthetic content, with a negligible computational complexity overhead at both encoder and decoder sides.

A few years after standardization of the High Efficiency Video Coding (HEVC), now the Joint Video Exploration Team (JVET) group is exploring post-HEVC video compression technologies. In the intra prediction domain, this effort has resulted in an algorithm with 67 internal modes, new filters and tools which significantly improve HEVC. However, the improved algorithm still suffers from the long distance prediction inaccuracy problem. In this paper, we propose an In-Loop Residual coding Intra Prediction (ILR-IP) algorithm which utilizes inner-block reconstructed pixels as references to reduce the distance from predicted pixels. This is done by using the ILR signal for partially reconstructing each pixel, right after its prediction and before its block-level out-loop residual calculation. The ILR signal is decided in the rate-distortion sense, by a brute-force search on a QP-dependent finite codebook that is known to the decoder. Experiments show that the proposed ILR-IP algorithm improves the existing method in the Joint Exploration Model (JEM) up to 0.45% in terms of bit rate saving, without complexity overhead at the decoder side.

In this study, a method for enhancing low‐contrast images is proposed. This method, called Gaussian mixture model‐based contrast enhancement (GMMCE), brings into play the Gaussian mixture modelling of histograms to model the content of the images. On the basis of the fact that each homogeneous area in natural images has a Gaussian‐shaped histogram, it decomposes the narrow histogram of low‐contrast images into a set of scaled and shifted Gaussians. The individual histograms are then stretched by increasing their variance parameters, and are diffused on the entire histogram by scattering their mean parameters, to build a broad version of the histogram. The number of Gaussians as well as their parameters are optimised to set up a Gaussian mixture modelling with lowest approximation error and highest similarity to the original histogram. Compared with the existing histogram‐based methods, the experimental results show that the quality of GMMCE enhanced pictures are mostly consistent and outperform other benchmark methods. Additionally, the computational complexity analysis shows that GMMCE is a low‐complexity method.