Coding standards for secure programming recommend "scrubbing" sensitive data once it is no longer needed; otherwise, secrets may be recovered, as illustrated in the Heartbleed attack. Despite being an effective software-based countermeasure, zeroization, i.e., overwriting with zeroes, turns out to be challenging and error-prone. Current verification approaches suffer from scalability or precision issues when applied to production software in practice. In this paper, we put forward the GCC Static Analyzer (GSA), which is a symbolic execution engine for error finding. Specifically, we extend the GSA to build GnuZero; our automated tool that detects missing zeroization for all stack/heap variables storing sensitive data, either directly or by derivation. Our experiments confirm GnuZero efficiency and effectiveness in verifying real-world benchmarks. In particular, GnuZero passes all the relevant Juliet’s test programs, namely associated to the MITRE’s CWE-244 and CWE-226. In addition, GnuZero succeeds in identifying new vulnerabilities in open-source cryptographic modules.

Overfitted neural video codecs offer a decoding complexity orders of magnitude smaller than their autoencoder counterparts. Yet, this low complexity comes at the cost of limited compression efficiency, in part due to their difficulty capturing accurate motion information. This paper proposes to guide motion information learning with an optical flow estimator. A joint rate-distortion optimization is also introduced to improve rate distribution across the different frames. These contributions maintain a low decoding complexity of 1300 multiplications per pixel while offering compression performance close to the conventional codec HEVC and outperforming other overfitted codecs. This work is made open-source at https://orange-opensource.github.io/Cool-Chic/.

Neural image compression, based on auto-encoders and overfitted representations, relies on a latent representation of the coded signal. This representation needs to be compact and uses low resolution feature maps. In the decoding process, those latents are upsampled and filtered using stacks of convolution filters and non linear elements to recover the decoded image.Therefore, the upsampling process is crucial in the design of a neural coding scheme and is of particular importance for overfitted codecs where the network parameters, including the upsampling filters, are part of the representation.This paper addresses the improvement of the upsampling process in order to reduce its complexity and limit the number of parameters. A new upsampling structure is presented whose improvements are illustrated within the Cool-Chic overfitted image coding framework. The proposed approach offers a rate reduction of 4.7%. The source code is available [1].

This paper summarises the design of the Cool-Chic candidate for the Challenge on Learned Image Compression. This candidate attempts to demonstrate that neural coding methods can lead to low complexity and lightweight image decoders while still offering competitive performance. The approach is based on the already published overfitted lightweight neural networks Cool-Chic, further adapted to the human subjective viewing targeted in this challenge.

We propose a lightweight learned video codec with 900 multiplications per decoded pixel and 800 parameters overall. To the best of our knowledge, this is one of the neural video codecs with the lowest decoding complexity. It is built upon the overfitted image codec Cool-chic and supplements it with an inter coding module to leverage the video’s temporal redundancies. The proposed model is able to compress videos using both low-delay and random access configurations and achieves rate-distortion close to AVC while outperforming other overfitted codecs such as FFNeRV. The system is made open-source: orange-opensource.github.io/Cool-Chic.

This paper summarises the design of the candidate ED for the Challenge on Learned Image Compression 2024. This candidate aims at providing an anchor based on conventional coding technologies to the learning-based approaches mostly targeted in the challenge.The proposed candidate is based on the Enhanced Compression Model (ECM) developed at JVET, the Joint Video Experts Team of ITU-T VCEG and ISO/IEC MPEG.Here, ECM is adapted to the challenge objective: to maximise the perceived quality, the encoding is performed according to a perceptual metric, also the sequence selection is performed in a perceptual manner to fit the target bit per pixel objectives.The primary objective of this candidate is to assess the recent developments in video coding standardisation and in parallel to evaluate the progress made by learning-based techniques. To this end, this paper explains how to generate coded images fulfilling the challenge requirements, in a reproducible way, targeting the maximum performance.

International audience

We introduce COOL-CHIC, a Coordinate-based Low Complexity Hierarchical Image Codec. It is a learned alternative to autoencoders with 629 parameters and 680 multiplications per decoded pixel. COOL-CHIC offers compression performance close to modern conventional MPEG codecs such as HEVC and is competitive with popular autoencoder-based systems. This method is inspired by Coordinate-based Neural Representations, where an image is represented as a learned function which maps pixel coordinates to RGB values. The parameters of the mapping function are then sent using entropy coding. At the receiver side, the compressed image is obtained by evaluating the mapping function for all pixel coordinates. COOL-CHIC implementation is made open-source1.

We propose a neural image codec at reduced complexity which overfits the decoder parameters to each input image. While autoencoders perform up to a million multiplications per decoded pixel, the proposed approach only requires 2300 multiplications per pixel. Albeit low-complexity, the method rivals autoencoder performance and surpasses HEVC performance under various coding conditions. Additional lightweight modules and an improved training process provide a 14% rate reduction with respect to previous overfitted codecs, while offering a similar complexity. This work is made open-source at http://orange-opensource.github.io/Cool-Chic/.

This paper introduces AIVC, an end-to-end neural video codec. It is based on two conditional autoencoders MNet and CNet, for motion compensation and coding. AIVC learns to compress videos using any coding configurations through a single end-to-end rate-distortion optimization. Furthermore, it offers performance competitive with the recent video coder HEVC under several established test conditions. A comprehensive ablation study is performed to evaluate the benefits of the different modules composing AIVC. The implementation is made available at https://orange-opensource.github.io/AIVC/.

In this paper, we propose to enhance learned image compression systems with a richer probability model for the latent variables. Previous works model the latents with a Gaussian or a Laplace distribution. Inspired by binary arithmetic coding, we propose to signal the latents with three binary values and one integer, with different probability models.A relaxation method is designed to perform gradient-based training. The richer probability model results in a better entropy coding leading to lower rate. Experiments under the Challenge on Learned Image Compression (CLIC) test conditions demonstrate that this method achieves 18 % rate saving compared to Gaussian or Laplace models.

This paper introduces a new method for inter-frame coding based on two complementary autoencoders: MOFNet and CodecNet. MOFNet aims at computing and conveying the Optical Flow and a pixel-wise coding Mode selection. The optical flow is used to perform a prediction of the frame to code. The coding mode selection enables competition between direct copy of the prediction or transmission through CodecNet.The proposed coding scheme is assessed under the Challenge on Learned Image Compression 2020 (CLIC20) P-frame coding conditions, where it is shown to perform on par with the state-of-the-art video codec ITU/MPEG HEVC. Moreover, the possibility of copying the prediction enables to learn the optical flow in an end-to-end fashion i.e. without relying on pre-training and/or a dedicated loss term.

The future video coding standard named Versatile Video Coding (VVC) is expected by the end of 2020. VVC will enable better coding efficiency than the current High Efficiency Video Coding (HEVC) standard. This coding gain is brought by several coding tools. The Multiple Transform Selection (MTS) is one of the key coding tools that have been introduced in VVC. The MTS concept relies on three transform types including Discrete Cosine Transform (DCT)-II, Discrete Sine Transform (DST)-VII and DCT-VIII. Unlike the DCT-II that has fast computing algorithms, the DST-VII and DCT-VIII rely on more complex matrix multiplication. In this paper an approximation approach is proposed to reduce the computational cost of the DST-VII and DCT-VIII. The approximation consists in applying adjustment stages, based on sparse block-band matrices, to a variant of DCT-II family mainly DCT-II and its inverse. Genetic algorithm is used to derive the optimal coefficients of the adjustment matrices. Moreover, an efficient hardware implementation of the forward and inverse approximate transform module is proposed. The architecture design includes a pipelined and reconfigurable forward-inverse DCT-II core transform as it is the main core for DST-VII and DCT-VIII computations. The proposed 32-point 1D architecture including low cost adjustment stages allows the processing of a video in 2K and 4K resolutions at 1095 and 273 frames per second, respectively. A unified 2D implementation of forward-inverse DCT-II, approximate DST-VII and DCT-VIII is also presented. The synthesis results show that the design is able to sustain a video in 2K and 4K resolutions at 386 and 96 frames per second, respectively, while using only 12% of Alms, 22% of registers and 30% of DSP blocks of the Arria10 SoC platform.

The next generation MPEG video coding standard is under development by the Joint Video Coding Experts Team (JVET). This new standard, called Versatile Video Coding (VVC), is expected by the end of 2020 and will offer better coding efficiency than its predecessor High Efficiency Video Coding (HEVC) standard. This coding gain is enabled by new coding tools such as more flexible block partitioning, more accurate Intra/Inter predictions, multiple transforms and adaptive in-loop filtering. In this paper we focus on the coding of the Intra Prediction Modes (IPM) that have been increased from 35 modes in HEVC to 67 modes in VVC. We propose a solution based on genetic algorithms to build an ordered list for the coding of IPM in the Joint Exploration Model (JEM) codec. We first give the theoretical upper bound performance in terms of required bits per IPM to encode the IPM using the available contextual information. The new ordering of the labels associated with more efficient codes is then proposed to efficiently leverage contextual informations available in the encoder and construct the Most Probable Modes (MPM) list. The proposed coding scheme enables to increase the BD-BR performance in average by 0.09% for the same level of complexity compared to the JEM.

An intra coding algorithm with layer separation is proposed. This algorithm is designed on top of an adopted tool in VVC, called Block DPCM (BDPCM), and benefits from texture information in a neighborhood to derive intensity levels of background and foreground layers. This information is used to reduce large rate of residual in case of incorrect layer prediction by BDPCM. For this purpose, three inter-layer transition states are defined that are either implicitly or explicitly conveyed to the decoder. Once a transition is signaled, the decoder corrects the prediction value using the derived layer information. Experiments on screen contents show a BD-rate gain of about 10% percent over VVC Test Model (VTM) and 1% over the regular BDPCM, with the cost of computational complexity.

Versatile Video Coding (VVC) is the next generation video coding standard expected by the end of 2020. The new concept of Multiple-Transform Selection (MTS) has been introduced in VVC. MTS enables the VVC encoder to select the transform that minimizes the rate-distortion cost among a set of pre-defined trigonometric transforms including the well known Discrete Cosine Transform (DCT)-II, DCT-VIII and Discrete Sine Transform (DST)-VII. Unlike the DCT-II that has fast computing algorithms, the DST-VII and DCT-VIII rely on more complex matrix multiplication.This paper tackles the problem of DST-VII and DCT-VIII approximations based on the DCT-II and an adjustment stage. This latter consists in a multiplication by a band-matrix with low number of non-zero coefficients per row. The approximation problem is first modeled as a constrained integer optimization problem minimizing both error and orthogonality. The genetic algorithm is then used to solve the optimization problem and find the adjustment band-matrix that minimizes a trade-off between error and orthogonality. The proposed solution enables to preserve the coding gain achieved by the MTS and considerably reduces the complexity in terms of required number of multiplications by coefficient. Moreover, the proposed approach is hardwarefriendly and will provide a lightweight shared hardware module for DST-II, DST-VII and DCT-VIII transforms.

Video compression and content quality have become one of the most research topic in the recent years. Predominantly, trends obviously signpost that the video usage over the Internet is on the upsurge. Simultaneously, users' requirement for enlarged resolution and higher quality is rising. Consequently, a huge effort has been made for video coding technologies and quality monitoring. In this paper, we present a subjective-based comparison as well as an objective measurement between the newest Versatile Video Coding (VVC) and the well-known High Efficiency Video Coding (HEVC) standards. Several videos of various content are selected as tested sequences. Both High Definition (HD) and Ultra High Definition (UHD) resolutions are used in this experiment. An extensive range of bit-rates from low to high bit-rates were selected. These sequences are encoded using both HEVC reference software (HM-16.2) and the latest reference software of VVC (VTM-5.0). Obtained results have shown that VVC outperforms consistently HEVC, for realistic bit rates and quality levels, in the range of 40% on the subjective scale. For the objective measurements, using PSNR, SSIM and VMAF as quality metrics, the quality enhancement of VVC over HEVC is ranging from 31% to 40%, depending on video content and spatial resolution.

Transform plays a crucial role in video coding, since it is in the transform domain that the bitrate reduction happens. Recently, coding with transform competition has emerged, especially inside the ITU-T and ISO/MPEG Joint Video Exploration Team, where the adaptive multiple transforms have been thoroughly tested. In this paper, it is proposed to investigate the design of transform competition systems for the purpose of inter-predicted residuals. Several systems are proposed and evaluated in terms of coding efficiency, storage requirement, and encoding complexity. A first approach selects the best transform for a residual block in a pre-established list of transforms. An extended version is also considered, where the number of transforms varies within an image area. Experimental results show that the proposed designs achieve significant bitrate savings over high efficiency video coding from -0.2% to -1.4%.

In October 2015, MPEG and ITU jointly launched the Joint Video Exploration Team (JVET) to prepare the next generation of video coding standard beyond HEVC. The tools added over HEVC, grouped into the Joint Exploration Model 3.0 (JEM-3.0), achieve over 25% of coding gains compared to HEVC for the Random-Access mode. In inter-coding mode, the transform stage consists in the Adaptive Multiple Transforms (AMT) which puts classical DCT-2 in competition with alternative transforms. In this paper, it is proposed to improve the AMT stage in inter-coding. The proposed method enables the encoder either to turn-off the AMT or to adjust its number of transforms. The introduced transforms are split into 3 nested sets: n°1 includes DST-7 and DCT-8; n°2 contains n°1 plus DST-7/DCT-8 combinations; n°3 contains n°2 plus 4 transforms learnt according to a Rate-Distortion Optimized Transform criterion (RDOT) [1]. The encoder performs a multiple-passes Coding-Tree-Unit (CTU) encoding to select the best set and signals its index into the bitstream, in a differential way. Acceleration strategies are implemented to mitigate the complexity, such as pass-cancellation and fast CTU partitioning from one pass to another.

The High Efficiency Video Coding (HEVC) standard defines 35 Intra Prediction Modes (IPM) to provide an efficient compression of intra coded blocks. Those IPMs are signalled to the decoder through the use of three compression tools: prediction, clustering and coding. In this paper we provide improvements to these three tools through: new labels for the prediction, new tests for the clustering and new coding schemes for the coding. The most significant improvement consists in the provision of a cluster-dependent code: adapting the coding scheme to the available information enables the average symbol cost to get within close margin of the entropy of the data. The system providing the best compression efficiency based on these improvements is then computed, enabling significant reduction in the average cost required to code the IPMs. The proposed method builds a new coding system with the same complexity as HEVC with 0.41% bit-rates savings in All Intra coding configuration.

With the drastic increasing of multimedia applications and video coarse consumption, video compression and content quality evaluation have become an exciting and challenging topic. Recently, a new coding tool has been developed under the Joint Exploration Model (JEM) software with the main goal to provide a high bit rate saving compared to the HEVC standard. In this paper we present a performance-based comparison between the JEM and HEVC reference software (HM) through an objective measurements and a subjective quality assessments. A set of video sequences, in two spatial resolutions High Definition (HD) and Ultra-High Definition (UHD), have been used in this study. These videos are encoded using both JEM and HM software at different bitrates. Results have shown that the JEM codec enables, subjectively, a quality enhancement up to 40% at similar low bit rates. Objectively, this quality improvement is ranging from 35% to 37% depending on the spatial resolution. However, at high bit rate, the HM reference software enables a high video quality and thus its becomes more difficult to perceive the quality enhancement is about brought by the JEM codec. In addition, some video contents are difficult to encode and, consequently, the JEM enables only slight perceived quality improvement especially at the highest considered bitrates and for 4K resolutions.

The new coding tools, developed under the Joint Exploration Model (JEM) software, have been proposed with the main goal to explore their potential coding gain in the perspective to develop a new video coding standard. In this paper we present a performance-based comparison between the JEM and HEVC reference software (HM) through a set of subjective quality assessments. Different video sequences, encoded using both JEM and HM software at different bitrates, have been used in this experiment. Results have shown that the JEM codec enables, subjectively, a quality enhancement up to 40% at similar low bit rates.

The use of multiple transforms in video coding can lead to substantial bit-rate savings. However, these savings come at the expense of increased coding complexity and storage requirements, which challenge the usability of this approach. In this paper, a systematic procedure is proposed to design low complexity systems making use of transform competition. Multiple trade-offs accommodating the complexity are unveiled and it is demonstrated that they can keep a certain level of performance. Compared to the HEVC standard, some of them provide bit-rate savings around 2% with a 50% increase in the encoding time, using less than 4 kB of extra ROM and no added decoding complexity.

Since the introduction of the H.261 standard in 1990, the transform step has always been a key feature of video coding technologies. The successive generations of standard have always found a better way to further de-correlate the residual signals and thus improve the coding efficiency. Recently, MPEG and the ITU have jointly launched the Joint Video Exploration Team (or JVET) to prepare the next generation of video coding standard. In this context, a new approach called Adaptive Multiple Transforms (or AMT) has been introduced. Despite the provided coding efficiency, the AMT solution raises several issues especially regarding the complexity of the selected set of transforms. This can be an important issue, particularly for a future industrial adoption. In this paper, a low-complexity transform set for AMT is proposed, which aims at providing a significant reduction of the required computational power. The proposed method has been implemented in the JEM-2.0 and tested under the JVET test conditions where it provides a substantial complexity reduction of -56.5% compared to the current AMT transform set with similar R-D performance.

The High Efficiency Video Coding (HEVC) standard defines 35 Intra Prediction Modes (IPM) to provide an efficient compression of intra coded blocks. To signal these IPMs to the decoder a list of 3 Most Probable Modes (MPM) is created based on the IPMs of the neighbour Intra coded blocks. These MPMs will be transmitted in the bit stream with a reduced number of bits. However, the signalling scheme used in HEVC is not optimal and can be further improved. In this paper a method is proposed to enhance the Intra mode signalling in HEVC. The IPM signaling is first modelled as a tree with forks and leaves representing tests and labels, respectively. The proposed solution introduces new decision tree process by adding new tests and new labels not considered in HEVC. This solution provides a systematic way to find the best signalling scheme for a given set of data. Experimental results show that the proposed solution enables to reduce the BD-Rate by 0.38% in all Intra coding configuration.

Transform coding plays a key role in state-of-the-art video coders, such as HEVC. However, transforms used in current solutions do not cover the varieties of video coding signals. This work presents an adaptive transform design method that enables the use of multiple transforms in HEVC. A different transform set is learnt for each intra prediction mode, allowing the video encoder to perform better decisions regarding block sizes, prediction modes and transforms. Different systems are proposed to accommodate trade-offs between complexity and performance. Bit rate reductions in the range of 2% to 7% are reported, depending on complexity.

Overcomplete transforms have received considerable attention over the past years. However, they often suffer from a complexity burden. In this paper, a low complexity approach is provided, where an orthonormal basis is complemented with a set of incomplete transforms: those incomplete transforms include a reduced number of basis vectors that allow a reduction on the coding complexity and ensure a certain level of sparsity. The solution has been implemented in the HEVC standard and compression gains of around 1% on average are reported while reducing the decoder complexity in about 5%.

Transform coding plays a crucial role in video coders. Recently, additional transforms based on the DST and the DCT have been included in the latest video coding standard, HEVC. Those transforms were introduced after a thoroughly analysis of the video signal properties. In this paper, we design additional transforms by using an alternative learning approach. The appropriateness of the design over the classical KLT learning is also shown. Subsequently, the additional designed transforms are applied to the latest HEVC scheme. Results show that coding performance is improved compared to the standard. Additional results show that the coding performance can be significantly further improved by using non-separable transforms. Bitrate reductions in the range of 2% over HEVC are achieved with those proposed transforms.

State of the art video coders are based on prediction and transform coding. The transform decorrelates the signal to achieve high compression levels. In this paper we propose improving the performances of the latest video coding standard, HEVC, by adding a set of rate-distortion optimised transforms (RDOTs). The transform design is based upon a cost function that incorporates a bit rate constraint. These new RDOTs compete against classical HEVC transforms in the rate-distortion optimisation (RDO) loop in the same way as prediction modes and block sizes, providing additional coding possibilities. Reductions in BD-rate of around 2% are demonstrated when making these transforms available in HEVC.

Cosine-modulated transforms such as the Modified Discrete Cosine Transform (MDCT) are key elements in audio coding. They allow efficient energy compaction and perceptual irrelevancy reduction. The frequency localization can be adapted to the signal characteristics and fast implementations exist. All of this has made MDCT the most popular transform in audio coding. In this paper we solve a problem never addressed in the literature: in order to enable variable MDCT sizes in communication codecs, we demonstrate how frequency resolution can be adapted on the fly without using transition windows hence decreasing coding delay. A low complexity implementation of the method is also proposed.

Probabilistic approaches can offer satisfactory solutions to source separation with a single channel, provided that the models of the sources match accurately the statistical properties of the mixed signals. However, it is not always possible to train such models. To overcome this problem, we propose to resort to an adaptation scheme for adjusting the source models with respect to the actual properties of the signals observed in the mix. In this paper, we introduce a general formalism for source model adaptation which is expressed in the framework of Bayesian models. Particular cases of the proposed approach are then investigated experimentally on the problem of separating voice from music in popular songs. The obtained results show that an adaptation scheme can improve consistently and significantly the separation performance in comparison with nonadapted models.

We study the application of wavelet packet filterbanks to low bit-rate transparent audio coding, taking the audio coders' delay requirements into account, and propose low-delay coders based on wavelet packet filterbanks. We first develop a method of comparison between filterbanks for perceptual audio coding by estimating the necessary bit-rate for a transparent compression. We use this comparison method in order to select the best filters for our audio compression scheme, from a large set of orthogonal and biorthogonal wavelets. Different wavelet filters may be used at different stages of the tree-structured decomposition with a constraint on the overall delay taken into account. The optimization is carried out with a simulated annealing procedure, proposing two wavelet packet filterbanks, exhibiting average and low delays. They are inserted in a complete audio coder that employs vector quantization and considers psychoacoustic models. The use of the proposed filterbanks leads to the design of a new bit allocation procedure, taking into account the lack of selectivity of the equivalent synthesis filters in a wavelet packet filterbank. The resulting audio scheme is validated through listening tests. The wavelet packet filterbanks are shown to be a promising tool for audio coding, especially for low-delay coding: with average delay, the quality of the wavelet packet filterbanks is as good as with MPEG-1 Layer-2, both with 80 kb/s, and when reducing the delay to 200 samples, 96 kb/s are needed to achieve the same quality.

We address the issue of choosing an optimal wavelet packets decomposition for low-delay audio bit-rate reduction. The decomposition tree and the choice of the wavelet filters for each 2-band cell of the tree are optimized. For a given filterbank we measure the entropy of the symbols resulting from a quantization stage ensuring transparent compression. This measure is the criterion to be minimized under a delay constraint. The optimization is carried out with simulated annealing techniques among a large family of filters. We show that there is no loss choosing the wavelet filters among a family of filters that have been optimized with respect to coding gain for AR(1) models. This provides efficient wavelet packets for low bit-rate low delay audio compression.

We address the issue of choosing an optimal wavelet packets transform for audio compression. We present a comparison method based on a perceptual approach, which provides an entropic bit-rate for "transparent" coding of a given audio signal. The test with different wavelets leads to the conclusion that the most significant synthesis criterion for audio compression is the so-called "coding gain", while frequency selectivity, regularity and orthogonality seem less relevant.