Distributed learning requires a frequent communication of neural network update data. For this, we present a set of new compression tools, jointly called differential neural network coding (dNNC). dNNC is specifically tailored to efficiently code incremental neural network updates and includes tools for federated BatchNorm folding (FedBNF), structured and unstructured sparsification, tensor row skipping, quantization optimization and temporal adaptation for improved context-adaptive binary arithmetic coding (CABAC). Furthermore, dNNC provides a new parameter update tree (PUT) mechanism, which allows to identify updates for different neural network parameter sub-sets and their relationship in synchronous and asynchronous neural network communication scenarios. Most of these tools have been included into the standardization process of the NNC standard (ISO/IEC 15938-17) edition 2. We benchmark dNNC in multiple federated and split learning scenarios using a variety of NN models and data including vision transformers and large-scale ImageNet experiments: It achieves compression efficiencies of 60% in comparison to the NNC standard edition 1 for transparent coding cases, i.e., without degrading the inference or training performance. This corresponds to a reduction in the size of the NN updates to less than 1% of their original size. Moreover, dNNC reduces the overall energy consumption required for communication in federated learning systems by up to 94%.

Data-driven optimization is employed to study alternative approaches [1] to the probability estimator of the the Enhanced Compression Model (ECM) (which includes additional coding tools on top of the Versatile Video Coding standard). In ECM, each context model uses a weighted sum of two hypotheses for probability estimation with different associated adaptation rates. Four alternative approaches are studied:

This paper presents an improved probability estimation scheme for the entropy coder of Incremental Neural Network Coding (INNC), which is currently under standardization in ISO/IEC MPEG. More specifically, the paper first analyzes the compression performance of INNC and how the bitstream size relates to the neural network (NN) layers. For the layers requiring the most bits, it analyzes the coded NN weight updates and their temporal dependencies. Major finding is that the probability of a significant (i.e., non-zero) update for a weight can depend considerably on whether the weight has been updated before. Based on this finding, the paper proposes a new probability estimation scheme: Depending on whether a significant update has been received before (i.e., based on the weight’s history), the entropy coder models the probability for a current significant update differently. This scheme achieves a bitstream size reduction of about 2% and 1% in a transfer and a federated learning scenario, respectively, without any accuracy loss or significant complexity increase. Therefore, MPEG adopted our history dependent significance probability (HDSP) scheme to its emerging standard for INNC.

Neural Network Coding and Representation (NNR) is the first international standard for efficient compression of neural networks (NNs). The standard is designed as a toolbox of compression methods, which can be used to create coding pipelines. It can be either used as an independent coding framework (with its own bitstream format) or together with external neural network formats and frameworks. For providing the highest degree of flexibility, the network compression methods operate per parameter tensor in order to always ensure proper decoding, even if no structure information is provided. The NNR standard contains compression-efficient quantization and deep context-adaptive binary arithmetic coding (DeepCABAC) as core encoding and decoding technologies, as well as neural network parameter pre-processing methods like sparsification, pruning, low-rank decomposition, unification, local scaling and batch norm folding. NNR achieves a compression efficiency of more than 97% for transparent coding cases, i.e. without degrading classification quality, such as top-1 or top-5 accuracies. This paper provides an overview of the technical features and characteristics of NNR.

The novel Neural Network Compression and Representation Standard (NNR), recently issued by ISO/IEC MPEG, achieves very high coding gains, compressing neural networks to 5% in size without accuracy loss. The underlying NNR encoder technology includes parameter quantization, followed by efficient arithmetic coding, namely DeepCABAC. In addition, NNR also allows very flexible adaptations, such as signaling specific local scaling values, setting quantization parameters per tensor rather than per network and supporting specific parameter fusion operations. This paper presents our new approach for optimally deriving these parameters, namely the derivation of parameters for local scaling adaptation (LSA), inference-optimized quantization (IOQ), and batch-norm folding (BNF). By allowing inference and fine tuning within the encoding process, quantization errors are reduced and the NNR coding efficiency is further improved to a compressed bitstream size of only 3% in comparison to the original model size.

In this paper we present a "State-Based Multi-Parameter Probability Estimation" (SBMP) for Context-Based Adaptive Binary Arithmetic Coding (CABAC) which employs a two hypotheses probability estimator based on exponentially weighted moving averages. It uses a logarithmic state representation and a single subsampled transition table with only 32 elements for the probability update. This reduces the memory requirements virtually without affecting the compression efficiency, compared to corresponding approaches that use a linear state representation and a computation-based probability update. The proposed scheme is based on simple operations like table look-ups and additions. Compared to the state-of-the-art probability estimator of the video compression standard H.265/HEVC, the compression efficiency is increased by up to 1 % Bjøntegaard-Delta bit rate (BD rate) when applied to draft 2 of the Versatile Video Coding (VVC) standard. Furthermore, SBMP was recently adopted to working draft 2 of the MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis.

Recent approaches to compression of deep neural networks, like the emerging standard on compression of neural networks for multimedia content description and analysis (MPEG-7 part 17), apply scalar quantization and entropy coding of the quantization indexes. In this paper we present an advanced method for quantization of neural network parameters, which applies dependent scalar quantization (DQ) or trellis-coded quantization (TCQ), and an improved context modeling for the entropy coding of the quantization indexes. We show that the proposed method achieves 5.778% bitrate reduction and virtually no loss (0.37%) of network performance in average, compared to the baseline methods of the second test model (NCTM) of MPEG-7 part 17 for relevant working points.

An increasing number of distributed machine learning applications require efficient communication of neural network parameterizations. DeepCABAC, an algorithm in the current working draft of the emerging MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis, has demonstrated high compression gains for a variety of neural network models. In this paper we propose a method for employing DeepCABAC in a Federated Learning scenario for the exchange of intermediate differential parameterizations. Furthermore, we discuss the efficiency of DeepCABAC when compressing trained neural networks. Our experiments on large neural networks show that in both scenarios, DeepCABAC achieves competitive compression rates, without degrading the network accuracy.

For the entropy coding of independent and identically distributed (i.i.d.) binary sources, variable-to-variable length (V2V) codes are an interesting alternative to arithmetic coding. Such a V2V code translates variable length words of the source into variable length code words by employing two prefix-free codes. In this article, several properties of V2V codes are studied, and new concepts are developed. In particular, it is shown that the redundancy of a V2V code cannot be zero for a binary i.i.d. source {X} with 0 <pX(1) < 0.5. Furthermore, the concept of prime and composite V2V codes is proposed, and it is shown why composite V2V codes can be disregarded in the search for particular classes of minimum redundancy codes. Moreover, a canonical representation for V2V codes is proposed, which identifies V2V codes that have the same average code length function. It is shown how these concepts can be employed to greatly reduce the complexity of a search for minimum redundancy (size-limited) V2V codes.

The state-of-the-art hybrid video coding standard H.264/AVC defines two entropy-coding schemes with different complexity-performance trade-offs. Supporting these two schemes within a single standard raises several problems ranging from higher efforts for product development to increased silicon costs for hardware implementations. To overcome these issues, this work proposes a unified and complexity-scalable entropy-coding framework that is based on PIPE/V2V. The proposed framework uses a single set of tools for all configurations and achieves the same complexity-performance trade-offs as the existing entropy-coding schemes through scalability.

In hybrid video coding, an entropy coding scheme transmits the quantized transform coefficients, resulting from block-based transformation and quantization of the difference between the prediction signal and the original signal, and additional side information. The state-of-the-art hybrid video coding standard H.264/AVC defines two different entropy coding schemes with different complexity-performance tradeoff. As a result, the support for two different entropy coding schemes has to be maintained and introduces several problems. To overcome these issues, a unified solution is proposed, which is based on the PIPE/V2V coding concept. It achieves the same complexity-performance trade-offs as the existing entropy coding schemes by scalability. The advantage of the proposed scheme over the existing concept is the usage of the same set of tools for all configurations. Simulation results and complexity analysis on hardware show the efficiency of the proposed scheme.

The recently emerging probability interval partitioning entropy (PIPE) coding scheme offers high coding efficiency at a comparably low complexity level. In this paper, a new set of systematic variable-to-variable length (v2v) codes is proposed for use within the PIPE coding concept that allows the complexity requirements to be reduced even further. The proposed systematic v2v codes can be efficiently implemented by using simple counters instead of memory consuming tables. At the same time, the average number of operations per decoded binary symbol can be reduced by more than a factor of 2 relative to a fast multiplication-free binary arithmetic decoder. In terms of coding efficiency, experimental results show that in a typical video coding environment the average Bjontegaard delta (BD) rate increase is typically less than 0.5% when compared to the use of nearly optimal binary arithmetic codes.

This paper describes a novel video coding scheme that can be considered as a generalization of the block-based hybrid video coding approach of H.264/AVC. While the individual building blocks of our approach are kept simple similarly as in H.264/AVC, the flexibility of the block partitioning for prediction and transform coding has been substantially increased. This is achieved by the use of nested and pre-configurable quadtree structures, such that the block partitioning for temporal and spatial prediction as well as the space-frequency resolution of the corresponding prediction residual can be adapted to the given video signal in a highly flexible way. In addition, techniques for an improved motion representation as well as a novel entropy coding concept are included. The presented video codec was submitted to a Call for Proposals of ITU-T VCEG and ISO/IEC MPEG and was ranked among the five best performing proposals, both in terms of subjective and objective quality.

Recent investigations have shown that the support of extended block sizes for motion-compensated prediction and transform coding can significantly increase the coding efficiency for high-resolution video relative to H.264/AVC. In this paper, we present a new context-modeling scheme for the coding of transform coefficient levels that is particularly suitable for transform blocks greater than 8 × 8. While the basic concept for transform coefficient coding is similar to CABAC, the probability model selection has been optimized for larger block transforms. The proposed context modeling is compared to a straightforward extension of the CABAC context modeling; both schemes have been implemented in a hybrid video codec design that supports block sizes of up to 128 × 128 samples. In our simulations, we obtained overall bit rate reductions of up to 4%, with an average of 1.7% with the proposed context modeling scheme.

Abstract-A video coding architecture is described that is based on nested and pre-configurable quadtree structures for flexible and signal-adaptive picture partitioning. The primary goal of this partitioning concept is to provide a high degree of adaptability for both temporal and spatial prediction as well as for the purpose of space-frequency representation of prediction residuals. At the same time, a leaf merging mechanism is included in order to prevent excessive partitioning of a picture into prediction blocks and to reduce the amount of bits for signaling the prediction signal. For fractional-sample motion-compensated prediction, a fixed-point implementation of the maximal-order minimum-support algorithm is presented that uses a combination of infinite impulse response and FIR filtering. Entropy coding utilizes the concept of probability interval partitioning entropy codes that offers new ways for parallelization and enhanced throughput. The presented video coding scheme was submitted to a joint call for proposals of ITU-T Visual Coding Experts Group and ISO/IEC Moving Picture Experts Group and was ranked among the five best performing proposals, both in terms of subjective and objective quality.

For compression of 3-D dynamic meshes, the novel framework of so-called frame-based animated mesh compression (FAMC) has been introduced recently. In this context, we propose an efficient scheme for representation and statistical coding which is conceptually based on our previous work on context-based adaptive binary arithmetic coding (CABAC). After reviewing the basic principles of both CABAC and FAMC, we present suitable modifications and adaptations of both concepts in order to build an integrated solution with a high degree of coding efficiency. To this end, particular focus of our study has been put on the design of appropriate binarization and context modeling schemes. In our experiments, we obtained average bit-rate savings of more than 30% for a typical test set of dynamic meshes, when comparing the final design of our CABAC enriched FAMC scheme to the original version of FAMC using a conventional N-ary arithmetic coder. Our integrated approach has been adopted recently as part of the MPEG-4 Animated Framework extension (AFX).

Context-based adaptive binary arithmetic coding (CABAC) has proven to be an efficient technique in the area of video coding. This paper presents an approach for integrating CABAC into the framework of frame-based animated mesh compression (FAMC). It presents the specific modifications and adaptations that have been worked out to adapt CABAC to the specific requirements of FAMC in order to build a solution with a higher degree of coding efficiency. For a typical test set of animated meshes, average bit rate savings of 25% have been observed for the combination of CABAC and FAMC when compared to a previous version of FAMC using a conventional N-ary arithmetic coder. The presented approach has been recently adopted as part of the MPEG-4 AFX standard.

Fidelity scalability involves the refinement of residual texture information. The entropy coding of texture refinement information in the scalable video coding (SVC) extension of H.264/AVC relies on a simple statistical model that is tuned to an encoder-specific way of quantization for generating a single fidelity enhancement layer on top of the backward compatible base layer. For fidelity enhancement layers above the first layer, we demonstrate how and why the current model fails to properly reflect the statistics of texture refinement information. By analyzing the specific properties of the typical quantization process in fidelity scalable coding of SVC, we are able to derive a generic modeling approach for coding of refinement symbols, independent of the specific choice of dead-zone parameters and classification rules. Experimental results for a broad range of quantization parameters show averaged bit-rate savings of around 5% (relative to the total bit rate) by using our proposed context modeling approach for a representative set of video sequences in a test scenario including up to three fidelity enhancement layers.

Block-based video-coding for 4:4:4 color sampling is extended by an adaptive color space transform. The presented technique enables an encoder to switch between several given color space representations in order to optimize rate-distortion performance. Simulations based on the current draft of the H.264/MPEG4-AVC 4:4:4 extensions demonstrate that our technique provides a rate-distortion performance equal or better than that obtained when using any of the individual color spaces only.