Adaptive live video streaming applications utilize a predefined collection of bitrate-resolution pairs, known as abitrate ladder, for simplicity and efficiency, eliminating the need for additional run-time to determine the optimal pairs during the live streaming session. These applications do not incorporate two-pass encoding methods due to increased latency. However, an optimized bitrate ladder could result in lower storage and delivery costs and improvedQuality of Experience(QoE). This paper presents a Just Noticeable Difference (JND)-aware constrained Variable Bitrate (cVBR) Two-pass Per-title encoding Scheme (JTPS) designed specifically for live video streaming. JTPS predicts a content- and JND-aware bitrate ladder using low-complexity features based onDiscrete Cosine Transform(DCT) energy and optimizes the constant rate factor (CRF) for each representation using random forest-based models. The effectiveness of JTPS is demonstrated using the open source video encoder x265, with an average bitrate reduction of 18.80% and 32.59% for the same PSNR and VMAF, respectively, compared to the standardHTTP Live Streaming(HLS) bitrate ladder using Constant Bitrate (CBR) encoding. The implementation of JTPS also resulted in a 68.96% reduction in storage space and an 18.58% reduction in encoding time for a JND of six VMAF points.

Optimization over trained machine learning models has applications including: verification, minimizing neural acquisition functions, and integrating a trained surrogate into a larger decision-making problem. This paper formulates and solves optimization problems constrained by trained graph neural networks (GNNs). To circumvent the symmetry issue caused by graph isomorphism, we propose two types of symmetry-breaking constraints: one indexing a node 0 and one indexing the remaining nodes by lexicographically ordering their neighbor sets. To guarantee that adding these constraints will not remove all symmetric solutions, we construct a graph indexing algorithm and prove that the resulting graph indexing satisfies the proposed symmetry-breaking constraints. For the classical GNN architectures considered in this paper, optimizing over a GNN with a fixed graph is equivalent to optimizing over a dense neural network. Thus, we study the case where the input graph is not fixed, implying that each edge is a decision variable, and develop two mixed-integer optimization formulations. To test our symmetry-breaking strategies and optimization formulations, we consider an application in molecular design.

For online analysis of the video content complexity in live streaming applications, selecting low-complexity features is critical to ensure low-latency video streaming without disruptions. To this light, for each video (segment), two features, i.e., the average texture energy and the average gradient of the texture energy, are determined. A DCT-based energy function is introduced to determine the block-wise texture of each frame. The spatial and temporal features of the video (segment) are derived from this DCT-based energy function. The Video Complexity Analyzer (VCA) project aims to provide an efficient spatial and temporal complexity analysis of each video (segment) which can be used in various applications to find the optimal encoding decisions. VCA leverages some of the x86 Single Instruction Multiple Data (SIMD) optimizations for Intel CPUs and multi-threading optimizations to achieve increased performance. VCA is an open-source library published under the GNU GPLv3 license.

The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audiovisual content accounts for the majority of today's internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs. This paper provides a practical evaluation of state of the art video codecs (i. e., AV1, AVC/libx264, HEVC/libx265, VP9/Iibvpx-vp9) for large-scale HTTP adaptive streaming services. In anticipation of the results, AV I shows promising performance compared to established video codecs. Additionally, AV I is intended to be royalty free making it worthwhile to be considered for large scale HTTP adaptive streaming services.

The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audio-visual content accounts for the majority of today's internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs.

In the current standardization process of the scalable extension to High Efficiency Video Coding (SHVC) a high level syntax multi-loop approach is close to completion. On the one hand this multi-loop approach offers a reasonable rate-distortion performance while only minimal modifications to the encoder and decoder in both layers are required. On the other hand this approach requires full reconstruction of all pictures of all layers at the decoder side which, in the case of quality scalability with two layers, doubles the decoder complexity. In this paper high layer modifications to the prediction structure similar to the scalable extension of H.264 - AVC are implemented in SHVC and studied. These modifications allow for an enhancement layer decoder implementation to skip a significant amount of motion compensation and deblocking operations in the base layer. It is shown that the decoder complexity can hereby be reduced up to 55% for the random access configuration and up to 64% for the low delay configuration compared to SHVC. An overall coding performance increase of 1.2% when decoding the enhancement layer is reported while when only decoding the base layer a drift can be observed between -0.16 dB for random access and -0.39 dB for low delay.

In scalable video coding, a stream contains multiple layers of different temporal resolution, spatial resolution, or quality. Each enhancement layer can utilize the information from the lower layers in order to improve the overall coding efficiency. In the draft scalable extension of HEVC the enhancement layer uses the lower layer reconstructed pixel values for prediction. While this dual-loop approach is efficient in terms of rate-distortion performance, it doubles the decoder complexity. In this paper a method is presented that allows for single-loop decoding using a new coefficient refinement scheme for the enhancement layer. The scheme conceptually allows for binary mapping between arbitrary quantizer step sizes in base and enhancement layer. Simulation results reveal a promising rate-distortion performance, especially for small quantizer deltas.

Upcoming display technologies like auto stereoscopic displays require synthesis of virtual viewpoints to allow for depth impression without requiring the viewer to wear glasses. View synthesis relies on the availability of depth information at the receiver side to warp the recorded video data to arbitrary viewpoints. This depth data contains information about the 3D position of recorded pixel values and therefore inherently also contains the pixel-wise disparity information between multiple views to be coded. In this paper we propose to use this depth information also for compression of multi view video+depth sequences. By warping the base view's frame of a multi view video to the position of the enhancement views' camera a prediction signal is formed, which can either be used for the proposed warped skip mode or as a prediction signal to reduce the residual's energy.

In this paper, we improve on our previous work regarding component-based image coding, a hybrid transform-based/perceptual image coding scheme based on a decomposition of the image into structure and texture characterized by a Gaussian Markov random field. The 2D Itakura distance allows us to evaluate the performance of our texture model in terms of rate vs. distortion. A minimal quantization step size for near-lossless coding of model parameters is determined. Furthermore, we show that texture contrast can be efficiently coded using transform-based techniques.

A modified Viterbi (1971) algorithm for convolutional codes is described that provides for signal-to-noise ratio (SNR) adaptive computational effort. The algorithm has three levels of prioritized effort. Movement from one level to the next is controlled by parameters that can be selected according to desired output bit error rate performance. For 3-bit soft decision detected signals, a coding gain within 0.06 dB of Viterbi at a 3-dB SNR is achieved for the same constraint-length code with modest parameter values and computational effort. At values of SNR above 6 dB, the algorithm decodes with very low computational effort. Effort levels are controlled by spanning the decoding trellis in steps that are one constraint-length long.