Coding of Gaussian splats has drawn the attention of academia and standardization bodies lately. A commonly used approach involves projecting explicit 3D attributes onto 2D planes of a video and compressing it with existing video coding standards. However, such an approach produces excessively high sample rates, resulting in a significant bottleneck that might make it unsuitable for existing hardware decoders if the number of splats is high. This paper presents a framework that utilizes an implicit representation, where Gaussian splat attributes are represented by compact feature planes. By reducing the dependency of video resolution on the number of splats, our approach significantly reduces the required sample rate, making it suitable for deployed devices. In addition, the proposed framework achieves a superior rate-distortion trade-off, providing high-fidelity reconstruction at low bitrates without the excessive sample rates associated with the conventional video-based anchor.

This work investigates video encoding parallelization techniques based on the Versatile Video Coding (VVC) standard, using the open and optimized encoder software implementation VVenC. Modern multi-processor systems offer significant opportunities for accelerating video encoding. By employing a proposed combination of parallelization methods, the VVenC encoder achieves an acceleration factor of up to 22 compared to single-threaded mode on a 32-core system, with potential increases to 27× at higher bitrates. Building upon prior work on Inter-frame Parallelization (IFP), the study introduces frame region-based synchronization, enabling further acceleration of up to 10%. Beyond that, the study demonstrates extending frame parallelization beyond Group of Pictures (GOP) boundaries, which improves IFP speed up by 37% and 11% at high-definition (HD) and ultra-high-definition (UHD) resolutions, respectively. Additional combinations with other VVC parallelization tools, such as tiles and VVC Wavefront Parallel Processing (WPP), are also explored. The article provides a comprehensive analysis of parallelization challenges and highlights areas for further improvement.

3D Gaussian Splatting is a recognized method for 3D scene representation, known for its high rendering quality and speed. However, its substantial data requirements present challenges for practical applications. In this paper, we introduce an efficient compression technique that significantly reduces storage overhead by using compact representation. We propose a unified architecture that combines point cloud data and feature planes through a progressive tri-plane structure. Our method utilizes 2D feature planes, enabling continuous spatial representation. To further optimize these representations, we incorporate entropy modeling in the frequency domain, specifically designed for standard video codecs. We also propose channel-wise bit allocation to achieve a better trade-off between bitrate consumption and feature plane representation. Consequently, our model effectively leverages spatial correlations within the feature planes to enhance rate-distortion performance using standard, non-differentiable video codecs. Experimental results demonstrate that our method outperforms existing methods in data compactness while maintaining high rendering quality. Our project page is available at https://fraunhoferhhi.github.io/CodecGS

Explicit feature-grid based NeRF models have shown promising results in terms of rendering quality and significant speed-up in training. However, these methods often require a significant amount of data to represent a single scene or object. In this work, we present a compression model that aims to minimize the entropy in the frequency domain in order to effectively reduce the data size. First, we propose using the discrete cosine transform (DCT) on the tensorial radiance fields to compress the feature-grid. This feature-grid is transformed into coefficients, which are then quantized and entropy encoded, following a similar approach to the traditional video coding pipeline. Furthermore, to achieve a higher level of sparsity, we propose using an entropy parameterization technique for the frequency domain, specifically for DCT coefficients of the feature-grid. Since the transformed coefficients are optimized during the training phase, the proposed model does not require any fine-tuning or additional information. Our model only requires a lightweight compression pipeline for encoding and decoding, making it easier to apply volumetric radiance field methods for real-world applications. Experimental results demonstrate that our proposed frequency domain entropy model can achieve superior compression performance across various datasets.

The Versatile Video Coding (VVC) standard promises high compression efficiency for diverse content types. Based on VVenC, an open and optimized VVC software video encoder, this work presents an inter-frame parallelization (IFP) method designed to exploit the processing power of modern platforms featuring a high number of computing cores. Encoding an ultrahigh definition video on a 32-core machine with the VVenC's faster preset, the proposed method shows more than 20% increase in encoder speed while only a 1% decrease in compression efficiency compared to the default multi-threading mode. In comparison to single-threaded mode, it corresponds to a speedup factor of 18, up from 15x achievable with the previous parallelization scheme. Furthermore, the synergy of the developed inter-frame parallelization technique with other parallelization methods is explored, including tiles and VVC wavefront parallel processing (WPP). The combination of these approaches enables a notable speedup factor of 21, albeit with a trade-off in coding efficiency. With a focus on VVC, this research contributes to the ongoing discourse on video coding optimization, providing valuable insights into possible pitfalls and the potential gains achievable through efficient parallelization techniques on high-core platforms.

In this work, we propose novel HARQ prediction schemes for Cloud RANs (C-RANs) that use feedback over a rate-limited feedback channel (2 - 6 bits) from the Remote Radio Heads (RRHs) to predict at the User Equipment (UE) the decoding outcome at the BaseBand Unit (BBU) ahead of actual decoding. In particular, we propose a Dual Autoencoding 2-Stage Gaussian Mixture Model (DA2SGMM) that is trained in an end-to-end fashion over the whole C-RAN setup. Using realistic link-level simulations in the sub-THz band at 100 GHz, we show that the novel DA2SGMM HARQ prediction scheme clearly outperforms all other adapted and state-of-the-art schemes. The DA2SGMM shows a superior performance in terms of blockage detection as well as HARQ prediction in the no-blockage and single-blockage cases. In particular, the DA2SGMM with 4 bit feedback achieves a more than 200 % higher throughput in average compared to its best alternative. Compared to regular HARQ, the DA2SGMM reduces the maximum transmission latency by more than 72.4 %, while maintaining more than 75 % of the throughput in the no-blockage scenario. In the single-blockage scenario, DA2SGMM significantly increases the throughput for most of the evaluated Signal-to-Noise-Ratios (SNRs) compared to regular HARQ.

In recent years, applications such as cloud gaming and virtual video conferencing have gained increasing popularity and new applications, such as immersive applications, have emerged that require very low latency in order to guarantee quality of service. Such applications can benefit from using the Low Latency, Low Loss, Scalable Throughput (L4S) specification that is currently being defined by the IETF. This paper presents a congestion control algorithm aiming at achieving a target latency over a 5G connection, which together with L4S can be used in delay-critical applications. The described algorithm has been implemented into WebRTC and uses ECN marking to adapt its sending rate. The developed algorithm has been compared to the Google Congestion Control (GCC) as a baseline on various data rate patterns and shown to be more responsive to throughput variations more successfully avoiding latency spikes that surpass the acceptable latency.

We provide a mathematical framework to analyze the limits of Hybrid Automatic Repeat reQuest (HARQ) and derive analytical expressions for the most powerful test for estimating the decodability under maximum-likelihood decoding and t-error decoding. Furthermore, we numerically approximate the most powerful test for sum-product decoding. We compare the performance of previously studied HARQ prediction schemes and show that none of the state-of-the-art HARQ prediction is most powerful to estimate the decodability of a partially received signal vector under maximum-likelihood decoding and sum-product decoding. Furthermore, we demonstrate that decoding in general is suboptimal for predicting the decodability.

In this paper, we investigate short packet transmissions in a Polar Coded Sparse Code Multiple Access (PC-SCMA) system. Since recently introduced by Arikan Polarization-Adjusted Convolutional (PAC) codes have shown remarkable performance at short block lengths with an ability of reaching theoretical limits, in this work we propose a PC-SCMA system based on the soft output PAC decoder. The proposed decoder is a combination of a sequential stack decoder and an enhanced Belief Propagation (BP) algorithm. The numerical analysis shows its ability to outperform the state-of-the-art PC-SCMA system with a Successive Cancellation List (SCL) decoder at short block lengths in terms of the bit error rate with smaller computational complexity. Furthermore, we enhance the SCL-based scheme by row-merging of the polarization kernel and evaluate its performance.

In this work, we extend two autoencoder-based HARQ prediction schemes to exploit subcode-based features and SNR-based features jointly. We apply the proposed HARQ prediction schemes to Cloud-RAN (C-RAN) and Single-RAN (S-RAN) architectures. Furthermore, we conduct realistic link-level simulations to test the performance and compare to state-of-the-art prediction schemes that rely solely on either subcode-based features or SNR-based features. Compared to the state-of-the-art, we show that the proposed schemes reduce the transmitted redundancy at a target error rate of$5\cdot 10^{-5}$and$2\cdot 10^{-5}$by 12.3% - 27.3% in C-RAN architectures and 10.5% - 11.0% in S- RAN architectures, respectively.

Independently coding areas in a video have potential for improved coding efficiency in Region-of-Interest (RoI) streaming applications where clients can freely switch between RoI and full view streams. This paper investigates three schemes for RoI coding in streaming scenarios based on VVC to enable harnessing open GOP coding efficiency gains. Two schemes are based on Motion-Constrained-Tile-Sets while a third scheme relies on the new VVC feature referred to as independent subpictures. The reported results show substantial bitrate gains for the RoI streams compared to naïve closed-GOP coding of up to -8.68% YUV BD-rate while for the full stream minor coding efficiency losses are reported.

In our MM'20 paper,, we presented a Kalman filter-based approach for prediction of head motion in 6DoF. The proposed approach was employed in our cloud-based volumetric video streaming system to reduce the interaction latency experienced by the user. In this companion paper, we present the dataset collected for our experiments and our simulation framework that reproduces the obtained experimental results. Our implementation is freely available on Github to facilitate further research.

The user experience in adaptive HTTP streaming relies on offering bitrate ladders with suitable operation points for all users and typically involves multiple resolutions. While open GOP coding structures are generally known to provide substantial coding efficiency benefit, their use in HTTP streaming has been precluded through lacking support of reference picture resampling (RPR) in AVC and HEVC. The newly emerging Versatile Video Coding (VVC) standard supports RPR, but only conversational scenarios were primarily investigated during the design of VVC. This paper aims at enabling usage of RPR in HTTP streaming scenarios through analysing the drift potential of VVC coding tools and presenting a constrained encoding method that avoids severe drift artefacts in resolution switching with open GOP coding in VVC. In typical live streaming configurations, the presented method achieves -8.7% BD-rate reduction compared to closed GOP coding while in a typical Video on Demand configuration, -1.89% BD-rate reduction is reported. The constraints penalty compared to regular open GOP coding is 0.65% BD-rate in the worst case. The presented method was integrated into the publicly available open source VVC encoder VVenC v0.3.

In this work, we investigate proactive Hybrid Automatic Repeat reQuest (HARQ) using link-level simulations for multiple packet sizes, modulation orders, BLock Error Rate (BLER) targets and two delay budgets of 1 ms and 2 ms, in the context of Industrial Internet of Things (IIOT) applications. In particular, we propose an enhanced proactive HARQ protocol using a feedback prediction mechanism. We show that the enhanced protocol achieves a significant gain over the classical proactive HARQ in terms of energy efficiency for almost all evaluated BLER targets at least for sufficiently large feedback delays. Furthermore, we demonstrate that the proposed protocol clearly outperforms the classical proactive HARQ in all scenarios when taking a processing delay reduction due to the less complex prediction approach into account, achieving an energy efficiency gain in the range of 11% up to 15% for very stringent latency budgets of 1 ms at 10-2BLER and from 4% up to 7.5% for less stringent latency budgets of 2 ms at 10-3BLER. Furthermore, we show that power-constrained proactive HARQ with prediction even outperforms unconstrained reactive HARQ for sufficiently large feedback delays.

Streaming of high-resolution 360-degree video is typically done in a viewport-dependent fashion such as in the tile-based viewport-dependent profile of MPEG OMAF wherein clients continuously adapt their tile selection according to the user viewport. From the perspective of a streaming service operator, tile rate assignment is crucial to ensure that a given target bitrate is obeyed while quality distribution among tiles leads to a favorable user experience. This paper addresses rate assignment in a distributed tile encoding system for such multi-resolution tiled streaming services based on the emerging Versatile Video Coding Standard. A model for rate assignment is derived based on random forest regression using spatio-temporal activity and encodings of a learning dataset.

Volumetric video allows viewers to experience highly-realistic 3D content with six degrees of freedom in mixed reality (MR) environments. Rendering complex volumetric videos can require a prohibitively high amount of computational power for mobile devices. A promising technique to reduce the computational burden on mobile devices is to perform the rendering at a cloud server. However, cloud-based rendering systems suffer from an increased interaction (motion-to-photon) latency that may cause registration errors in MR environments. One way of reducing the effective latency is to predict the viewer's head pose and render the corresponding view from the volumetric video in advance.

Volumetric video is an emerging technology for immersive representation of 3D spaces that captures objects from all directions using multiple cameras and creates a dynamic 3D model of the scene. However, processing volumetric content requires high amounts of processing power and is still a very demanding task for today's mobile devices. To mitigate this, we propose a volumetric video streaming system that offloads the rendering to a powerful cloud/edge server and only sends the rendered 2D view to the client instead of the full volumetric content. We use 6DoF head movement prediction techniques, WebRTC protocol and hardware video encoding to ensure low-latency in different parts of the processing chain. We demonstrate our system using both a browser-based client and a Microsoft HoloLens client. Our application contains generic interfaces that allow for easy deployment of various augmented/mixed reality clients using the same server implementation.

Volumetric video is an emerging key technology for immersive representation of 3D spaces and objects. Rendering volumetric video requires lots of computational power which is challenging especially for mobile devices. To mitigate this, we developed a streaming system that renders a 2D view from the volumetric video at a cloud server and streams a 2D video stream to the client. However, such network-based processing increases the motion-to-photon (M2P) latency due to the additional network and processing delays. In order to compensate the added latency, prediction of the future user pose is necessary. We developed a head motion prediction model and investigated its potential to reduce the M2P latency for different look-ahead times. Our results show that the presented model reduces the rendering errors caused by the M2P latency compared to a baseline system in which no prediction is performed.

360° video streaming has drawn a lot of attention in the last years. In order to provide a good quality of experience with the existing device capability and available bandwidths several viewport dependent techniques have been investigated and developed. One of these techniques, as specified in MPEG OMAF, is to use tile-based streaming, with the 360° video being offered in a tiled manner at various resolutions. Thus, each user can retrieve the tiles at different resolutions, thereby selecting the tiles that match its viewport at high-resolution. This paper provides an analysis on different configurations for tile-based 360° video streaming, comparing the quality that can be achieved by using them for different values of end-to-end delays.

Virtual Reality (VR) and 360-degree video streaming have gained significant attention in recent years. First standards have been published in order to avoid market fragmentation. For instance, 3GPP released its first VR specification to enable 360-degree video streaming over 5G networks which relies on several technologies specified in ISO/IEC 23090-2, also known as MPEG-OMAF. While some implementations of OMAF-compatible players have already been demonstrated at several trade shows, so far, no web browser-based implementations have been presented. In this demo paper we describe a browser-based JavaScript player implementation of the most advanced media profile of OMAF: HEVC-based viewport-dependent OMAF video profile, also known as tile-based streaming, with multi-resolution HEVC tiles. We also describe the applied workarounds for the implementation challenges we encountered with state-of-the-art HTML5 browsers. The presented implementation was tested in the Safari browser with support of HEVC video through the HTML5 Media Source Extensions API. In addition, the WebGL API was used for rendering, using region-wise packing metadata as defined in OMAF.

We investigate Early Hybrid Automatic Repeat reQuest (E-HARQ) feedback schemes enhanced by machine learning techniques as a path towards ultra-reliable and low-latency communication (URLLC). To this end, we propose machine learning methods to predict the outcome of the decoding process ahead of the end of the transmission. We discuss different input features and classification algorithms ranging from traditional methods to newly developed supervised autoencoders. These methods are evaluated based on their prospects of complying with the URLLC requirements of effective block error rates below 10-5at small latency overheads. We provide realistic performance estimates in a system model incorporating scheduling effects to demonstrate the feasibility of E-HARQ across different signal-to-noise ratios, subcode lengths, channel conditions and system loads, and show the benefit over regular HARQ and existing E-HARQ schemes without machine learning.

Tile-based video systems have recently emerged as a viable solution to overcome the challenges of 360-degree video. For instance, the HEVC based viewport-dependent profile of MPEG OMAF allows serving clients independently coded tiles of the 360-degree video at varying resolution to enhance fidelity within the actual user viewport. During streaming, the client constantly adapts its tile selection and feeds a single merged bitstream to the video decoder. This paper addresses the open issue of rate assignment in a distributed encoding system in such a multi-resolution tiled streaming scenario. A model for tile rate assignment based on the spatio-temporal activity of the video is presented to reduce variance of the quality distribution and experimental results are reported.

360° video streaming to clients using Virtual Reality head mounted displays is a challenge for traditional video delivery. As transmission of the complete content in a desirable quality sacrifices a large fraction of available client and network resources, adaptivity to the user viewport promises substantial benefits. An efficient way to achieve viewport adaptive streaming without per-user or per-orientation encoding, i.e. essentially transcoding, is to make use of motion-constrained HEVC tiles. DASH can be used for tiled streaming, where tiled content resides on the server at multiple resolutions. The DASH client selects the resolutions of each tile according to the current viewport. This demonstration paper presents an agile and responsive streaming prototype system for 360° video content. In order to achieve acceptable responsiveness, the DASH client relies on small buffer sizes and shifted random access points across tiles when suitable.

360° video streaming introduces stricter requirements to the established transmission chain than in traditional streaming services. Transmission of the complete 360° video in desirable quality can lead to multiple times UHD resolution which would waste a large fraction of network resources as the larger fraction of the video is not presented on the end device. Adaptivity to the user viewport promises substantial benefits and contrary to per-user or per-orientation encoding, tiled based streaming provides a scalable solution. We consider tiled streaming of 360° video using the cubic projection, where tiled content resides on the server at 2 different resolutions. User traces have been collected for a range of content and based on consumption patterns, tiles at the server have been optimized. More concretely, the paper optimizes the Random Access Point period that leads to the minimum transmitted bitrate, while ensuring that users watch most of the time high resolution content.

360 degree video streaming has gained much interest recently. The Omnidirectional MediA Format (OMAF) standard, currently in development by the Moving Picture Experts Group (MPEG), standardizes means for storage and delivery of 360 degree coded video based on a well-established standards ecosystem. This demonstration system uses the viewport-dependent OMAF media profile in which visual content within the current viewport is transmitted and displayed in higher fidelity than content outside the viewport to exceed the visual fidelity of viewport-independent solutions. This is achieved by dividing the video frame into tiles and using HEVC motion-constrained tile set (MCTS) encoding at multiple resolutions.

Panoramic streaming enables users to interactively navigate through high-spatial resolution videos and create an immersive and personalized user experience. Since transmission of high-resolution videos in desirable quality is not feasible given the limited throughput of access and home network links, our work is based on tile-based streaming, where only a spatial subset of the video is transmitted. In this paper, we propose a highly responsive DASH client algorithm that allows users to rapidly change the set of downloaded tiles. The high responsiveness is achieved by using small buffers, which are usually very sensitive to variations of throughput and instantaneous media bitrate and are therefore prone to playback interruptions. The proposed rate adaptation algorithm for DASH, in combination with a peak bitrate reducing RAP configuration referred to as shifted IDRs, outperforms the state of the art rate adaptation algorithms. Experiments report a decreased number of playback interruptions and quality changes while maintaining the average video quality.

In panorama streaming an arbitrary Region-of-Interest (RoI) of a high-resolution video is transmitted, allowing users to navigate interactively within the videos. Transmitting the whole video becomes unfeasible due to the required high bitrates and sending a single video per user, which is encoded for that specific user (i.e. its RoI), has scalability issues. Tile based panoramic streaming overcomes the mentioned drawbacks by allowing users to receive a set of tiles that match their RoI instead of the whole set of tiles. However, optimal tiling -- so that the transmitted bitrate of the RoI content is minimized - is content dependent. In this paper, we propose a model based on a spatio-temporal activity metric so that optimization of the tiling process can be performed in a low complexity manner.

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) is transmitted from a high-spatial resolution video, i.e. a video covering a very “wide-angle” (much larger than the human field-of-view – e.g. 360°). Some transport schemes for panoramic video delivery have been proposed and demonstrated within the past decade, which allow users to navigate interactively within the high-resolution videos. With the recent advances of head mounted displays, consumers may soon have immersive and sufficiently convenient end devices at reach, which could lead to an increasing demand for panoramic video experiences. The solution proposed within this paper is built upon tile-based panoramic streaming, where users receive a set of tiles that match their RoI, and consists in a low-complexity compressed domain video processing technique for using H.265/HEVC and its scalable extensions (H.265/SHVC and H.265/MV-HEVC). The proposed technique generates a single video bitstream out of the selected tiles so that a single hardware decoder can be used. It overcomes the scalability issue of previous solutions not using tiles and the battery consumption issue inherent of tile-based panorama streaming, where multiple parallel software decoders are used. In addition, the described technique is capable of reducing peak streaming bitrate during changes of the RoI, which is crucial for allowing a truly immersive and low latency video experience. Besides, it makes it possible to use Open GOP structures without incurring any playback interruption at switching events, which provides a better compression efficiency compared to closed GOP structures.

Real-time video applications, such as multi party video conferencing, involve the simultaneous transport of multiple and potentially multi-layered video sources to participating or interested parties. It is desirable to mix these multiple source videos into a single video stream at intermediary nodes in the network, e.g. at Multipoint Control Units (MCU). This has the advantage of reduced application and transport complexity on the client device while allowing single hardware decoder devices to consume the content. This paper proposes a solution, which uses the scalable extension of H.265/HEVC (SHVC), and that generates a single bitstream out of several sources by a low-complexity operation. In addition, the presented technique drastically reduces the peak bitrate at layout change events in comparison to state-of-the-art solutions.

Dynamic Adaptive Streaming over HTTP (DASH) has gained a lot of popularity in the last years and has become a widespread solution for media delivery. It is mainly used for VoD services but has being also standardized and can be used for live scenarios. Although there has been a lot of work on low delay DASH services, it is still very challenging to cope with network variations. In fact, low delay configurations prevent DASH users from having a large media buffer to cope with network variations and they can easily lead to deadline misses, which results in a service of low Quality of Experience (QoE). In this paper we propose a solution based on HEVC Tiles and an advanced DASH adaptation algorithm that improves drastically the performance by heavily reducing the deadline misses.

360° Video services with resolutions of UHD and beyond for Virtual Reality head mounted displays are a challenging task due to limits of video decoders in constrained end devices. Adaptivity to the current user viewport is a promising approach but incurs significant encoding overhead when encoding per user or set of viewports. A more efficient way to achieve viewport adaptive streaming is to facilitate motion-constrained HEVC tiles. Original content resolution within the user viewport is preserved while content currently not presented to the user is delivered in lower resolution. A lightweight aggregation of varying resolution tiles into a single HEVC bitstream can be carried out on-the-fly and allows usage of a single decoder instance on the end device.

In this paper we propose a hybrid tracking method which detects moving objects in videos compressed according to H.265/HEVC standard. Our framework largely depends on motion vectors (MV) and block types obtained by partially decoding the video bit stream and occasionally uses pixel domain information to distinguish between two objects. The compressed domain method is based on a Markov Random Field (MRF) model that captures spatial and temporal coherence of the moving object and is updated on a frame-to-frame basis. The hybrid nature of our approach stems from the usage of a pixel domain method that extracts the color information from the fully-decoded I frames and is updated only after completion of each Group-of-Pictures (GOP). We test the tracking accuracy of our method using standard video sequences and show that our hybrid framework provides better tracking accuracy than a state-of-the-art MRF model.

Adaptive streaming is a widely-used solution to avoid playback interruptions and compensate for insufficient bandwidth or momentary congestion. A temporary reduction in bitrate results in less or no video break-up and rebuffering. The rate control mechanism of a codec implementation allows the selection of a target bitrate and tries to meet this target within certain constraints e.g. number of frames, with a certain percentage overshoot. However the target bitrate is not always instantaneously achievable, and the number of bits required to encode a frame are highly content-dependent. In this paper the Codec Step Response metric is proposed to measure the effectiveness of a codec rate control at responding to instantaneous rate variations. This metric allows the measurements of the suitability of a rate control implementation for an application or access network.

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) of high-spatial resolution videos is transmitted. It allows users to navigate interactively around the video and thus select anytime the portion of it they are interested in. The most basic approach consists of each user interacting with the system indicating the desired RoI. Then an encoder associated with each user encodes the desired RoI. However, such a system does not scale well. Instead, we consider tile based panoramic streaming, where users receive a set of tiles that match their RoI, and propose a low-complexity compressed domain video processing technique for tile based panoramic streaming using H.265/HEVC that generates a single video bitstream out of the selected tiles so that single hardware decoders can be used to decode the RoI video stream.

Video compositing such as blending of user interfaces or advertisements on top of video content is used in many applications. Compositing is usually carried out in the pixel domain either after decoding on the end device or based on transcoding before or during transport, e.g. on cloud resources. This paper proposes a novel method to create a composition of several coded input videos in the compressed domain, i.e. without performing entropy coding at runtime. The method entails merging of input video bitstreams into a single output video bitstream and insertion of pre-encoded inter-predicted composition pictures. Such a lightweight approach is computationally much less demanding than transcoding-based compositing and can be beneficial for service scalability. This paper explores the coding performance of the proposed method though experiments with the composition of a transparent ticker overlay on top of video sequences. The proposed method is reported to outperform transcoding-based pixel domain compositing in rate distortion and quality.

Single instruction multiple data (SIMD) instructions have been commonly used to accelerate video codecs. The recently introduced High Efficiency Video Coding (HEVC) codec like its predecessors is based on the hybrid video codec principle and, therefore, is also well suited to be accelerated with SIMD. In this paper we present the SIMD optimization for the entire HEVC decoder for all major SIMD instruction set architectures. Evaluation has been performed on 14 mobile and PC platforms covering most major architectures released in recent years. With SIMD, up to 5× speedup can be achieved over the entire HEVC decoder, resulting in up to 133 and 37.8 frames/s on average on a single core for Main profile 1080p and Main10 profile 2160p sequences, respectively.

Real-time video applications, such as video conferencing and video surveillance systems, typically involve the simultaneous transport of multiple video sources to interested parties that consume the content. It may be desirable to mix these multiple source videos into a single video stream at intermediary nodes in the network. This has the advantage of reduced application and transport complexity on the client device and also makes it possible for devices with a single hardware decoder to consume the content. A typical approach is to apply transcoding operations to the original videos, i.e. the videos are decoded, merged and encoded into a single video stream. This paper proposes an alternative solution to video transcoding, which uses the new video coding standard HEVC and has a much lower processing complexity. We consider how our approach can be realized in real-world applications such as a cloud video mixer. Such systems typically require some degree of dynamics and personalization and we provide some insight into how transport signaling complexities can be addressed.

Dynamic Adaptive Streaming over HTTP (DASH) is becoming the de facto technique for video delivery, especially for VoD services. Although 3GPP has specified carriage of DASH over eMBMS for Live Streaming, eMBMS is not available everywhere (operators are starting service rollout) and it is only worthwhile for reasonably large number of users due to the static SFN resource allocation for those services. Thus, Live Streaming using DASH over unicast connections is still necessary, which may suffer from playback interruptions when the throughput varies since the low end-to-end latency for live streaming requires small buffers. In order to cope with network throughput variations in mobile networks, we propose the usage of scalable video coding, combining it with parallel TCP connections and prioritizing the most important data of the scalable video. We show that using LTE non-GBR bearers for prioritization playback interruptions can be avoided.

In this paper a buffer-constrained rate control (RC) algorithm for High Efficiency Video Coding (HEVC) with hierarchical group of pictures structures is proposed. Specifically, a quantization parameter (QP) cascading approach, which the QP value is increased from one temporal layer to the next, is employed to achieve high coding efficiency while maintaining the buffer fullness at secure levels. When compared to the current state-of-the-art RC algorithm, the experimental results show that our proposal achieves a slightly better rate-distortion performance and a remarkably better buffer control with an acceptable increase in computational complexity.

In this paper we propose an enhancement of the Application-Layer FEC codes introduced by the Pro-MPEG Forum in its Code of Practice 3 r2 (Pro-MPEG COP3 codes) through allowing the introduction of a third dimension. The potential addition of an extra set of protection packets augments the number of possible combinations of data packets within a FEC block for parity packet computation. This enables a finer optimization process of the parameters of the FEC codes for a better adaptation to the specific conditions of the communication channel, increasing their capability. Additionally, we propose a Layer-Aware FEC scheme in which the enhanced Pro-MPEG COP3 codes are used to protect two-layered video streams. Experiment results reveal a gain in the introduction of this protection mechanism, when compared to the standard codes.

In this paper we propose and evaluate a parallelization strategy for the emerging HEVC video coding standard. The proposed strategy is based on entropy slices which allows exploiting parallelism in the entropy decoding stage while maintaining high coding efficiency. Our approach requires to encode videos with one entropy slice per LCU row in order to decode multiple LCU rows in a wavefront parallel manner. Evaluations performed on a PC with 12 Intel Xeon cores running at 3.3 GHz show that it is possible to achieve real-time performance for 1920×1080p50 (53.1 fps) and 2560×1600 (29.5fps) video resolutions with speedups of 5.2× and 6.3× compared to sequential execution, respectively.

In this paper we present a new parallelization approach for HEVC decoding called Overlapped Wavefront (OWF). It is based on wavefront processing and improves its parallelization efficiency by allowing overlapped execution of consecutive pictures. Furthermore, in this strategy of the decoding steps are performed on a CTB basis rather than on a picture basis, which improves data locality. Our implementation achieves between 29.6%, 42.4%, and 66.6% higher frame rates compared to previous results and 11.3%, 21.0%, and 38.0% higher frame rates compared to Tiles, for 2160p, 1600p, and 1080p, respectively.

Video traffic contributes to the majority of data packets transported over cellular wireless. Future broadband wireless access networks based on 3GPP's Long Term Evolution offer mechanisms for optimized transmission with high data rates and low delay. However, especially when packets are transmitted in the LTE downlink and if services are run over-the-top (OTT), optimization of radio resources in a multi-user environment for video services becomes infeasible. The current market trend is moving to OTT solutions, also for video transmission, where an emerging standard based on HTTP streaming - DASH - is expected to have a huge success in the upcoming years. The solution presented in this paper consists of a novel technique, which combines LTE features with knowledge on DASH sessions for optimization of the wireless resources. The combined optimization yields an improved transmission of videos over cellular wireless systems which are based on LTE and LTE-Advanced.

Unlike H.264/advanced video coding, where parallelism was an afterthought, High Efficiency Video Coding currently contains several proposals aimed at making it more parallel-friendly. A performance comparison of the different proposals, however, has not yet been performed. In this paper, we will fill this gap by presenting efficient implementations of the most promising parallelization proposals, namely tiles and wavefront parallel processing (WPP). In addition, we present a novel approach called overlapped wavefront (OWF), which achieves higher performance and efficiency than tiles and WPP. Experiments conducted on a 12-core system running at 3.33 GHz show that our implementations achieve average speedups, for 4k sequences, of 8.7, 9.3, and 10.7 for WPP, tiles, and OWF, respectively.

This paper describes the integration of High Efficiency Video Coding (HEVC) into end-to-end multimedia systems, formats, and protocols such as Real-time transport Protocol, the transport stream of the MPEG-2 standard suite, and dynamic adaptive streaming over the Hypertext Transport Protocol. This paper gives a brief overview of the high-level syntax of HEVC and the relation to the Advanced Video Coding standard (H.264/AVC). A section on HEVC error resilience concludes the HEVC overview. Furthermore, this paper describes applications of video transport and delivery such as broadcast, television over the Internet Protocol, Internet streaming, video conversation, and storage as provided by the different system layers.

HTTP-based delivery for Video on Demand (VoD) has been gaining popularity within recent years. Progressive Download over HTTP, typically used in VoD, takes advantage of the widely deployed network caches to release video servers from sending the same content to a high number of users in the same VoD service. However, due to the inherent heterogeneity of user demands, which may result in requesting the same video content in different resolutions or qualities, the caching efficiency is expected to decrease due to a higher variety in requested media files. The use of Scalable Video Coding allows different representations of the same content to be combined in a single file, whose parts, aka layers, are requested sequentially by a user up to the maximum desired quality. In this paper we show the benefits of using Scalable Video Coding to maintain the same set of possible video content representations, while at the same time maximizing the caching efficiency.

Today's solution to realize wide coverage with the Release 9 MBMS specification is to transmit an H.264/AVC video service with a robustness determined by the worst reception conditions of the target area. This sacrifices service data rate for robustness due to lower modulation and FEC code-rate. The combination of bearer channel multiplex and SVC promises significant gains in terms of channel capacity by establishing unequal error protection on the physical layer. In comparison with single layer transmission, SVC achieves equal performance in terms of continuous playout while allocating fewer resources for the expensive robust channel. This benefit is gained by providing temporary lower quality video to users within bad reception conditions. Simulation results within an MBSFN Release 9 network show significant capacity gains over a given bandwidth with only minor quality degradation for users located in bad reception areas while preserving playout robustness of H.264/AVC single layer services.

HTTP-based delivery for Video on Demand (VoD) has been gaining popularity within recent years. Progressive Download over HTTP, typically used in VoD, takes advantage of the widely deployed network caches to relieve video servers from sending the same content to a high number of users in the same access network. However, due to a sharp increase in the requests at peak hours or due to cross-traffic within the network, congestion may arise in the cache feeder link or access link respectively. Since the connection characteristics may vary over the time, with Dynamic Adaptive Streaming over HTTP (DASH), a technique that has been recently proposed, video clients may dynamically adapt the requested video quality for ongoing video flows, to match their current download rate as good as possible. In this work we show the benefits of using the Scalable Video Coding (SVC) for such a DASH environment.

Three-dimensional video based on stereo and multiview video representations is currently being introduced to the home through various channels, including broadcast such as via cable, terrestrial and satellite transmission, streaming and download through the Internet, as well as on storage media such as Blu-ray discs. In order to deliver 3-D content to the consumer, different media system technologies have been standardized or are currently under development. The most important standards are MPEG-2 systems, which is used for digital broadcast and storage on Blu-ray discs, real-time transport protocol (RTP), which is used for real-time transmissions over the Internet, and the ISO base media file format, which can be used for progressive download in video-on-demand applications. In this paper, we give an overview of these three system layer approaches, where the main focus is on the multiview video coding (MVC) extension of H.264/AVC and the application of the system approaches to the delivery and storage of MVC.

A unified optimization framework for rate allocation among multiple video multicast sessions sharing a wireless network is presented. Our framework applies to delivery of both scalable and non-scalable video streams. In both cases, the optimization objective is to minimize the total video distortion of all peers without incurring excessive network utilization. Our system model explicitly accounts for heterogeneity in wireless link capacities, traffic contention among neighboring links, as well as different video rate-distortion (RD) characteristics. The proposed distributed rate allocation scheme leverages cross-layer information exchange between the media access control and application layers to achieve fast convergence at the optimal, media-aware allocation. Performance of the proposed media-aware rate allocation protocol is compared against a heuristic scheme based on TCP-friendly rate control (TFRC). In network simulations of standard-definition video streaming over single or multiple multicast trees, the proposed scheme consistently achieves higher overall video quality than the TFRC-based heuristics. When delivering scalable streams, the flexibility of per-peer rate adaptation inside each multicast tree yields a further slight improvement in overall video quality over multicast of non-scalable streams.

The bitstream structure of layered media formats such as scalable video coding (SVC) or multiview video coding (MVC) opens up new opportunities for their distribution in Mobile TV services. Features like graceful degradation or the support of the 3-D experience in a backwards-compatible way are enabled. The reason is that parts of the media stream are more important than others with each part itself providing a useful media representation. Typically, the decoding of some parts of the bitstream is only possible, if the corresponding more important parts are correctly received. Hence, unequal error protection (UEP) can be applied protecting important parts of the bitstream more strongly than others. Mobile broadcast systems typically apply forward error correction (FEC) on upper layers to cope with transmission errors, which the physical layer FEC cannot correct. Today's FEC solutions are optimized to transmit single layer video. The exploitation of the dependencies in layered media codecs for UEP using FEC is the subject of this paper. The presented scheme, which is called layer-aware FEC (LA-FEC), incorporates the dependencies of the layered video codec into the FEC code construction. A combinatorial analysis is derived to show the potential theoretical gain in terms of FEC decoding probability and video quality. Furthermore, the implementation of LA-FEC as an extension of the Raptor FEC and the related signaling are described. The performance of layer-aware Raptor code with SVC is shown by experimental results in a DVB-H environment showing significant improvements achieved by LA-FEC.

P2P-streaming has become of high interest in the last years, since it reduces the load on expensive servers, due to the participation of receivers in the media transmission. In this paper, P2P content delivery is shown to be a promising technique for video group communication, for which the main requirement is low-delay. Combining low-delay encoding and low-delay P2P Application Layer Multicast makes it possible to fulfill the delay constraints for interactive group communication applications. For such an application, congestion is a considerable problem, since it causes packet loss or late arrival of the packets, degrading the quality of the service. The results presented in this paper show how rate adaptation in combination with the Scalable Video Coding (SVC) helps to overcome problems in the network, providing a better solution than when non-adaptive single layer coding is transmitted.

This paper investigates how Scalable Video Coding (SVC) can benefit from different Forward Error Correction (FEC) and transmission schemes in mobile broadcast systems. Simulation are performed in DVB-H (Digital Video Broadcasting - Handheld) systems. In DVB-H, a differentiation in robustness for the different SVC layers can be achieved at the link layer using intra-burst MPE-FEC (multi-Protocol Encapsulation FEC). The paper evaluates the gain that can be achieved with the MPE-FEC using equal (EEP) and unequal error protection (UEP), and the performance improvements of an SVC layer-aware FEC (LA-FEC) approach that can be implemented in DVB-H either at the link layer with MPE-iFEC (inter-burst MPE-FEC) or at the application layer with AL-FEC. LA-FEC improves the SVC base layer robustness generating parity information across existing dependencies within the SVC video coding structure. Laboratory measurement results using a TU6 channel model show that the performance of an SVC service in DVB-H can be significantly increased by a proper link layer FEC scheme. It is shown that using SVC in combination with LA-FEC and a proper transmission scheduling does not only give a better performance in terms of PSNR and amount of video outages compared to a single layer service at the same service bit rate, but also gives an additional lower quality layer which can be used for applications like conditional access.

In this paper, we will present and discuss selected results obtained for the ESA study on “Scalable Video Coding Applications and Technologies for mobile satellite based hybrid networks”. The reference use case under investigation concerns the exploitation of heterogeneous reception conditions on a DVB-SH satellite link via scalability and unequal error protection.

P2P content delivery techniques for video transmission have become of high interest in the last years. With the involvement of client into the delivery process, P2P approaches can significantly reduce the load and cost on servers, especially for popular services. However, previous studies have already pointed out the unreliability of P2P-based live streaming approaches due to peer churn, where peers may ungracefully leave the P2P infrastructure, typically an overlay networks. Peers ungracefully leaving the system cause connection losses in the overlay, which require repair operations. During such repair operations, which typically take a few roundtrip times, no data is received from the lost connection. While taking low delay for fast-channel tune-in into account as a key feature for broadcast-like streaming applications, the P2P live streaming approach can only rely on a certain media pre-buffer during such repair operations. In this paper, multi-tree based Application Layer Multicast as a P2P overlay technique for live streaming is considered. The use of Flow Forwarding (FF), a.k.a. Retransmission, or Forward Error Correction (FEC) in combination with Scalable video Coding (SVC) for concealment during overlay repair operations is shown. Furthermore the benefits of using SVC over the use of AVC single layer transmission are presented.

This paper investigates the effects of peer churns in peer-to-peer networks when used to provide video-on-demand services. To evaluate the impact of user behavior in terms of server load and peer satisfaction, three system models have been developed, which are based on: the Gilbert-Elliot chain, the fluidic representation of the user behavior and a queuing analysis of the system. The models are compared evaluating the additional resources that can increase the reliability of P2P network. Simulations show important relationships between playback buffer length, peer request rate, peer average lifetime, and server upload rate.

Modern layered or scalable video coding technologies generate a video bit stream with various inter layer dependencies due to references between the layers. This work proposes a method for extending forward error correction (FECs) codes following dependency structures within the media. The proposed layer-aware FEC (L-FEC) generates repair symbols so that protection of less important dependency layers can be used with protection of more important layers for combined error correction. The L-FEC approach is exemplary applied to rateless LT and Raptor codes. Gains for more important layers can be achieved without increasing the total FEC code rate. The performance gain of the L-FEC is shown by simulation results with receiver-driven layered multicast transmission using scalable video coding (SVC) with a Raptor-based L-FEC.

The deployment of mobile multimedia broadcast services like mobile TV over cellular networks has just started. The coverage in terms of delivered quality per receiver is an important measure to evaluate the experienced quality. In order to provide a reasonable quality for all receivers in a network cell, we analyze graceful degradation techniques in Rel. 6 of 3rd Generation Partnership Project's (3QPP) Multimedia Broadcast and Multicast Services (MBMS). We show the benefits of using H.264/AVC temporal scalability in an existing system for graceful degradation in order to add coverage for users in poor reception conditions, although at a reduced level of quality. For that we define assessment techniques for measuring the quality coverage describing the amount of users reached with a certain media play-out quality. We connect this quality coverage to a measure for used network cell capacity giving the used network resources for 3QPP Rel. 6 network cells. We evaluate graceful degradation under simulated 3QPP Rel. 6 network conditions showing the benefits of graceful degradation for 3QPP MBMS mobile television services.

A wide range of different capabilities and connection qualities typically characterizes receivers of mobile television services. Receiver driven layered multicast (RDLM) offers an efficient way for providing different capabilities over such a broadcast channel. Scalable video coding (SVC) allows for the transmission of multiple video qualities within one media stream. Using SVC generates a video bit stream with various inter layer dependencies due to references between the layers. This work proposes a layer-aware forward error correction (L-FEC) approach in combination with SVC. L-FEC increases robustness of the more important layers by generating protection across layers following existing dependencies of the media stream. The L-FEC is integrated as an extension of a Raptor FEC implementation in a DVB-H broadcast system. It is shown by experimental results that L-FEC outperforms traditional UEP protection schemes.

A new approach to error protection for scalable media is presented for mobile TV applications. Mobile TV is typically characterized by a number of receiver capabilities and connection qualities. A broadcast service should preferably work for multiple receiver capabilities without the need for downscaling or transcoding at the battery-powered mobile devices. Moreover, a media quality that gracefully degrades with reception quality instead of a complete signal loss is also a desirable feature. The scalable video coding (SVC) extension of H.264/AVC offers an efficient way to support the aforementioned features. In mobile broadcast channels forward error correction (FEC) is used to overcome packet losses. This work proposes a layer-aware forward error correction (L-FEC) approach in combination with SVC. L-FEC increases robustness of the more important layers by generating protection across layers. L-FEC is integrated as an extension of an Raptor FEC implementation. It is shown by experimental results that L-FEC outperforms traditional FEC and UEP protection schemes.

Recent advances in forward error correction and scalable video coding enable new approaches for robust, distributed streaming in mobile ad hoc networks (MANETs). This work presents an approach for distribution of real time video by different uncoordinated peer-to-peer relay or source nodes in an overlay network on top of a MANET. The approach proposed here allows for distributed, rate-distortion optimized transmission-rate allocation for competing scalable video streams at relay nodes in the overlay network. Furthermore the approach has the desirable feature of path/source diversity for enhancing reliability in connectivity to serving nodes. Signaling overhead within the overlay network is kept at a minimum, since optimizations are done at relay nodes and clients rather than at servers.

A streaming system based on the scalable video coding (SVC) extension of H.264/AVC is shown. SVC allows for data rate adaptation without re-encoding just by dropping packets of the bit stream. By that, it enables for instance multicast services to clients of heterogeneous capabilities at the same time, while consuming less bit rate compared to simulcasting the services. Additionally, the robustness of a streaming connection against packet losses can be significantly increased if the different layers of the coded video stream are unequally protected by a forward error correction scheme. In this case, users will experience graceful degradation of image quality rather than visible errors or interruptions. The introduction of new services using SVC can also take advantage of a backward-compatible base layer. Different use cases for SVC streaming in wireless IP networks and selected simulation results of the improved error robustness for a DVB-H wireless transmission are presented.

The scalable video coding (SVC) standard as an extension of H.264/AVC allows efficient, standard-based temporal, spatial, and quality scalability of video bit streams. Scalability of a video bit stream allows for media bit rate as well as for device capability adaptation. Moreover, adaptation of the bit rate of a video signal is a desirable key feature, if limitation in network resources, mostly characterized by throughput variations, varying delay or transmission errors, need to be considered. Typically, in mobile networks the throughput, delay and errors of a connection (link) depend on the current reception conditions, which are largely influenced by a number of physical factors. In order to cope with the typically varying characteristics of mobile communication channels in unicast, multicast, or broadcast services, different methods for increasing robustness and achieving quality of service are desirable. We will give an overview of SVC and its relation to mobile delivery methods. Furthermore, innovative use cases are introduced which apply SVC in mobile networks.

The transport of scalable media, and in particular of scalable video conforming to the forthcoming Scalable Video Coding (SVC) technology, presents challenges not only in the video compression technology, but also in transport and signaling. This paper discusses the current status of standardization of the support for scalable media, and SVC in particular, over IP based networks. Both the transport of SVC over the Real-time Transport Protocol (RTP), and the signaling support-namely the additional mechanisms in the Session Description Protocol (SDP)-are covered. As it turns out, the support of SVC over RTP is not quite as straightforward as that of nonscalable video bit streams. Specifically, the signaling architecture requires an almost complete overhaul, and new protocol mechanisms need to be introduced into the packetization.

Abstract The integration of reliable Video‐on‐Demand (VoD) broadcasting schemes in mobile datacast systems, specifically in DVB‐H, is studied and enhanced. Sophisticated VoD broadcasting schemes such as Harmonic Broadcasting (HB) allows receivers to tune into the ongoing transmission of a video stream at arbitrary time, while still being able to receive the multimedia sequence from beginning to end, after short initial playout latency. In addition, we address service enhancements by using scalable video coding (SVC) to support heterogeneous receiver capabilities and receiving conditions as well as the reception of the signal from more than on transmission site. We present and discuss options for the integration of VoD broadcasting schemes in combination with fountain codes. Optimizations in parameter selection are discussed. A realistic protocol environment is only slightly modified to support our system concept. Simulation results show the benefits of the discussed VoD scheme compared to existing approaches if integrated in DVB‐H. Copyright © 2007 John Wiley & Sons, Ltd.

Video transmission over networks is getting increasingly important, thus upcoming network types like mobile ad-hoc networks (MANETs) can also become a suitable platform for exchanging/sharing real-time video streams. We present an approach using different sources with linear independent representations of a layered video stream for increasing the robustness in transmission. This approach is based on the scalable video coding (SVC) extensions of H.264/MPEG-4 AVC with different layers for assigning importance for transmission. Additionally a novel unequal packet loss protection (UPLP) scheme based on Raptor forward error correction codes is employed. This scheme allows for reception from different sources and benefits from it. While the reception of a single stream guarantees base quality at least, the combined reception enables play-back of video of full quality and/or lower error rates.

We propose a dynamic sharing of radio resources in a wireless multiuser system by combining a flexible video coding scheme, namely the currently developed scalable video coding (SVC) extension of H.264/AVC with appropriate radio link buffer management for multiuser streaming services. The paper introduces the components of the proposed system and the used cross-layer interface. Simulation results verify the benefits of our approach

In 3GPP a standard for multimedia streaming over mobile networks called PSS (packet-switched streaming service) has been specified. In order to deal with changing transmission characteristics like varying link bit-rate or handover, adaptive streaming features like client feedback have been included into Release 6 of 3GPP PSS. The adaptive behavior of the streaming server is not specified in PSS. In this paper, an adaptive video streaming system is presented that is compliant with PSS. Using PSS client feedback, transmission characteristics and client buffer status are determined. By means of transmission rate control, packet scheduling, video thinning, and bit-stream switching, the streaming server reacts to the determined transmission characteristics and the client buffer status. As video coding standard, H.264/AVC is used in the adaptive streaming system. Experimental results show that using the presented adaptive streaming system the overall video quality at the client is significantly improved in case of varying link bit-rate or handover.

We present a streaming system that utilizes interleaved transmission for real-time H.264/AVC video in 3G wireless environments with benefits shown especially in the presence of link outages. In the 3GPP packet-switched streaming Service Rel. 6, H.264/AVC and its RTP payload format are specified. The RTP payload format allows interleaved transmission of NAL units of H.264/AVC. Our simulations also include audio into the interleaving framework and are conducted within a testbed that emulates a 3G network including block error rates on the physical layer, a buffer for retransmission on the link layer for different error rates, and link outages. The experimental results demonstrate the superior performance of interleaving for typical link outage settings.

For the ongoing standardization activities for IP-based broadcast video and audio streaming services over DVB H, DMB or 3GPP’s MBMS the use of new forward error protection techniques is investigated, since these activities only propose the use of equal error protection schemes. Especially in distributing video hierarchical protection schemes can be used for more efficient content aware protection. Even older video coding standards allow simple quality reducing mechanism like temporal scalability. New scalable video coding schemes like the scalable extensions of H. 264/AVC can be used for a further enhancement of the protection schemes. In combination with an unequal erasure protection scheme the efficiency of network delivery can be enhanced significantly. We propose a scheme, which shows benefits compared to single layer stream protection schemes. The scheme allows protection over a larger range of error rates compared to single layer protection while showing “ graceful degradation” like behavior.

The scalability extension of H.264/AVC (H. Schwarz, et al., 2004) uses a layered approach for providing spatial, temporal, and SNR scalability. Due to this concept, only a restricted set of spatio-temporal-SNR points can be extracted and decoded from a global scalable bit-stream, and this set of points is defined by the chosen encoder configuration. In this paper, we present a new approach for providing flexible combined spatial, temporal, and SNR scalability. The increased flexibility is achieved by introducing NAL units that represent a refinement signal for a picture in a coarse-to-fine-description and can be truncated at any arbitrary point. The simulation results show that this approach is capable of providing flexible combined scalability while the coding efficiency is only slightly worse than that of the layered approach.