Adaptive streaming technologies are used heavily to deliver video content over the Internet, but still face problems in networks with limited bandwidth, for videos with ultra-high definition (UHD) or greater, even when using the latest High-Efficiency Video Coding (HEVC) standard. In this paper, we propose a solution to these problems based on three things: a specific HEVC-compliant motion-constrained tiled encoding allowing tile-based decoding and rendering; a specific derivation of the ISO base media file format enabling a tile-based encapsulation and delivery; and their use in adaptive streaming of tiled video content using MPEG dynamic adaptive streaming over HTTP requiring only a single standard HEVC decoder on the client side. We present an evaluation of constrained and unconstrained HEVC tiling compression, and also of the associated packaging and streaming overheads. The results of this evaluation indicate that our approach incurs a complete overhead of at most 7.3% for UHD content, at 10 Mbps or above, when split in 25 tiles, and only 3.5% for nine tiles, when compared with the streaming of the non-tiled video. Our approach also comes with a small impact on client-side players but provides the interesting benefit of being able to deliver UHD videos over bandwidth-constrained networks adaptively, based on its tiled content.

HTTP Adaptive Streaming (HAS) techniques are now the dominant solution for video delivery in mobile networks. Over the past few years, several HAS algorithms have been introduced in order to improve user quality-of-experience (QoE) by bit-rate adaptation. Their difference is mainly the required input information, ranging from network characteristics to application-layer parameters such as the playback buffer. Interestingly, despite the recent outburst in scientific papers on the topic, a comprehensive comparative study of the main algorithm classes is still missing. In this paper we provide such comparison by evaluating the performance of the state-of-the-art HAS algorithms per class, based on data from field measurements. We provide a systematic study of the main QoE factors and the impact of the target buffer level We conclude that this target buffer level is a critical classifier for the studied HAS algorithms. While buffer-based algorithms show superior QoE in most of the cases, their performance may differ at the low target buffer levels of live streaming services. Overall, we believe that our findings provide valuable insight for the design and choice of HAS algorithms according to networks conditions and service requirements.

Interactive spatial navigation within a video is getting attention in the research community, especially in scenarios such as 360° video, video surveillance or region-of-interest viewing. With more content being delivered over the top, the complexity of spatial access to the video is increased by the need to perform bitrate adaptation to react to network variations. In this paper, we describe how spatial access can be performed in an adaptive HTTP streaming context, using tiling of the source content, MPEG-DASH and its SRD extensions. We describe a configurable implementation of these technologies, within the GPAC open-source player, allowing experimentations of different adaptation policies for tiled video content.

This paper presents the Spatial Representation Description (SRD) feature of the second amendment of MPEG DASH standard part 1, 23009-1:2014 [1]. SRD is an approach for streaming only spatial sub-parts of a video to display devices, in combination with the form of adaptive multi-rate streaming that is intrinsically supported by MPEG DASH. The SRD feature extends the Media Presentation Description (MPD) of MPEG DASH by describing spatial relationships between associated pieces of video content. This enables the DASH client to select and retrieve only those video streams at those resolutions that are relevant to the user experience. The paper describes the design principles behind SRD, the different possibilities it enables and examples of how SRD was used in different experiments on interactive streaming of ultra-high resolution video.

In this paper we describe a multiscreen-oriented approach for segmenting web pages. The segmentation is an automatic and hybrid visual and structural method. It aims at creating coherent blocks which have different functions determined by the multiscreen environment. It is also characterized by a dynamic adaptation to the page content.Experiments are conducted on a set of existing applications that contain multimedia elements, in particular YouTube and video player pages. Results are compared with one segmentation method from the literature and with a ground truth manually created. With a 81% precision, the MSoS is a promising method that is capable of producing good segmentation results.

While over-the-top video distribution is now widely deployed, it still suffers from much higher latencies than traditional broadcast, typically from a few seconds up to half a minute. In this paper, we demonstrate a novel DASH system with latency close to broadcast channels, and show how such a system can be used to enable combined broadcast and broadband services while keeping the client buffering requirements on the broadcast link low.

Adaptive HTTP Streaming has become a widespread technology for delivering video to the end users. However, this technology usually suffers from various latencies, requiring an initial delay before playback. This is problematic for low delay or interactive applications, such as gaming or live event webcasting. A long initial waiting time is perceived by most users as bad quality, or even non-working content. In this paper, we review the causes of initial delays in MPEG-DASH and common strategies used to reduce this latency. We propose a new method based on HTTP/1.1 and compatible with existing infrastructures for the initial setup of an MPEG-DASH session. We compare our proposal to several existing approaches based on HTTP/1.x and on the HTTP/2 server push technique. Our experimental results show that 2 RTTs can be avoided and that the total downloaded size is reduced by 25% when using HTTP/1.1 and by 20% in HTTP/2 with similar latency.

Creating web applications for the multiscreen environment is still a challenge. One approach is to transform existing single-screen applications but this has not been done yet automatically or generically. This paper proposes a refactoring system. It consists of a generic and extensible mapping phase that automatically analyzes the application content based on a semantic or a visual criterion determined by the author or the user, and prepares it for the splitting process. The system then splits the application and as a result delivers two instrumented applications ready for distribution across devices. During runtime, the system uses a mirroring phase to maintain the functionality of the distributed application and to support a dynamic splitting process. Developed as a Chrome extension, our approach is validated on several web applications, including a YouTube page and a video application from Mozilla.

Video streaming has become a very popular application on the web, with rich player interfaces and subtitle integration. Additionally, live streaming solutions are deployed based on HTTP Streaming technologies. However, the integration of video and subtitles in live streaming solutions still poses some problems. This paper describes a demonstration of live streaming of video and subtitle data using the MPEG-DASH technology and its synchronous playback in a web browser. It presents the formats, architecture and technical choices made for this demonstration and shows that it is feasible with upcoming browsers, paving the way for richer web video applications.

The delivery of multimedia content over HTTP and on top of existing Internet infrastructures is becoming the preferred method within heterogeneous environment. The basic design principle is having an intelligent client which selects given and applicable media representations by issuing HTTP requests for individual segments based on the users' context and current conditions. Typically, this client behavior differs between implementations of the same kind and for the objective evaluations thereof appropriate datasets are needed. This paper presents a distributed dataset for the recently published MPEG-DASH standard which is mirrored at different sites across Europe, namely Klagenfurt, Paris, and Prague. A client implementation may choose to request segments from these sites and dynamically switch to a different location, e.g., in case the one currently used causes any issues. Thus, this distributed DASH dataset can be used for real-world evaluations enabling the simulation of switching between different content delivery networks.

There is currently significant interest in freeform, curve-based authoring of graphic images. In particular, "diffusion curves" facilitate graphic image creation by allowing an image designer to specify naturalistic images by drawing curves and setting colour values along either side of those curves. Recently, extensions to diffusion curves based on the biharmonic equation have been proposed which provide smooth interpolation through specified colour values and allow image designers to specify colour gradient constraints at curves. We present a Boundary Element Method (BEM) for rendering diffusion curve images with smooth interpolation and gradient constraints, which generates a solved boundary element image representation. The diffusion curve image can be evaluated from the solved representation using a novel and efficient line-by-line approach. We also describe "curve-aware" upsampling, in which a full resolution diffusion curve image can be upsampled from a lower resolution image using formula evaluated orrections near curves. The BEM solved image representation is compact. It therefore offers advantages in scenarios where solved image representations are transmitted to devices for rendering and where PDE solving at the device is undesirable due to time or processing constraints.

Existing broadcast networks can deliver identical content to a large number of users. Broadband networks can deliver personalized content to specific users. Hybrid delivery tries to use the best of both types of networks to provide customized services to many users. This paper studies the theoretical aspects and challenges behind the delivery of multimedia content over hybrid broadcast broadband networks, including bootstrapping, synchronization and resynchronization. It presents a solution based on the use of a global clock which does not require communication across networks and is compatible with existing technologies. This solution is implemented in a multimedia player and evaluated against two real-world scenarios, mixing DVB or FM broadcast networks with a broadband IP network. The results confirm the theoretical approach and show that some fine tuning in the networks is needed for a tighter synchronization.

Multimedia applications are challenging software and require collaboration of very different components such as networking, rendering, or scripting to provide a nice user experience. GPAC is an open source multimedia framework that implements a vast number of components and helps experimenting with different types of multimedia applications. It provides a multimedia packager, some servers and a player for dynamic and interactive multimedia content. In this paper, we present the recent additions to the project which allow experimenting with new applications such as dynamic home networking or rich broadcasting of interactive content on the latest devices, including with 3D displays. Such experiments can be useful in research work and, as illustrated, in academic environment for educational purposes.

The Huawei/3DLife Grand Challenge Dataset provides multimodal recordings of Salsa dancing, consisting of audiovisual streams along with depth maps and inertial measurements. In this paper, we propose a system for augmented reality-based evaluations of Salsa dancer performances. An essential step for such a system is the automatic temporal synchronisation of the multiple modalities captured from different sensors, for which we propose efficient solutions. Furthermore, we contribute modules for the automatic analysis of dance performances and present an original software application, specifically designed for the evaluation scenario considered, which enables an enhanced dance visualisation experience, through the augmentation of the original media with the results of our automatic analyses.

In recent years, audio-visual distribution over Internet has witnessed the growing usage of HTTP based delivery systems. While these systems have their drawbacks for some use-cases, they also have many advantages, the most important one being reusing the existing delivery infrastructure such as HTTP servers, proxies and caches. The MPEG group has started the standardization of the Dynamic Adaptive Streaming over HTTP (DASH) of major transport formats, MPEG-2 TS and ISO Base Media File, and mostly focuses on audio, video and subtitle formats. We believe Rich Media services have a role to play in this landscape, as a presentation layer for the audio-visual content first, but also as a dedicated media in the DASH content for real-time media-synchronized interactive services. In this paper, we present a study on usages of DASH for Rich Media Services.

Widgets are small and focused multimedia applications that can be found on desktop computers, mobile devices or even TV sets. Widgets rely on structured documents to describe their spatial, temporal and interactive behavior but also to communicate with remote data sources. However, these sources have to be known at authoring time and the communication process relies heavily on scripting. In this paper, we describe a mechanism enabling the communication between widgets and their dynamic environment (other widgets, remote data sources). The proposed declarative mechanism is compatible with existing Widgets technologies, usable with script-based widgets as well as with fully declarative widgets. A description of an implementation is also provided.

International audience

Several scalable media codecs have been standardized in recent years to cope with heterogeneous usage conditions and to aim at always providing audio, video and image content in the best possible quality. Today, interactive multimedia presentations are becoming accessible on handheld terminals and face the same adaptation challenges as the media elements they present: quite diversified screen, memory and processing power capabilities. In this paper, we address the adaptation of multimedia documents by applying the concept of scalability to their presentation.

In this paper, we demonstrate the adaptation of multimedia digital radio services in broadcast environments based on scalable multimedia documents. The authoring of our multimedia services relies on the Scalable MSTI model that decomposes multimedia documents into three ordered dimensions: Spatial, Temporal and Interactive descriptions. Our demonstration shows Scalable MSTI multimedia documents that can be adapted to typical T-DMB digital radio usage scenarios.

The heterogeneity of handheld terminals in terms of screen resolution, processing capabilities or available decoding memory is a challenge for multimedia services that have been tackled by many scene adaptation techniques so far. In broadcast environment, the adaptation intelligence must be transmitted along with the content and may induce critical costs that must be minimized. In this paper, we propose a broadcast-friendly adaptation technique of the spatial layout of multimedia content based on the use of incremental scene updates. The advantages of our approach have been evaluated on a T-DMB digital radio service and compared to other adaptation techniques applicable to broadcasted multimedia services. Experimental results show that fine grained spatial adaptation on constrained handheld terminals can successfully be achieved through adaptation scene updates with a limited bandwidth overhead.

The Scalable Vector Graphics (SVG) language allows in its version 1.2 the description of multimedia scenes including audio, video, vector graphics, interactivity and animations. This standard has been selected by the mobile industry as the format for vector graphics and rich media content. For this purpose, additional tools were introduced in the language to solve the problem of the playback of long-running SVG sequences on memory-constrained devices like mobile phones. However, the proposed tools are not entirely sufficient and solutions outside the scope of SVG are needed.

GPAC is a multimedia framework for research and academic purposes in different aspects of multimedia, with a focus on presentation technologies (graphics, animation and interactivity). The project started in 2003 with the initial goal to develop from scratch, in ANSI C, clean software compliant to the MPEG-4 Systems standard, a small and flexible alternative to the MPEG-4 reference software. Since then, the project has evolved into an advanced multimedia player, a multimedia packager and several servers. The project is intended to a wide audience ranging from end-users or content creators with development skills who want to experiment the new standards for interactive technologies or want to convert files for mobile devices, to developers who need players and/or server for multimedia streaming applications.

Our work focuses on the encoding of two-dimensional (2-D) graphics animated cartoons with MPEG-4. This stems from a need to stop encoding cartoons as videos and to start using methods more adapted to animated 2-D vector graphics sequences. We expected this could be an efficient coding with good versatility for various types of terminals. First, we present some technical specification for such cartoons. Next, we give an overview of MPEG-4 BIFS encoding methods. Then, we analyze how to optimize the use of these methods; some of our proposals could be useful for 2-D graphics applications other than animated cartoon. Last, we show some results with significantly better encoding efficiency over simple BIFS encoding.