Workplace stress significantly affects employee well-being and task efficiency yet current stress recognition approaches remain impractical for real-world deployment. Existing methods mostly depend on intrusive physiological signals which require sensors attached to the body. Inspired by how humans naturally infer stress through observable behavior this PhD proposes a non-intrusive multimodal deep learning approach. This method uses overt expressive cues combining facial expressions to identify emotional valence with voice tone to capture emotional intensity and eye movements to monitor mental fatigue. To ensure reliability the system incorporates facial thermal imaging as a covert but non-intrusive cue to detect involuntary heat changes that reveal genuine stress even when a user attempts to mask their outward expression.

Haptic teleoperation systems mark a critical breakthrough in remote manipulation technologies, delivering immersive user experiences through precise control and tactile feedback. This paper aims to explore the interplay between Quality of Service (QoS) network metrics, such as latency, jitter, and packet loss, and Quality of Experience (QoE) features, including immersion, control, and engagement. This investigation is conducted through a combination of objective measurements and subjective evaluations over a private Fifth Generation Standalone (5G SA) network. In addition, it aims to realize the extent to which transport-layer protocols operating over IP, such as TCP and UDP, as well as environmental factors like indoor and outdoor settings, influence system performance.Results confirm the well-established trade-off between reliability and latency in transport protocols, with TCP offering higher reliability in controlled indoor environments, and UDP exhibiting better responsiveness in dynamic outdoor scenarios. While these findings align with existing knowledge, their empirical validation in the context of immersive haptic applications over a private 5G SA network reinforces their relevance. In addition, QoE metrics were found to be linked to QoS indicators, highlighting the importance of balancing speed, stability, and reliability. These findings provide valuable insights towards designing adaptive teleoperation systems capable of dynamically optimizing performance under diverse conditions.

In this paper, we investigate the effects of asynchrony between the visual and haptic feedback in virtual reality (VR) on user experience, specifically focusing on understanding users' awareness of this asynchrony and its effect on their level of satisfaction. Using Unreal Engine, we created an experimental setup to adjust the timing between these sensory inputs. Our experiment featured a VR dodge game that provides haptic feedback on the body when the player is hit by a multitude of virtual objects. Conducting a targeted, small-scale user study, we aim to understand in what ways an introduced asynchrony influences the VR experience. The results highlight the perceptibility of asynchrony, which significantly affects the overall user experience. Nonetheless, we also find an asymmetry that benefits scenarios where haptic feedback precedes visual cues. Furthermore, our findings suggest that users can generally accept minor levels of asynchrony without significant disadvantages to their satisfaction. However, it is interesting to note that even when users cannot explicitly identify any asynchrony, they might still experience a slight decrease in satisfaction.

With the growth of Extended Reality (XR) and capturing devices, point cloud representation has become attractive to academics and industry. Point Cloud Compression (PCC) algorithms further promote numerous XR applications that may change our daily life. However, in the literature, PCC algorithms are often evaluated with heterogeneous datasets, metrics, and parameters, making the results hard to interpret. In this article, we propose an open-source benchmark platform called PCC Arena. Our platform is modularized in three aspects: PCC algorithms, point cloud datasets, and performance metrics. Users can easily extend PCC Arena in each aspect to fulfill the requirements of their experiments. To show the effectiveness of PCC Arena, we integrate seven PCC algorithms into PCC Arena along with six point cloud datasets. We then compare the algorithms on ten carefully selected metrics to evaluate the quality of the output point clouds. We further conduct a user study to quantify the user-perceived quality of rendered images that are produced by different PCC algorithms. Several novel insights are revealed in our comparison: (i) Signal Processing (SP)-based PCC algorithms are stable for different usage scenarios, but the trade-offs between coding efficiency and quality should be carefully addressed, (ii) Neural Network (NN)-based PCC algorithms have the potential to consume lower bitrates yet provide similar results to SP-based algorithms, (iii) NN-based PCC algorithms may generate artifacts and suffer from long running time, and (iv) NN-based PCC algorithms are worth more in-depth studies as the recently proposed NN-based PCC algorithms improve the quality and running time. We believe that PCC Arena can play an essential role in allowing engineers and researchers to better interpret and compare the performance of future PCC algorithms.

Subjective user studies are important to ensure the fidelity and usability of systems that generate multimedia content. Testing how end-users and domain experts perceive multimedia assets might provide crucial information. In this paper, we present our experiences with the open source hybrid crowdsourced-remote user survey framework called Huldra, which is intended for conducting web-based subjective user studies and aims to integrate the individual benefits associated with traditional, crowdsourced, and remote methods. We disseminate our experiences and insights from two actively deployed use cases and discuss challenges and opportunities associated with using Huldra as a framework for conducting user studies.

Hardware acceleration of network functions is essential to meet the challenging Quality of Service requirements in nowadays computer networks. Graphical Processing Units (GPU) are a widely deployed technology that can also be used for computing tasks, including acceleration of network functions. In this work, we demonstrate how commodity GPUs, which do not provide any network interfaces, can be used to accelerate network functions. Our approach leverages PCIe peer-to-peer capabilities and allows the GPU to control the network interface card directly, without any assistance from the operating system or control application. The presented evaluation results demonstrate the feasibility of our approach and its performance of up to 10 Gbit/s, even for small packets.

Crowdsourced and remote user studies have recently gained popularity as alternatives to traditional laboratory studies. However, they are subject to unreliability, and it is challenging to ensure that valid results are collected, especially when conducting user studies with experts. Experts are a sparse resource, usually having busy schedules and heavy workloads, and are not necessarily geographically close. They are therefore often unwilling to participate in studies which require physical attendance. In this paper, we compare three alternative methods: crowd sourced user study with non-experts, remote user study with non-experts, and remote user study with domain experts, for a use case involving investigative child interview training. We present the results from three subjective studies about the perception of AI-generated child avatars, which is developed using various technologies such as dialogue models, game engine, text-to-speech and speech-to-text components. The study was conducted with three different user groups, and our results indicate the importance of using best practice measures for ensuring the collection of reliable results in crowdsourced settings as compared to remote studies, and highlight the difference between the perspectives of domain experts and non-experts.

This paper introduces the Meshroom software and its underlying 3D computer vision framework AliceVision. This solution provides a photogrammetry pipeline to reconstruct 3D scenes from a set of unordered images. It also features other pipelines for fusing multi-bracketing low dynamic range images into high dynamic range, stitching multiple images into a panorama and estimating the motion of a moving camera. Meshroom's node-graph architecture allows the user to customize the different pipelines to adjust them to their domain specific needs. The user can interactively add other processing nodes to modify a pipeline, export intermediate data to analyze the result of the algorithms and easily compare the outputs given by different sets of parameters. The software package is released in open source and relies on open file formats. These features enable researchers to conveniently run the pipelines, access and visualize the data at each step, thus promoting the sharing and the reproducibility of the results.

LiDARs can capture distances with high accuracy and should be very useful to create point clouds that provide highly detailed representations of an environment. If these reconstructions are meant as baseline or ground truth for other algorithms, they must have a high density and accuracy.

Cloud Gaming (CG) is an immersive multimedia service that promises many benefits. In CG, the games are rendered in a cloud server, and the resulted scenes are streamed as a video sequence to the client. Using CG users are not forced to update their gaming hardware frequently, and available games can be played on any operating system or suitable device. However, cloud gaming requires a reliable and low-latency network, which makes it a very challenging service. Transmission latency strongly affects the playability of a cloud game and consequently reduces the users' Quality of Experience (QoE). In this paper, we propose a latency compensation technique using game adaptation that mitigates the influence of delay on QoE. This technique uses five game characteristics for the adaptation. These characteristics, in addition to an Aim-assistance technique, were implemented in four games for evaluation. A subjective study using 194 participants was conducted using a crowdsourcing approach. The results showed that the majority of the proposed adaptation techniques lead to significant improvements in the cloud gaming QoE.

The gaming industry is one of the largest digital markets for decades and is steady developing as evident by new emerging gaming services such as gaming video streaming, online gaming, and cloud gaming. While the market is rapidly growing, the quality of these services depends strongly on network characteristics as well as resource management. With the advancement of encoding technologies such as hardware accelerated engines, fast encoding is possible for delay sensitive applications such as cloud gaming. Therefore, already existing video quality models do not offer a good performance for cloud gaming applications. Thus, in this paper, we provide a gaming video quality dataset that considers hardware accelerated engines for video compression using the H.264 standard. In addition, we investigate the performance of signal-based and parametric video quality models on the new gaming video dataset. Finally, we build two novel parametric-based models, a planning and a monitoring model, for gaming quality estimation. Both models are based on perceptual video quality dimensions and can be used to optimize the resource allocation of gaming video streaming services.

Cloud Gaming is an emerging service that is considered by many as the future of the gaming industry. This service requires a highly reliable network with low latency and high bandwidth. If these requirements are not satisfied, cloud gaming services cannot create a good Quality of Experience (QoE) for its users. However, gaming QoE can vary significantly among different game scenarios and users. For an optimal resource allocation and quality estimation, it is highly important for cloud providers, game developers, and network planners to consider the influence of the game content and gamers. This paper presents the result of a subjective study that investigated the impact of different player strategies and user inputs on their perceived delay. The results indicated that the user input characteristics vary among the games but stays the same between different users and different strategies. In addition to the users' inputs, the input quality and the overall gaming experience of the users were also investigated, and results did not show any main effect of user strategy on the delay sensitivity of the games.

The large variety of compute-heavy and data-driven applications accelerate the need for a distributed I/O solution that enables cost-effective scaling of resources between networked hosts. For example, in a cluster system, different machines may have various devices available at different times, but moving workloads to remote units over the network is often costly and introduces large overheads compared to accessing local resources. To facilitate I/O disaggregation and device sharing among hosts connected using Peripheral Component Interconnect Express (PCIe) non-transparent bridges, we present SmartIO. NVMes, GPUs, network adapters, or any other standard PCIe device may be borrowed and accessed directly, as if they were local to the remote machines. We provide capabilities beyond existing disaggregation solutions by combining traditional I/O with distributed shared-memory functionality, allowing devices to become part of the same global address space as cluster applications. Software is entirely removed from the data path, and simultaneous sharing of a device among application processes running on remote hosts is enabled. Our experimental results show that I/O devices can be shared with remote hosts, achieving native PCIe performance. Thus, compared to existing device distribution mechanisms, SmartIO provides more efficient, low-cost resource sharing, increasing the overall system performance.

The Round-Trip Time (RTT) is a property of the path between a sender and a receiver communicating with Transmission Control Protocol (TCP) over an IP network and over the public Internet. The end-to-end RTT value influences significantly the dynamics and performance of TCP, which is by far the most used communication protocol. Thus, in communication networks, RTT is an important network performance variable. By measuring the traffic at an intermediate node, a network operator or service provider can estimate the RTT and use the estimation to study and troubleshoot the per-connection characteristics and performance. This paper aims at improving the accuracy and timeliness of the RTT estimation, to help network operators improving their analysis. We propose and evaluate a novel deep learning-based model capable of dynamically predicting at real-time the RTT between the sender and receiver with high accuracy based on passive measurements collected at an intermediate node, taking advantage of the commonly used TCP timestamps. We validate extensively our prediction methodology in a controlled experimental testbed and in a realistic scenario on the Google Cloud platform. We show that our model, which is based on classical deep learning algorithms, gives reasonably effective state-of-the-art performance results across multiple TCP congestion control variants. We also show that the model works well for transfer learning. Even though the RTT prediction model was trained on an emulated network, it performs well also when applied to a realistic scenario setting, as demonstrated in our experimental evaluation.

Video streaming is one of the top traffic contributors in the Internet and a frequent research subject. It is expected that streaming traffic will grow 4-fold for video globally and 9-fold for mobile video between 2017 and 2022. In this paper, we present an automatized measurement framework for evaluating video streaming QoE in operational broadband networks, using headless streaming with a Docker-based client, and a server-side implementation allowing for the use of multiple video players and adaptation algorithms. Our framework allows for integration with the acsMONROE testbed and Bitmovin Analytics, which bring on the possibility to conduct large-scale measurements in different networks, including mobility scenarios, and monitor different parameters in the application, transport, network, and physical layers in real-time.

Recent approaches to tiled 360° adaptive bitrate video streaming present significant bandwidth savings at little risk of stalling when only parts of the video, e.g., the current and predicted viewport, are transferred in high quality while the rest of the 360° video tiles are transferred in a lower quality. While this is currently feasible for video on demand scenarios, it poses a difficult problem for 360° live streaming as naive methods produce a considerable overhead owing to the lack of tiling support in existing hardware encoders.

Both online and cloud gaming services require a very low network delay to create a good Quality of Experience (QoE) for their users. The required network latency cannot be guaranteed due to the current best effort-nature of the network, and as a result, network latency often degrades the gamers performance and QoE. In this paper, the adaptability of gamers to different variations on delay is investigated both subjectively and objectively using three self-developed games. The results show that gamers can adapt to constant delay while they are playing and change their behavior if the actions in a game are predictable. Such adaptation leads to a significant increase in gamers performance and QoE. The paper also provides evidence that regardless of performance frequent delay switching annoys gamers. The result of this study can be used to create a network resource allocation technique which controls a congested network by giving more priority and resource to the unadaptable games than the adaptable games.

Video analysis including classification, segmentation or tagging is one of the most challenging but also interesting topics multimedia research currently try to tackle. This is often related to videos from surveillance cameras or social media. In the last years, also medical institutions produce more and more video and image content. Some areas of medical image analysis, like radiology or brain scans, are well covered, but there is a much broader potential of medical multimedia content analysis. For example, in colonoscopy, 20% of polyps are missed or incompletely removed on average. Thus, automatic detection to support medical experts can be useful. In this paper, we present and evaluate several machine learning-based approaches for real-time polyp detection for live colonoscopy. We propose pixel-wise localization and frame-wise detection methods which include both handcrafted and deep learning based approaches. The experimental results demonstrate the capability of analyzing multimedia content in real clinical settings, the possible improvements in the work flow and the potential improved detection rates for medical experts.

With 360° panorama video technology becoming commonplace, the need for efficient streaming methods for such videos arises. We go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-off between bandwidth usage and the video quality in the user’s field-of-view. We have created an architecture that combines RTP and DASH to deliver 360° VR content to a Huawei set-top-box and a Samsung Galaxy S7. Our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-offs through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the field-of-view changes.

With emerging delay sensitive gaming services such as cloud gaming and online gaming, the importance of understanding and reducing the effect of delay on the gamer's Quality of Experience (QoE) becomes highly important for the success of these services. In this paper, the findings of two subjective experiments investigating the relationship between delay and QoE are reported. In the first study, it was shown that in addition to the direct effect of the delay on QoE, there is a significant indirect effect between delay and QoE through the relationship with performance. In the second part of the paper, we illustrate that adapting characteristics of a game can strongly mitigate the negative effect of delay on gaming QoE due to increased player performance. This adaptation in addition to compensation the effect of the delay, in contrast to the other difficulty adjustment systems, does not require to track the gamer's interaction, behaviors, and profile.

The keypoint detector and descriptor Scalable Invariant Feature Transform (SIFT) [8] is famous for its ability to extract and describe keypoints in 2D images of natural scenes. It is used in ranging from object recognition to 3D reconstruction. However, SIFT is considered compute-heavy. This has led to the development of many keypoint extraction and description methods that sacrifice the wide applicability of SIFT for higher speed. We present our CUDA implementation named PopSift that does not sacrifice any detail of the SIFT algorithm, achieves a keypoint extraction and description performance that is as accurate as the best existing implementations, and runs at least 100x faster on a high-end consumer GPU than existing CPU implementations on a desktop CPU. Without any algorithmic trade-offs and short-cuts that sacrifice quality for speed, we extract at >25 fps from 1080p images with upscaling to 3840x2160 pixels on a high-end consumer GPU.

Video streaming is a very popular service among the end-users of Mobile Broadband (MBB) networks. DASH and WebRTC are two key technologies in the delivery of mobile video. In this work, we empirically assess the performance of video streaming with DASH and WebRTC in operational MBB networks, by using a large number of programmable network probes spread over several countries in the context of the MONROE project. We collect a large dataset from more than 300 video streaming experiments. Our dataset consists of network traces, performance indicators captured during the streaming sessions, and experiment metadata. The dataset captures the wide variability in video streaming performance, and unveils how mobile broadband is still not offering consistent quality guarantees across different countries and networks, especially for users on the move. We open source our complete software toolset and provide the video dataset as open data.

International audience

While cloud servers provide a tremendous amount of resources for networked video applications, most successful stories of cloud-assisted video applications are presentational video services, such as YouTube and NetFlix. This article surveys the recent advances on delay-sensitive video computations in the cloud, which are crucial to cloud-assisted conversational video services, such as cloud gaming, Virtual Reality (VR), Augmented Reality (AR), and telepresence. Supporting conversational video services with cloud resources is challenging because most cloud servers are far away from the end users while these services incur the following stringent requirements: high bandwidth, short delay, and high heterogeneity. In this article, we cover the literature with a top-down approach: from applications and experience, to architecture and management, and to optimization in and outside of the cloud. We also point out major open challenges, hoping to stimulate more research activities in this emerging and exciting direction.

Exploring and annotating collections of images without meta-data is a complex task which requires convenient ways of presenting datasets to a user. Visual analytics and information visualization can help users by providing interfaces, and in this paper, we present an open source application that allows users from any domain to use feature-based clustering of large image collections to perform explorative browsing and annotation. For this, we use various image feature extraction mechanisms, different unsupervised clustering algorithms and hierarchical image collection visualization. The performance of the presented open source software allows users to process and display thousands of images at the same time by utilizing heterogeneous resources such as GPUs and different optimization techniques.

Analysis of medical videos for detection of abnormalities and diseases requires both high precision and recall, but also real-time processing for live feedback and scalability for massive screening of entire populations. Existing work on this field does not provide the necessary combination of retrieval accuracy and performance.; [email protected] this paper, a multimedia system is presented where the aim is to tackle automatic analysis of videos from the human gastrointestinal (GI) tract. The system includes the whole pipeline from data collection, processing and analysis, to visualization. The system combines filters using machine learning, image recognition and extraction of global and local image features. Furthermore, it is built in a modular way so that it can easily be extended. At the same time, it is developed for efficient processing in order to provide real-time feedback to the doctors. Our experimental evaluation proves that our system has detection and localisation accuracy at least as good as existing systems for polyp detection, it is capable of detecting a wider range of diseases, it can analyze video in real-time, and it has a low resource consumption for scalability.

Automatic detection of diseases by use of computers is an important, but still unexplored field of research. Such innovations may improve medical practice and refine health care systems all over the world. However, datasets containing medical images are hardly available, making reproducibility and comparison of approaches almost impossible. In this paper, we present KVASIR, a dataset containing images from inside the gastrointestinal (GI) tract. The collection of images are classified into three important anatomical landmarks and three clinically significant findings. In addition, it contains two categories of images related to endoscopic polyp removal. Sorting and annotation of the dataset is performed by medical doctors (experienced endoscopists). In this respect, KVASIR is important for research on both single- and multi-disease computer aided detection. By providing it, we invite and enable multimedia researcher into the medical domain of detection and retrieval.

Bowel preparation (cleansing) is considered to be a key precondition for successful colonoscopy (endoscopic examination of the bowel). The degree of bowel cleansing directly affects the possibility to detect diseases and may influence decisions on screening and follow-up examination intervals. An accurate assessment of bowel preparation quality is therefore important. Despite the use of reliable and validated bowel preparation scales, the grading may vary from one doctor to another. An objective and automated assessment of bowel cleansing would contribute to reduce such inequalities and optimize use of medical resources. This would also be a valuable feature for automatic endoscopy reporting in the future. In this paper, we present Nerthus, a dataset containing videos from inside the gastrointestinal (GI) tract, showing different degrees of bowel cleansing. By providing this dataset, we invite multimedia researchers to contribute in the medical field by making systems automatically evaluate the quality of bowel cleansing for colonoscopy. Such innovations would probably contribute to improve the medical field of GI endoscopy.

Energy efficiency is a timely topic for modern mobile computing. Reducing the energy consumption of devices not only increases their battery lifetime, but also reduces the risk of hardware failure. Many researchers strive to understand the relationship between software activity and hardware power usage. A recurring strategy for saving power is to reduce operating frequencies. It is widely acknowledged that standard frequency scaling algorithms generally overreact to changes in hardware utilisation. More recent and original efforts attempt to balance software workloads on heterogeneous multicore architectures, such as the Tegra K1, which includes a quad-core CPU and a CUDA-capable GPU. However, it is not known whether it is possible to utilise these processor elements in parallel to save energy. Research into these types of systems are unfortunately often evaluated with the Performance Per Watt (PPW) metric, which is an unaccurate method because it ignores constant power usage from idle components. We show that this metric can end up increase energy usage on the Tegra K1, and give a false impression of how such systems consume energy. In reality, we show that it is much harder to save energy by balancing workloads between the heterogeneous cores of the Tegra K1, where we demonstrate only a 5% energy saving by offloading 10% DCT workload from the GPU to the CPU. Significantly more energy can be saved (up to 50 %) using the appropriate processor for different workloads.

Analysis of medical videos from the human gastrointestinal (GI) tract for detection and localization of abnormalities like lesions and diseases requires both high precision and recall. Additionally, it is important to support efficient, real-time processing for live feedback during (i) standard colonoscopies and (ii) scalability for massive population-based screening, which we conjecture can be done using a wireless video capsule endoscope (camera-pill). Existing related work in this field does neither provide the necessary combination of accuracy and performance for detecting multiple classes of abnormalities simultaneously nor for particular disease localization tasks. In this paper, a complete end-to-end multimedia system is presented where the aim is to tackle automatic analysis of GI tract videos. The system includes an entire pipeline ranging from data collection, processing and analysis, to visualization. The system combines deep learning neural networks, information retrieval, and analysis of global and local image features in order to implement multi-class classification, detection and localization. Furthermore, it is built in a modular way, so that it can be easily extended to deal with other types of abnormalities. Simultaneously, the system is developed for efficient processing in order to provide real-time feedback to the doctors and for scalability reasons when potentially applied for massive population-based algorithmic screenings in the future. Initial experiments show that our system has multi-class detection accuracy and polyp localization precision at least as good as state-of-the-art systems, and provides additional novelty in terms of real-time performance, low resource consumption and ability to extend with support for new classes of diseases.

Holistic medical multimedia systems covering end-to-end functionality from data collection to aided diagnosis are highly needed, but rare. In many hospitals, the potential value of multimedia data collected through routine examinations is not recognized. Moreover, the availability of the data is limited, as the health care personnel may not have direct access to stored data. However, medical specialists interact with multimedia content daily through their everyday work and have an increasing interest in finding ways to use it to facilitate their work processes. In this article, we present a novel, holistic multimedia system aiming to tackle automatic analysis of video from gastrointestinal (GI) endoscopy. The proposed system comprises the whole pipeline, including data collection, processing, analysis, and visualization. It combines filters using machine learning, image recognition, and extraction of global and local image features. The novelty is primarily in this holistic approach and its real-time performance, where we automate a complete algorithmic GI screening process. We built the system in a modular way to make it easily extendable to analyze various abnormalities, and we made it efficient in order to run in real time. The conducted experimental evaluation proves that the detection and localization accuracy are comparable or even better than existing systems, but it is by far leading in terms of real-time performance and efficient resource consumption.

The process of finding diseases and abnormalities during live medical examinations has for a long time depended mostly on the medical personnel, with a limited amount of computer support. However, computer-based medical systems are currently emerging in domains like endoscopies of the gastrointestinal (GI) tract. In this context, we aim for a system that enables automatic analysis of endoscopy videos, where one use case is live computer-assisted endoscopy that increases disease-and abnormality-detection rates. In this paper, a system that tackles live automatic analysis of endoscopy videos is presented with a particular focus on the system's ability to perform in real time. The presented system utilizes different parts of a heterogeneous architecture and can be used for automatic analysis of high-definition colonoscopy videos (and a fully automated analysis of video from capsular endoscopy devices). We describe our implementation and report the system performance of our GPU-based processing framework. The experimental results show real-time stream processing and low resource consumption, and a detection precision and recall level at least as good as existing related work.

Using fiducial markers ensures reliable detection and identification of planar features in images. Fiducials are used in a wide range of applications, especially when a reliable visual reference is needed, e.g., to track the camera in cluttered or textureless environments. A marker designed for such applications must be robust to partial occlusions, varying distances and angles of view, and fast camera motions. In this paper, we present a robust, highly accurate fiducial system, whose markers consist of concentric rings, along with its theoretical foundations. Relying on projective properties, it allows to robustly localize the imaged marker and to accurately detect the position of the image of the (common) circle center. We demonstrate that our system can detect and accurately localize these circular fiducials under very challenging conditions and the experimental results reveal that it outperforms other recent fiducial systems.

Health care has a long history of adopting technology to save lives and improve the quality of living. Visual information is frequently applied for disease detection and assessment, and the established fields of computer vision and medical imaging provide essential tools. It is, however, a misconception that disease detection and assessment are provided exclusively by these fields and that they provide the solution for all challenges. Integration and analysis of data from several sources, real-time processing, and the assessment of usefulness for end-users are core competences of the multimedia community and are required for the successful improvement of health care systems. We have conducted initial investigations into two use cases surrounding diseases of the gastrointestinal (GI) tract, where the detection of abnormalities provides the largest chance of successful treatment if the initial observation of disease indicators occurs before the patient notices any symptoms. Although such detection is typically provided visually by applying an endoscope, we are facing a multitude of new multimedia challenges that differ between use cases. In real-time assistance for colonoscopy, we combine sensor information about camera position and direction to aid in detecting, investigate means for providing support to doctors in unobtrusive ways, and assist in reporting. In the area of large-scale capsular endoscopy, we investigate questions of scalability, performance and energy efficiency for the recording phase, and combine video summarization and retrieval questions for analysis.

This paper presents an open source video quality assessment tool called OpenVQ and OPVQ, an objective quality metric based on work of the Video Quality Experts Group. It was primarily developed to satisfy the need of the community for a freely accessible and easy-to-use quality assessment tool, which would remove the need for using throughly outdated methods that is still prevalent because only those are easily available. OpenVQ is implemented as a toolbox of full-reference quality assessment methods, which can be extended with improved methods in the future. The primary method provided by OpenVQ at this time is OPVQ, the Open Perceptual Video Quality metric. It is very much inspired by the formal model of ITU-T Rec. J.247 Annex B, however with many modifications to deal with errors and inconsistencies in formulas and pseudo code, and restrictions to avoid violating known patents. OPVQ has been validated with several IRRCyN datasets and was found to yield results similar to those reported in ITU.T J.247. OpenVQ implements also PSNR (Peak signal-to-noise-ratio), because it is in high demand in spite of its bad performance, and SSIM (Structural similarity index), an image quality metric that is known to perform quite well in many cases.

This demo dramatically illustrates how replacing 'Classic' TCP congestion control (Reno, Cubic, etc.) with a 'Scalable' alternative like Data Centre TCP (DCTCP) keeps queuing delay ultra-low; not just for a select few light applications like voice or gaming, but even when a variety of interactive applications all heavily load the same (emulated) Internet access. DCTCP has so far been confined to data centres because it is too aggressive---it starves Classic TCP flows. To allow DCTCP to be exploited on the public Internet, we developed DualQ Coupled Active Queue Management (AQM), which allows the two TCP types to safely co-exist. Visitors can test all these claims. As well as running Web-based apps, they can pan and zoom a panoramic video of a football stadium on a touch-screen, and experience how their personalized HD scene seems to stick to their finger, even though it is encoded on the fly on servers accessed via an emulated delay, representing 'the cloud'. A pair of VR goggles can be used at the same time, making a similar point. The demo provides a dashboard so that visitors can not only experience the interactivity of each application live, but they can also quantify it via a wide range of performance stats, updated live. It also includes controls so visitors can configure different TCP variants, AQMs, network parameters and background loads and immediately test the effect.

In the Augmented Reality (AR) applications, high quality relates to an accurate augmentation of virtual objects in the real scene. This can be accomplished only if the position of the observer is accurately known. This boils down to solving image-based location problem by an accurate camera pose (relative position and orientation) estimation, when a stereo or multiple camera setup is used. Consider a relevant application scenario as in a movie production set, where the director is able to preview a scene as an integrated view of the real scene augmented with animated 3D models. The main camera shoots the scene, where as secondary stereo camera pair is used for image registration and localization. The director can view the integrated preview from any viewpoint perfectly, as long as the camera pose estimation is accurate.

This demonstration will showcase mixed reality technologies that we developed for a series of public art performances in Vienna in October 2015 in a collaboration of performance artists and researchers. The focus of the demonstration is on natural interaction techniques that can be used intuitively to control an avatar in a virtual 3D world. We combine virtual reality devices with optical location tracking, hand gesture recognition and smart devices. Conference attendees will be able to walk around in a Minecraft world by physically moving in the real world and to perform actions on virtual world items using hand gestures. They can also test our initial system for shared avatar control, in which a user in the real world cooperates with a user in the virtual world. Finally, attendees will have the opportunity to give us feedback about their experience with our system.

In this paper, we present a demo that utilizes Device Lending via PCI Express (PCIe) in the context of a multi-auditory environment. Device Lending is a transparent, low-latency cross-machine PCIe device sharing mechanism without any the need for implementing application-specific distribution mechanisms. As workload, we use a computer-aided diagnosis system that is used to automatically find polyps and mark them for medical doctors during a colonoscopy. We choose this scenario because one of the main requirements is to perform the analysis in real-time. The demonstration consists of a setup of two computers that demonstrates how Device Lending can be used to improve performance, as well as its effect of providing the performance needed for real-time feedback. We also present a performance evaluation that shows its real-time capabilities of it.

In this paper, we present Heimdallr, a dataset that aims to serve two different purposes. The first purpose is action recognition and pose estimation, which requires a dataset of annotated sequences of athlete skeletons. We employed a crowdsourcing platform where people around the world were asked to annotate frames and obtained more than 3000 fully annotated frames for 42 different sequences with a variety of poses and actions. The second purpose is an improved understanding of crowdworkers, and for this purpose, we collected over 10000 written feedbacks from 592 crowdworkers. This is valuable information for crowdsourcing researchers who explore algorithms for worker quality assessment. In addition to the complete dataset, we also provide the code for the application that has been used to collect the data as an open source software.

In this paper, we present the dataset Right Inflight developed to support the exploration of the match between video content and the situation in which that content is watched. Specifically, we look at videos that are suitable to be watched on an airplane, where the main assumption is that that viewers watch movies with the intent of relaxing themselves and letting time pass quickly, despite the inconvenience and discomfort of flight. The aim of the dataset is to support the development of recommender systems, as well as computer vision and multimedia retrieval algorithms capable of automatically predicting which videos are suitable for inflight consumption. Our ultimate goal is to promote a deeper understanding of how people experience video content, and of how technology can support people in finding or selecting video content that supports them in regulating their internal states in certain situations. Right Inflight consists of 318 human-annotated movies, for which we provide links to trailers, a set of pre-computed low-level visual, audio and text features as well as user ratings. The annotation was performed by crowdsourcing workers, who were asked to judge the appropriateness of movies for inflight consumption.

In this paper, we present the computer-aided diagnosis part of the EIR system [9], which can support medical experts in the task of detecting diseases and anatomical landmarks in the gastrointestinal (GI) system. This includes automatic detection of important findings in colonoscopy videos and marking them for the doctors. EIR is designed in a modular way so that it can easily be extended for other diseases. For this demonstration, we will focus on polyp detection, as our system is trained with the ASU-Mayo Clinic polyp database [5].

Energy efficiency of multimedia processing is a hot topic in modern, mobile computing where the lifetime of battery-powered devices is low. Authors often use power models as tools to evaluate the energy-efficiency of multimedia workloads and processing schemes. A challenge with these models is that they are built without sufficiently deep hardware knowledge and as a result they have the potential to mispredict substantially depending on hardware configuration. Typical rate-based power models can for example mispredict up to 70 % on the Tegra K1 SoC. Inspired by multimedia workloads, we introduce a modelling methodology which can be used to build a generic, high-precision power model for the Tegra K1's GPU and memory. By considering hardware utilisation, rail voltages, leakage currents and clocks, the model achieves an average accuracy above 99 % over all operating frequencies, and has been rigorously tested on several multimedia workloads. Our method exposes detailed insight into hardware and how it consumes energy. This knowledge is not only useful for researchers to understand how power models should be built, but also helps to understand what developers can do to minimise power usage. For example, experiments show that for a DCT benchmark, 3 % power can be saved by utilising non-coherent caches and smaller datatypes.

The challenge of scaling IO performance of multimedia systems to demands of their users has attracted much research. A lot of effort has gone into development of distributed systems that add little latency and computing overhead. For machines in PCI Express (PCIe) clusters, we propose Device Lending as a novel solution which works at a system level.

Interactive panoramic systems are currently on the rise. However, one of the major challenges in such a system is the overhead involved in transferring a full-quality panorama to the client when only a part of the panorama is used to extract a virtual view. Thus, such a system should maximize the user experience while simultaneously minimizing the bandwidth required. In this paper, we apply tiling to deliver different quality levels for different parts of the panorama. Tiling has traditionally been applied to the delivery of very high-resolution content to clients. Here, we apply similar ideas in a real-time interactive panoramic video system. A major challenge lies in the movement of such a virtual view, for which clients' regions of interest change dynamically and independently from each other. We show that our algorithms, which progressively increase in quality toward the point of the view, manage to (i) reduce the bandwidth requirement and (ii) provide a similar quality of experience (QoE) compared to a full panorama system.

Time-dependent applications using TCP often send thin-stream traffic, characterised by small packets and high inter-transmission-times. Retransmissions after packet loss can result in very high delays for such flows as they often cannot trigger fast retransmit. Redundant Data Bundling is a mechanism that preempts the experience of loss for a flow by piggybacking unacknowledged segments with new data as long as the total packet size is lower than the flow maximum segment size. Although successful at reducing retransmission latency, this mechanism had design issues leaving it open for abuse, effectively making it unsuitable for general Internet deployment. In this paper, we have redesigned the RDB mechanism to make it safe for deployment. We improve the trigger for when to apply it and evaluate its fairness towards competing traffic. Extensive experimental results confirm that our proposed modifications allows for inter-flow fairness while maintaining the significant latency reductions from the original RDB mechanism.

In this paper, we present a novel application for elucidating all kind of videos that require expert knowledge, e.g., sport videos, medical videos etc., focusing on endoscopic surgery and video capsule endoscopy. In the medical domain, the knowledge of experts for tagging and interpretation of videos is of high value. As a result of the stressful working environment of medical doctors, they often simply do not have time for extensive annotations. We therefore present a semi-supervised method to gather the annotations in a very easy and time saving way for the experts and we show how this information can be used later on.

By processing video footage from a camera array, one can easily make wide-field-of-view panorama videos. From the single panorama video, one can further generate multiple virtual cameras supporting personalized views to a large number of users based on only the few physical cameras in the array. However, giving personalized services to large numbers of users potentially introduces both bandwidth and processing bottlenecks, depending on where the virtual camera is processed.

Energy consumption is an important concern for mobile devices, where the evolution in battery storage capacity has not followed the power usage requirements of modern hardware. However, innovative and flexible hardware platforms give developers better means of optimising the energy consumption of their software. For example, the Tegra K1 System-on-Chip (SoC) offers two CPU clusters, GPU offloading, frequency scaling and other mechanisms to control the power and performance of applications. In this demonstration, the scenario is live video encoding, and participants can experiment with power usage and performance using the Tegra K1's hardware capabilities. A popular power-saving approach is a "race to sleep" strategy where the highest CPU frequency is used while the CPU has work to do, and then the CPU is put to sleep. Our own experiments indicate that an energy reduction of 28 % can be achieved by running the video encoder on the lowest CPU frequency at which the platform achieves an encoding frame rate equal to the minimum frame rate of 25 Frames Per Second (FPS).

The growing power of processors allows us to implement increasingly complex multimedia algorithms. However, this processor power is only available if the algorithms are implemented in a way that exploits the multi-core parallelism of these processors. Today, this requires that the skillsets required for algorithm development and for parallel programming are tightly combined to achieve this.

This paper is the product of recent advances in research on users' intent during multimedia content retrieval. Our goal is to save bandwidth while streaming video clips from a browsable on-demand service, while maintaining or even improving the users' quality of experience (QoE). Understanding user intent allows us to predict whether streaming a particular video in a low quality constitutes a reduced QoE for a user. However, many VoD streaming services today are used by users for a wide variety of reasons, meaning that user intent cannot be inferred from their use of the service alone. However, our investigation demonstrates that user intent does in most cases coincide with producer intent. We can also demonstrate that the latter can be inferred from the content itself as well as associated metadata. By transitivity, we can choose a default video quality that satisfies the users QoE in the majority of cases.

Delivering high resolution, high bitrate panorama video to a large number of users introduces huge scaling challenges. To reduce the resource requirement, researchers have earlier proposed tiling in order to deliver different qualities in different spatial parts of the video. In our work, providing an interactive moving virtual camera to each user, tiling may be used to reduce the quality depending on the position of the virtual view. This raises new challenges compared to existing tiling approaches as the need for high quality tiles dynamically change. In this paper, we describe a tiling approach of panorama video for interactive virtual cameras where we provide initial results showing the introduced overheads and the potential reduction in bandwidth requirement.

Rules-of-thumb for noticeable and detrimental asynchrony between audio and video streams have long since been established from the contributions of several studies. Although these studies share similar findings, none have made any discernible assumptions regarding audio and video quality. Considering the use of active adaptation in present and upcoming streaming systems, audio and video will continue to be delivered in separate streams; consequently, the assumption that the rules-of-thumb hold independent of quality needs to be challenged. To put this assumption to the test, we focus on the detection, not the appraisal, of asynchrony at different levels of distortion. Cognitive psychologists use the term temporal integration to describe the failure to detect asynchrony. The term refers to a perceptual process with an inherent buffer for short asynchronies, where corresponding auditory and visual signals are merged into one experience. Accordingly, this paper discusses relevant causes and concerns with regards to asynchrony, it introduces research on audiovisual perception, and it moves on to explore the impact of audio and video quality on the temporal integration of different audiovisual events. Three content types are explored, speech from a news broadcast, music presented by a drummer, and physical action in the form of a chess game. Within these contexts, we found temporal integration to be very robust to quality discrepancies between the two modalities. In fact, asynchrony detection thresholds varied considerably more between the different content than they did between distortion levels. Nevertheless, our findings indicate that the assumption concerning the independence of asynchrony and audiovisual quality may have to be reconsidered.

With teleconferencing becoming more accessible as a communication platform, researchers are working to understand the consequences of the interaction between human perception and this unfamiliar environment. Given the enclosed space of a teleconference room, along with the physical separation between the user, microphone and speakers, the transmitted audio often becomes mixed with the reverberating auditory components from the room. As a result, the audio can be perceived as smeared in time, and this can affect the user experience and perceived quality. Moreover, other challenges remain to be solved. For instance, during encoding, compression and transmission, the audio and video streams are typically treated separately. Consequently, the signals are rarely perfectly aligned and synchronous. In effect, timing affects both reverberation and audiovisual synchrony, and the two challenges may well be inter-dependent. This study explores the temporal integration of audiovisual continuous speech and speech syllables, along with a non-speech event, across a range of asynchrony levels for different reverberation conditions. Non-reverberant stimuli are compared to stimuli with added reverberation recordings. Findings reveal that reverberation does not affect the temporal integration of continuous speech. However, reverberation influences the temporal integration of the isolated speech syllables and the action-oriented event, with perceived subjective synchrony skewed towards audio lead asynchrony and away from the more common audio lag direction. Furthermore, less time is spent on simultaneity judgements for the longer sequences when the temporal offsets get longer and when reverberation is introduced, suggesting that both asynchrony and reverberation add to the demands of the task.

In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.

In this article, we take advantage of the user behavior of requesting videos from the top of the related list provided by YouTube to improve the performance of YouTube caches. We recommend that local caches reorder the related lists associated with YouTube videos, presenting the cached content above noncached content. We argue that the likelihood that viewers select content from the top of the related list is higher than selection from the bottom, and pushing contents already in the cache to the top of the related list would increase the likelihood of choosing cached content. To verify that the position on the list really is the selection criterion more dominant than the content itself, we conduct a user study with 40 YouTube-using volunteers who were presented with random related lists in their everyday YouTube use. After confirming our assumption, we analyze the benefits of our approach by an investigation that is based on two traces collected from a university campus. Our analysis shows that the proposed reordering approach for related lists would lead to a 2 to 5 times increase in cache hit rate compared to an approach without reordering the related list. This increase in hit rate would lead to reduction in server load and backend bandwidth usage, which in turn reduces the latency in streaming the video requested by the viewer and has the potential to improve the overall performance of YouTube's content distribution system. An analysis of YouTube's recommendation system reveals that related lists are created from a small pool of videos, which increases the potential for caching content from related lists and reordering based on the content in the cache.

Data-bundling is a useful technique that decreases the delivery delay of packet streams when they are transmitted over noisy channels and are subject to retransmission-based error control. In this paper, we investigate the packet delay statistics for a fully reliable selective repeat automatic repeat request (SR ARQ) where a data-bundling mechanism is employed. In more detail, we discuss a model for data-bundling to analyze the SR ARQ mechanism over wireless channels based on Markov chains. We evaluate various channel error distributions and analyze the buffer occupancy to check if the data-bundling mechanism provides efficient results. We further analyze the queueing, delivery and overall delay statistics at link layer. We found that using data-bundling can improve the delay performance of the SR ARQ mechanism, especially when bursty channels with heavily correlated errors are considered. Thus, this technique can bring useful improvements for real-time services, multimedia, and other delay-sensitive applications over wireless networks.

Modern video cameras normally only capture a single color per pixel, commonly arranged in a Bayer pattern. This means that we must restore the missing color channels in the image or the video frame in post-processing, a process referred to as debayering. In a live video scenario, this operation must be performed efficiently in order to output each frame in real-time, while also yielding acceptable visual quality. Here, we evaluate debayering algorithms implemented on a GPU for real-time panoramic video recordings using multiple 2K-resolution cameras.

Over the last years, video streaming has become one of the most dominant Internet services. A trend now is that due to the increased availability of high-speed internet access, multimedia services are becoming more interactive and immersive. Examples of such applications are both cloud gaming and systems where users can interact with high-resolution content. Over the last few years, hardware video encoders have been built into commodity hardware. We evaluate one of these encoders in a scenario where we have individual streams delivered to the end users. Our results show that we can reduce almost half of the CPU time spent on video processing, while also greatly reducing the power consumption on the system. We also compare the visual video quality and the frame size of the hardware based encoder, and we find no significant difference compared to a software based approach.

The proposed demo shows how our system automatically zooms and pans into tracked objects in panorama videos. At the conference site, we will set up a two-camera version of the system, generating live panorama videos, where the system zooms and pans tracking people using colored hats. Additionally, using a stored soccer game video from a five 2K camera setup at Alfheim stadium in Tromsø from the European league game between Tromsø IL and Tottenham Hotspurs, the system automatically follows the ball.

The interest for wide field of view panorama video is increasing. In this respect, we have an application that uses an array of cameras that overlook a soccer stadium. The input of these cameras are stitched together to provide a panoramic view of the stadium. One of the challenges we face is that large parts of the field are obscured by shadows on sunny days. Such circumstances cause unsatisfying video quality. We have therefore implemented and evaluated multiple algorithms related to high dynamic range (HDR) video. The evaluation shows that a combination of several approaches gives the most useful results in our scenario.

High-resolution panoramic video with a wide field-of-view is popular in many contexts. However, in many examples, like surveillance and sports, it is often desirable to zoom and pan into the generated video. A challenge in this respect is real-time support, but in this demo, we present an end-to-end real-time panorama system with interactive zoom and panning. Our system installed at Alfheim stadium, a Norwegian premier league soccer team, generates a cylindrical panorama from five 2K cameras live where the perspective is corrected in real-time when presented to the client. This gives a better and more natural zoom compared to existing systems using perspective panoramas and zoom operations using plain crop. Our experimental results indicate that virtual views can be generated far below the frame-rate threshold, i.e., on a GPU, the processing requirement per frame is about 10 milliseconds.

Bagadus is a prototype of a soccer analysis application which integrates a sensor system, a video camera array and soccer analytics annotations. The current prototype is installed at Alfheim Stadium in Norway, and provides a large set of new functions compared to existing solutions. One important feature is to automatically extract video events and summaries from the games, i.e., an operation that traditionally consumes a huge amount of time. In this demo, we demonstrate how our integration of subsystems enable several types of summaries to be generated automatically, and we show that the video summaries are displayed with a response time around one second.

This paper presents a dataset of body-sensor traces and corresponding videos from several professional soccer games captured in late 2013 at the Alfheim Stadium in Tromsø, Norway. Player data, including field position, heading, and speed are sampled at 20Hz using the highly accurate ZXY Sport Tracking system. Additional per-player statistics, like total distance covered and distance covered in different speed classes, are also included with a 1Hz sampling rate. The provided videos are in high-definition and captured using two stationary camera arrays positioned at an elevated position above the tribune area close to the center of the field. The camera array is configured to cover the entire soccer field, and each camera can be used individually or as a stitched panorama video. This combination of body-sensor data and videos enables computer-vision algorithms for feature extraction, object tracking, background subtraction, and similar, to be tested against the ground truth contained in the sensor traces.

Panorama video is becoming increasingly popular, and we present an end-to-end real-time system to interactively zoom and pan into high-resolution panoramic videos. Compared to existing systems using perspective panoramas with cropping, our approach creates a cylindrical panorama. Here, the perspective is corrected in real-time, and the result is a better and more natural zoom. Our experimental results also indicate that such zoomed virtual views can be generated far below the frame-rate threshold. Taking into account recent trends in device development, our approach should be able to scale to a large number of concurrent users in the near future.

This special issue is concerned with the latest developments in state-of-the-art adaptive media streaming technologies and applications.

The importance of winning has increased the role of performance analysis in the sports industry, and this underscores how statistics and technology keep changing the way sports are played. Thus, this is a growing area of interest, both from a computer system view in managing the technical challenges and from a sport performance view in aiding the development of athletes. In this respect, Bagadus is a real-time prototype of a sports analytics application using soccer as a case study. Bagadus integrates a sensor system, a soccer analytics annotations system, and a video processing system using a video camera array. A prototype is currently installed at Alfheim Stadium in Norway, and in this article, we describe how the system can be used in real-time to playback events. The system supports both stitched panorama video and camera switching modes and creates video summaries based on queries to the sensor system. Moreover, we evaluate the system from a systems point of view, benchmarking different approaches, algorithms, and trade-offs, and show how the system runs in real time.

High resolution, wide field of view video generated from multiple camera feeds has many use cases. However, processing the different steps of a panorama video pipeline in real-time is challenging due to the high data rates and the stringent requirements of timeliness. We use panorama video in a sport analysis system where video events must be generated in real-time. In this respect, we present a system for real-time panorama video generation from an array of low-cost CCD HD video cameras. We describe how we have implemented different components and evaluated alternatives. We also present performance results with and without co-processors like graphics processing units (GPUs), and we evaluate each individual component and show how the entire pipeline is able to run in real-time on commodity hardware.

Sports analytics is a growing area of interest, both from a computer system view to manage the technical challenges and from a sport performance view to aid the development of athletes. In this paper, we present Bagadus, a prototype of a sports analytics application using soccer as a case study. Bagadus integrates a sensor system, a soccer analytics annotations system and a video processing system using a video camera array. A prototype is currently installed at Alfheim Stadium in Norway, and in this paper, we describe how the system can follow and zoom in on particular player(s). Next, the system will playout events from the games using stitched panorama video or camera switching mode and create video summaries based on queries to the sensor system. Furthermore, we evaluate the system from a systems point of view, benchmarking different approaches, algorithms and tradeoffs.

In this paper, we take advantage of the user behavior of requesting videos from the related list provided by YouTube and the user behavior of requesting videos from the top of this related list to improve the performance of YouTube's caches. We recommend a related list reordering approach which modifies the order of the videos shown on the related list based on the content in the cache. The main goal of our reordering approach is to push the contents already in the cache to the top of the related list and push non-cached contents towards the bottom, which increases the likelihood that the already cached content will be chosen by the viewer. We analyze the benefits of our approach by an investigation that is based on two traces collected from an university campus. Our analysis shows that the proposed reordering approach for related list would lead to a 2 to 5 times increase in cache hit rate compared to an approach without reordering the related list. The increase in hit rate would lead to a 5.12% to 18.19% reduction in server load or back-end bandwidth usage. This increase in hit rate and reduction in back-end bandwidth reduces the latency in streaming the video requested by the viewer and has the potential to improve the overall performance of YouTube's content distribution system. An analysis of YouTube's recommendation system reveals that related lists are created from a small pool of videos, which increases the potential for caching content from related lists and reordering based on the content in the cache.

In this dataset paper, we present and make available real-world measurements of the throughput that was achieved at the application layer when adaptive HTTP streaming was performed over 3G networks using mobile devices. For the streaming sessions, we used popular commute routes in and around Oslo (Norway) traveling with different types of public transportation (metro, tram, train, bus and ferry). We also have a few logs using a car. Each log provides a times-tamp, GPS coordinates and the measured number of bytes downloaded for approximately every second of the route. The dataset can be used in several ways, but the most obvious application is to emulate the same network bandwidth behavior (on specific geographical positions) for repeated experiments.

Following advice from the YouTube recommendation system is one of the ways users browse through the videos offered by YouTube. The system presents related videos based on several factors depending on the current video requested. This related videos list can be used by caching infrastructure to reduce network bandwidth consumption. In this paper, we analyze the differences between user-specific recommendation lists. We perform this analysis on 100s of user nodes from all around the world divided into 4 geographical regions using PlanetLab. Based on our analysis, we find that the related videos differ less in the top half (1-10) of the related video list offered by YouTube compared to the bottom half (11-20). Based on our analysis, we suggest that, caching or prefetching of the Top 10 of the related videos is advantageous over a period of time than caching the whole list offered by YouTube.

Even though shared-memory concurrency is a paradigm frequently used for developing parallel applications on small- and middle-sized machines, experience has shown that it is hard to use. This is largely caused by synchronization primitives which are low-level, inherently non-deterministic, and, consequently, non-intuitive to use. In this paper, we present the Nornir run-time system. Nornir is comparable to well-known frameworks such as MapReduce and Dryad that are recognized for their efficiency and simplicity. Unlike these frameworks, Nornir also supports process structures containing branches and cycles. Nornir is based on the formalism of Kahn process networks, which is a shared-nothing, message-passing model of concurrency. We deem this model a simple and deterministic alternative to shared-memory concurrency. Experiments with real and synthetic benchmarks on up to 8 CPUs show that performance in most cases scales almost linearly with the number of CPUs, when not limited by data dependencies. We also show that the modeling flexibility allows Nornir to outperform its MapReduce counterparts using well-known benchmarks.

A large number of live segmented adaptive HTTP video streaming services exist in the Internet today. These quasi-live solutions have been shown to scale to a large number of concurrent users, but the characteristic on-off traffic pattern makes TCP behave differently compared to the bulk transfers the protocol is designed for. In this paper, we analyze the TCP performance of such live on-off sources, and we investigate possible improvements in order to increase the resource utilization on the server side. We observe that the problem is the bandwidth wastage because of the synchronization of the on period. We investigate four different techniques to mitigate this problem. We first evaluate the techniques on pure on-off traffic using a fixed quality and then repeat the experiments with quality adaptation.

In this tutorial we present dynamic adaptive streaming over HTTP ranging from content creation to consumption. It particular, it provides an overview of the recently ratified MPEG-DASH standard, how to create content to be delivered using DASH, its consumption, and the evaluation thereof with respect to competing industry solutions. The tutorial can be roughly clustered into three parts. In part I we will provide an introduction to DASH, part II covers content creation, delivery, and consumption, and, finally, part III deals with the evaluation of existing (open source) MPEG-DASH implementations compared to state-of-art deployed industry solutions.

We present a dataset -- a real-world, server-side packet trace [8] -- from Anarchy Online [9]. Anarchy Online is a science fiction-themed massively multiplayer online roleplaying game (MMORPG), published and developed by Funcom [7]. We present statistics from the network traffic and show that it is a representative dataset for similar games. From the dataset, one can extract several key characteristics from such scenarios like payload sizes, packet rates, data delivery latencies, retransmission statistics, loss rates and stream correlation. The dataset can be used several ways: by replaying the game traffic, components like congestion control mechanisms, middlewares, packet schedulers, router queue behaviour, etc. can be analysed. Based on the observed statistics from the trace, such interactive game traffic shows completely different behaviour compared to the greedy, high-rate streams most network mechanisms are designed for, e.g., file download, video streaming and web-surfing. We hope that the dataset can be used to push research forward in the field of system support for games.

Locating content in existing video archives is both a time and bandwidth consuming process since users might have to download and manually watch large portions of superfluous videos. In this paper, we present two novel prototypes using an Internet based video composition and streaming system with a keyword-based search interface that collects, converts, analyses, indexes, and ranks video content. At user requests, the system can automatically sequence out portions of single videos or aggregate content from multiple videos to produce a single, personalized video stream on-the-fly.

A lot of people around the world commute using public transportation and would like to spend this time viewing streamed video content such as news or sports updates. However, mobile wireless networks typically suffer from severe bandwidth fluctuations, and the networks are often completely unresponsive for several seconds, sometimes minutes. Today, there are several ways of adapting the video bitrate and thus the video quality to such fluctuations, for example, using scalable video codecs or segmented adaptive HTTP streaming that switches between nonscalable video streams encoded in different bitrates. Still, for a better long-term video playout experience that avoids disruptions and frequent quality changes while using existing video adaptation technology, it is desirable to perform bandwidth prediction and planned quality adaptation. This article describes a video streaming system for receivers equipped with a GPS. A receiver's download rate is constantly monitored, and periodically reported back to a central database along with associated GPS positional data. Thus, based on the current location, a streaming device can use a GPS-based bandwidth-lookup service in order to better predict the near-future bandwidth availability and create a schedule for the video playout that takes likely future availability into account. To create a prototype and perform initial tests, we conducted several field trials while commuting using public transportation. We show how our database has been used to predict bandwidth fluctuations and network outages, and how this information helps maintain uninterrupted playback with less compromise on video quality than possible without prediction.

One of the main challenges when streaming video to mobile devices is to handle fluctuating bandwidth and frequent network outages as the device is brought in and out of areas with network coverage. In this paper, we propose bitrate and video quality planning algorithms for mobile streaming scenarios using a GPS-based bandwidth-lookup service in order to reduce frequently changing video quality and the number of playout interruptions. Our real world experiments using an adaptive segmented HTTP video streaming system while travelling on popular commute routes indicate that the users' quality of experience is greatly increased.

Networked devices often come equipped with multiple network interfaces, and bandwidth aggregation is one of the many possible benefits of using multiple interfaces simultaneously. Real-world networks introduce several challenges that have often been ignored by related work on bandwidth aggregation. The challenges include limited connectivity due to NAT-boxes, link heterogeneity and link variability. In this paper, we present a transparent solution for proxy-based bandwidth aggregation that is able to overcome the different deployment and link heterogeneity challenges present in real-world networks. Our focus has been on increasing the performance of bandwidth-intensive UDP-based applications, and through evaluation we show that our solution efficiently aggregates bandwidth and increases the in-order throughput. Previously, we introduced a multi-link UDP proxy solution that improves in-order throughput. This paper presents a significant extension and improvement in terms of support for middle-boxes (NAT), congestion control, a client-based resequencer and support for all operating systems.

Nowadays, several live and on-demand streaming solutions use HTTP for signaling and data delivery. A frequently used technique is to chop a continuous stream into segments, encode these in multiple qualities and make these available for download using plain HTTP methods. This approach has become known as dynamic adaptive segment streaming over HTTP. Its advantage is that the deployed web infrastructure is easily reused, even for live segment streaming. In this case, however, it is not strictly bulk traffic. We show in this paper, that the streaming source is essentially an on-off source. Furthermore, this paper analyzes several client-controlled segment request strategies for live adaptive HTTP segment streaming. We present experimental results showing the benefits and drawbacks of each strategy with respect to achieved video quality, smoothness of playback and end-to-end delay. We show that it matters how clients request segments. The results indicate strongly that synchronization of client requests has a negative impact on router queues and leads to increased packet loss, and should thus be avoided to achieve a high goodput.

In this demo, we present the P2G framework designed for processing distributed real-time multimedia data. P2G supports arbitrarily complex dependency graphs with cycles, branches and deadlines. P2G is implemented to scale transparently with available resources, i.e., a concept familiar from the cloud computing paradigm. Additionally, P2G supports heterogeneous computing resources, such as x86 and GPU processing cores. We have implemented an interchangeable P2G kernel language which is meant to expose fundamental concepts of the P2G programming model and ease the application development. Here, we demonstrate the P2G execution node using a MJPEG encoder as an example workload when dynamically adding and removing processing cores.

Streaming video over the Internet requires mechanisms that limit the streams' bandwidth consumption within its fair share. TCP streaming guarantees this and provides lossless streaming as a side-effect. Adaptation by packet drop does not occur in the network, and excessive startup latency and stalling must be prevented by adapting the bandwidth consumption of the video itself. However, when the adaptation is performed during an ongoing session, it may influence the perceived quality of the entire video and result in improved or reduced visual quality of experience. We have investigated visual artifacts that are caused by adaptive layer switching -- we call them flicker effects -- and present our results for handheld devices in this paper. We considered three types of flicker, namely noise, blur and motion flicker. The perceptual impact of flicker is explored through subjective assessments. We vary both the intensity of quality changes (amplitude) and the number of quality changes per second (frequency). Users' ability to detect and their acceptance of variations in the amplitudes and frequencies of the quality changes are explored across four content types. Our results indicate that multiple factors influence the acceptance of different quality variations. Amplitude plays the dominant role in delivering satisfactory video quality, while frequency can also be adjusted to relieve the annoyance of flicker artifacts.

Devices capable of connecting to multiple, overlapping networks simultaneously are becoming increasingly common. For example, most laptops are equipped with LAN- and WLAN-interfaces, and smart phones can typically connect to both WLANs and 3G mobile networks. At the same time, streaming high-quality video is becoming increasingly popular. However, due to bandwidth limitations or the unreliable and unpredictable nature of some types of networks, streaming video can be subject to frequent periods of rebuffering and characterised by a low picture quality.

No abstract available.

A well known challenge with mobile video streaming is fluctuating bandwidth. As the client devices move in and out of network coverage areas, the users may experience varying signal strengths, competition for the available resources and periods of network outage. These conditions have a significant effect on video quality.

Program obfuscation is often employed by malware in order to avoid detection by anti-virus software, but it has many other legitimate uses, such as copy protection, software licensing or private computing in the cloud. In this paper, we present a program obfuscation method that is based on the combination of strong encryption of code and data and a CPU simulator(CSPIM) that implements the MIPS I instruction set. Our method is different from existing methods in that only a single word (32-bits) of the protected code or data is present as plain-text in main memory. Furthermore, our method allows the possibility of externally supplying the decryption key to the simulator. We have extensively tested the simulator, and it is able to successfully execute C programs compiled by the gcc cross-compiler. Even though purely software-based method cannot provide perfect protection, we argue that this approach significantly raises the bar for reverse-engineers, especially when combined with existing program obfucation techniques.

Multimedia streaming is increasing in popularity and has become one of the dominating services on the Internet today. Even though user devices are often equipped with multiple network interfaces and in reach of several access networks at the same time, media streams are normally communicated over only one of the available Internet connections. In this paper, we explore the challenges and potential benefits of using multiple access networks simultaneously. Exploiting HTTP's capability of handling requests for specific byte ranges of a file, we present the implementation of a lightweight, application-layer, on-demand streaming service that requires no changes to existing servers and infrastructure. Based on real-world experiments with a multihomed host, we investigate the potential performance gains of video-on-demand playout. We achieve a bandwidth aggregation efficiency of 90% when downloading over 3 heterogeneous access networks in parallel. In addition, we analyze the effect of file segmentation on the buffer requirements and the startup latency.

Work-stealing is todays algorithm of choice for dynamic load-balancing of irregular parallel applications on multiprocessor systems. We have evaluated the algorithm's efficiency on a variety of workloads, including scatter-gather workloads, which occur in common algorithms such as MapReduce. We have discovered that work-stealing scheduling suffers serious scalability problems with fine-grained parallelism because of contention over run-queues. We therefore propose a simple modification to the work-stealing algorithm that significantly improves its performance on scatter-gather workloads, without any negative impact on other types of workloads.

We conjecture that composition of video events from various sources into personalized video playouts will become an important part of next generation streaming systems. Here, video search is a key component since it enables users to retrieve candidate video events based on their interests. One of the main challenges, however, is to analyze the videos in order to correctly identify the various events used to annotate and index the video data. Key problems with current video analysis solutions include that they 1) are complex and therefore require a lot of processing time resulting in large delays; 2) that they can only identify a limited set of events; and 3) that they are still too inaccurate, both giving false positives and failing to find all events. In our Davvi prototype, we therefore extract metadata for our video search engine by combining existing automatic video analysis tools with currently untapped textual information available in the Internet. This provides an end-user experience where textual query results can be combined dynamically into seamless, highly personalized video playouts using an adaptive torrent-like HTTP streaming solution.

Multicast delivery for video streaming gains credibility with the introduction of commercial IPTV. We therefore revisit patching, a video-on-demand idea from the 1990s. We have built Pull-Patching, an approach that combines the patching ideas with adaptive segmented HTTP streaming, a unicast technique that is used by most commercial providers of large-scale, true video-on-demand in the Internet today. The prototype is tested in a combined Internet and lab en- vironment where we show the influence of practical factors like packet loss, delay and limited resource availability, and identify several details that require further study.

Current segmented HTTP streaming systems provide scalable and quality adaptive video delivery services to a huge number of users. However, while they support a wide range of bandwidths and enable arbitrary content-based composition, their current formats have shortcomings like large overheads, live streaming delays, etc. We have therefore developed an adaptive media player that works around these problems while still using standard components like H.264/AVC for video, and MP3 for audio. The system's adaptivity allows the player to pick a quality level that makes good use of available bandwidth and CPU resources while at the same time maintaining smooth uninterrupted playback, as well as offering near instant seek and startup times.This paper presents an appropriate way of coding the segments and a simple multimedia container format that is optimized for adaptive streaming and video composition over HTTP. We show that our format is sufficiently advanced to contain any payload type, while being trivial to parse and translate to other container formats. Additionally, we show that our format is second to none in terms of overhead, without incurring any penalties on live streaming.

Video streaming ranks among the most popular services offered through the Internet today. At the same time, accessing the Internet over public WiFi and 3G networks has become part of our everyday lives. However, streaming video in wireless environments is often subject to frequent periods of rebuffering and characterized by low picture quality. In particular, achieving smooth and quality-adaptive streaming of live video poses a big challenge in mobile scenarios.

When used efficiently, modern multicore architectures, such as Cell and GPUs, provide the processing power required by resource demanding multimedia workloads. However, the diversity of resources exposed to the programmers, intrinsically requires specific mindsets for efficiently utilizing these resources - not only compared to an x86 architecture, but also between the Cell and the GPUs. In this context, our analysis of 14 different Motion-JPEG implementations indicates that there exists a large potential for optimizing performance, but there are also many pitfalls to avoid. By experimentally evaluating algorithmic choices, inter-core data communication (memory transfers) and architecture-specific capabilities, such as instruction sets, we present tips, tricks and troubles with respect to efficient utilization of the available resources.

Abstract In this article, we evaluate the performance effects of I/O bound workloads on a specific virtual machine, an important component of an enterprise cloud computing infrastructure. In particular, we demonstrate that (1) the I/O workload of one guest system may adversely affect the I/O performance of another for the XEN hypervisor and (2) the general I/O performance is degraded due to various overheads. Next, we have devised a light‐weight, complementary, backwards‐compatible alternative to hypervisor‐based virtualization techniques called BONSAI. Our software provides low‐overhead I/O performance isolation by transparently applying traffic shaping to system calls in a cost‐effective manner. Using this system, I/O resource consumption can be controlled at a very fine granularity. Furthermore, using video streaming experiments, where we limit the I/O bandwidth available to each service, we show that we are able to achieve the required level of resource isolation on a per process basis with only a negligible CPU overhead and without reducing the I/O performance. Copyright © 2010 John Wiley & Sons, Ltd.

Software development tools have not adapted to the growing popularity of multi-core CPUs, and developers are still "stuck"' with low-level and high-cost thread abstractions. The situation is becoming even more complicated with the advent of heterogenuous computing. In this article, we point out some drawbacks of high-level abstractions currently in use, and propose Kahn process networks (KPN) as a more high-level and efficient abstraction for developing parallel applications. We show that the native POSIX mechanisms (threads and message queues) perform suboptimally as an implementation vehicle for KPNs, and we present an implementation of a run-time environment that can execute KPNs with less overhead. Our evaluation shows the advantages and disadvantages of statically mapping Kahn processes to CPUs.

Experience has shown that development using shared-memory concurrency, the prevalent parallel programming paradigm today, is hard and synchronization primitives nonintuitive because they are low-level and inherently nondeterministic. To help developers, we propose Kahn process networks, which are based on message-passing and shared-nothing model, as a simple and flexible tool for modeling parallel applications. We argue that they are more flexible than MapReduce, which is widely recognized for its efficiency and simplicity. Nevertheless, Kahn process networks are equally intuitive to use, and, indeed, MapReduce is implementable as a Kahn process network. Our presented benchmarks (word count and k-means) show that a Kahn process network framework permits alternative implementations that bring significant performance advantages: the two programs run by a factor of up to $\sim 2.8$ (word-count) and $\sim 1.8$ (k-means) faster than their implementations for Phoenix, which is a MapReduce framework specifically optimized for executing on multicore machines.

With the increasing deployment of wireless technologies, such as WLAN, HSDPA, and WiMAX, it is often the case that simultaneous coverage of several access networks is available to a single user device. In addition, devices are also often equipped with multiple network interfaces. Thus, if we can exploit all available network interfaces at the same time, we can obtain advantages like the aggregation of bandwidth and increased fault tolerance. However, the heterogeneity and dynamics of the links also introduce challenges. Due to different link delays, sending packets of the same flow over multiple heterogeneous paths causes the reordering of packets. In this paper, we quantify the impact of network heterogeneity and the use of multiple links on IP packet reordering. We show with practical measurements, according to commonly used metrics, that packet reordering over multiple links exceeds the reordering caused by common connections in high-speed, wide- area networks. We also demonstrate that heterogeneity and reordering exceed the assumptions presented in related work. By using sufficiently large buffers, packet reordering can be avoided. However, for devices with high resource constraints, the workload of using large buffers is expensive. Sender-side solutions of dividing and scheduling a packet sequence over multiple links can reduce the buffer requirements at the receiver. Initial experiments with a static scheduler, that has knowledge of average link delay and throughput estimates, show that packet reordering can be reduced by only 38% due to the dynamic heterogeneity of the two links.

With today's widespread deployment of wireless technologies, it is often the case that a single communication device can select from a variety of access networks. At the same time, there is an ongoing trend towards integration of multiple network interfaces into end-hosts, such as cell phones with HSDPA, Bluetooth and WLAN. By using multiple Internet connections concurrently, network applications can benefit from aggregated bandwidth and increased fault tolerance. However, the heterogeneity of wireless environments introduce challenges with respect to implementation, deployment, and protocol compatibility. Variable link characteristics cause reordering when sending IP packets of the same flow over multiple paths. This paper introduces a multilink proxy that is able to transparently stripe traffic destined for multihomed clients. Operating on the network layer, the proxy uses path monitoring statistics to adapt to changes in throughput and latency. Experimental results obtained from a proof-of-concept implementation verify that our approach is able to fully aggregate the throughput of heterogeneous downlink streams, even if the path characteristics change over time. In addition, our novel method of equalizing delays by buffering packets on the proxy significantly reduces IP packet reordering and the buffer requirements of clients.

In this demo, we present DAVVI, a prototype of the next generation multimedia entertainment platform. It delivers multi-quality video content in a torrent-similar way like known systems from Move Networks, Microsoft and Apple do. However, it also provides a brand new, personalized user experience. Through applied search, personalization and recommendation technologies, end-users can efficiently search and retrieve highlights and combine arbitrary events in a customized manner using drag and drop. The created playlists of video segments are then delivered back to the system to improve future search and recommendation results. Here, we demonstrate this system using a soccer example.

There is an increasing trend to include streamed video data in 3D environments. Such environments allow potentially several concurrently visible videos on a single display device, and consequently, network and processing bottlenecks. As a first step towards an avoidance of such problems, we have performed subjective assessments using a 3D application prototype to determine how positioning of video in the 3D environment influences the user perception of reduced-quality videos. Using video clips from several genres, we have compared the influence of various ways to reduce video quality and users' perception of degraded quality. We evaluated the influence of distance and angle of the placement.

Scalable video is an attractive option for adapting the bandwidth consumption of streaming video to the available bandwidth. Fine-grained scalability can adapt most closely to the available bandwidth, but this comes at the cost of a high compression penalty. In the context of VoD streaming to mobile end systems, we have therefore explored whether a similar adaptation to the available bandwidth can be achieved by performing layer switching in coarse-grained scalable videos. In this approach, enhancement layers of a video stream are switched on and off to achieve any desired longer-term bandwidth. We performed user studies to evaluate the idea, and came to the far-from-obvious conclusion that layer switching is viable way for bit-rate savings and fine-grained bit-rate adaptation even for rather short times between layer switches.

Shared-memory concurrency is the prevalent paradigm used for developing parallel applications targeted towards small- and middle-sized machines, but experience has shown that it is hard to use. This is largely caused by synchronization primitives which are low-level, inherently nondeterministic, and, consequently, non-intuitive to use. In this paper, we present the \textit{Nornir} run-time system. Nornir is comparable to well-known frameworks like MapReduce and Dryad, but has additional support for process structures containing cycles. It is based on the formalism of Kahn process networks, which we deem as a simple and deterministic alternative to shared-memory concurrency. Experiments with real and synthetic benchmarks on up to 8 CPUs show that performance in most cases improves almost linearly with the number of CPUs, when not limited by data dependencies.

Massively multi-player online games (MMOGs) have stringent latency requirements and must support large numbers of concurrent players. To handle these conflicting requirements, it is common to divide the virtual environment into virtual regions. As MMOGs are world-spanning games, it is plausible to disperse these regions on geographically distributed servers. Core selection can then be applied to locate an optimal server for placing a region, based on player latencies. Functionality for migrating objects supports this objective, with a distributed name server ensuring that references to the moved objects are maintained. As a result we anticipate a decrease in the aggregate latency for the affected players. The core selection relies on a set of servers and measurements of the interacting players latencies. Measuring these latencies by actively probing the network is not scalable for a large number of players. We therefore explore the use of latency estimation techniques to gather this information.

A large number of network services rely on IP and reliable transport protocols. For applications that provide abundant data for transmission, loss is usually handled satisfactorily, even if the application is latency-sensitive (Wang et al. 2004 ). For data streams where small packets are sent intermittently, however, applications can occasionally experience extreme latencies (Griwodz and Halvorsen 2006 ). As it is not uncommon that such thin-stream applications are time-dependent, any unnecessarily induced delay can have severe consequences for the service provided. Massively Multiplayer Online Games (MMOGs) are a defining example of thin streams . Many MMOGs (like World of Warcraft and Age of Conan) use TCP for the benefits of reliability, in-order delivery and NAT/firewall traversal. It has been shown that TCP has several shortcomings with respect to the latency requirements of thin streams because of the way it handles retransmissions (Griwodz and Halvorsen 2006 ). As such, an alternative to TCP may be SCTP (Stewart et al. 2000 ), which was originally developed to meet the requirements of signaling transport. In this paper, we evaluate the Linux-kernel SCTP implementation in the context of thin streams. To address the identified latency challenges, we propose sender-side only enhancements that reduce the application-layer latency in a manner that is compatible with unmodified receivers. These enhancements can be switched on by applications and are used only when the system identifies the stream as thin. To evaluate the latency performance, we have performed several tests over various real networks and over an emulated network, varying parameters like RTT, packet loss and amount of competing cross traffic. When comparing our modifications with SCTP on Linux and FreeBSD and TCP New Reno, our results show great latency improvements and indicate the need for a separate handling of thin and thick streams.

introduction Share on Introduction to the special section for the best papers of ACM multimedia 2008 Authors: K. Selçuk Candan Arizona State University, USA Arizona State University, USAView Profile , Alberto Del Bimbo Università degli Studi di Firenze, Italy Università degli Studi di Firenze, ItalyView Profile , Carsten Griwodz Simula Research Laboratory, Norway Simula Research Laboratory, NorwayView Profile , Alejandro Jaimes Telefonica Research, Spain Telefonica Research, SpainView Profile Authors Info & Claims ACM Transactions on Multimedia Computing, Communications, and ApplicationsVolume 5Issue 3August 2009 Article No.: 18pp 1–3https://doi.org/10.1145/1556134.1556135Published:14 August 2009Publication History 0citation329DownloadsMetricsTotal Citations0Total Downloads329Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

The rapid growth of the Internet multimedia services brings new challenges to how multimedia streams can be delivered to the users over bandwidth-constraint networks. Different strategies that exploit multipath streaming in order to provide better utilization of the Internet resources have been proposed by the research community. However, there exists no metric that allows us to evaluate how close these strategies are to the optimal resource utilization. This paper proposes a static benchmarking system that models the best possible distribution of streams along multiple paths in an overlay network that is shared by several senders and receivers. We have tested it with several different network topologies, and present the test results in this paper.

The High-Speed Downlink Packet Access (HSDPA) communication protocol improves downlink performance on mobile networks and is currently being deployed in networks around the world. In Norway, two operators started offering this service in 2007. This article discusses performance measurements done in Telenor’s HSDPA network in Oslo in the context of video streaming. In particular, we present the results of streaming experiments measuring characteristics such as packet loss, latency, jitter and bit-errors. Based on this data, we evaluate the HSDPA network’s suitability for real-time video streaming, and discuss what can be done to improve the quality of video streaming under the observed network characteristics.

Latency reduction in distributed interactive applications has been studied intensively. Such applications may have stringent latency requirements and dynamic user groups. We focus on using application-layer multicast with a centralized approach to the group management. The groups are organized in overlay networks that are created using graph algorithms. We investigate many spanning tree problems with particular focus on reducing the diameter of a tree, i.e., the maximum pairwise latency between users. In addition, we focus on reducing the time it takes to execute membership changes. In that context, we use core-selection heuristics to find well-placed group nodes, and edge-pruning algorithms to reduce the number of edges in an otherwise fully meshed overlay. Our edge-pruning algorithms strongly connect well-placed group nodes to the remaining group members, to create new and pruned group graphs, such that, when a tree algorithm is applied to a pruned group graph, it is manipulated into creating trees with a smaller diameter. We implemented and analyzed experimentally spanning-tree heuristics, core-selection heuristics and edge-pruning algorithms. We found that faster heuristics that do not explicitly optimize the diameter are able to compete with slower heuristics that do optimize it.

A wide range of Internet-based services that use reliable transport protocols display what we call thin-stream properties. This means that the application sends data with such a low rate that the retransmission mechanisms of the transport protocol are not fully effective. In time-dependent scenarios where the user experience depends on the data delivery latency, packet loss can be devastating for the service quality. In order to reduce application-layer latency when packets are lost, we have implemented modifications to the TCP retransmission mechanisms in the Linux kernel. The changes are only active when thin-stream properties are detected, thus not affecting TCP behaviour when the stream is not thin. In this paper, we show the latency improvements from these thin-stream modifications. We have tested several thin-stream applications like remote terminals (SSH) and audio conferencing (Skype), and we evaluate the user experience with and without the TCP modifications. Our experimental results show that our modifications allow TCP to recover earlier from packet loss. Furthermore, user surveys indicate that the majority of users easily detect improvements in the perceived quality of the tested applications.

Distributed interactive applications may have stringent latency requirements and dynamic user groups. These applications may benefit from a group communication system, and to improve the system support for such applications, we investigate graph algorithms that construct low-latency overlay networks for application-layer multicast. In particular, we focus on reducing the diameter and the pair-wise latencies in the overlay. The overlay construction time is also considered, as it is often time-dependent in our dynamic target applications. Here, we have implemented and experimentally analyzed spanning-tree heuristics and mesh construction heuristics, and compared their performance and applicability to distributed interactive applications. We found that trees are faster to construct and save considerable amounts of resources in the network. Meshes, on the other hand, yield lower pair-wise latencies and increases the fault tolerance, but at the expense of increased resource consumption.

We describe the ADIMUS architecture which addresses the problem of maintaining the subjective quality of multimedia streaming for a mobile user. In contrast to other works, the entire end-to-end path of the video stream is considered. Adaptation mechanisms for maintaining quality include time-critical handovers, overlay routing and network estimation techniques. Our architecture is built on overlays that provides the necessary functionality for a video streaming service. The paper highlights the key components that ADIMUS advocates to support quality streaming from server to mobile client.

Today, major newspapers and TV stations make live and on-demand audio/video content available, video-on-demand services are becoming common and even personal media are frequently uploaded to streaming sites. The discussion about the best transport protocol for streaming has been going on for years. Currently, HTTP-streaming is usual although the transport of streaming media data over TCP is hindered by TCP's probing behavior, which results in the rapid reduction and slow recovery of the packet rates. On the other hand, UDP has been criticized for being unfair against TCP, and it is therefore often blocked by access network providers.

TCP is frequently used for interactive multimedia applications like online games and voice-over-IP (VoIP) because it avoids firewall issues. However, traffic analysis shows that these streams usually have small packets and a low packet rate, and that in case of loss, severe latency penalties occur for all existing TCP variations in Linux [5]. In this demonstration, we show how small TCP enhancements greatly improve the perceived quality of such low latency, interactive applications.

General-purpose CPUs with multiple cores are established products, and new heterogeneous technology like the Cell broadband engine and general-purpose GPUs bring an even higher degree of true multi-processing into the market. However, means for utilizing the processing power is immature. Current tools typically assume that exclusive use of these resources is sufficient, but this assumption will soon be invalid because the interest in using their processing power for general-purpose tasks. Among the applications that can benefit from such technology is transcoding support for distributed media applications, where remote participants join and leave dynamically. Transcoding consists of several clearly separated processing operations that consume a lot of resources, such that individual processing units are unable to handle all operations of a session of arbitrary size. The individual operations can then be distributed over several processing units, and data must be moved between them according to the dependencies between operations. Many multi-processor scheduling approaches exist, but to the best of our knowledge, a challenge is still to find mechanisms that can schedule dynamic workloads of communicating operations while taking both the processing and communication requirements into account. For such applications, we believe that feasible scheduling can be performed in two levels, i.e., divided into the task of placing a job onto a processing unit and the task of multitasking time-slices within a single processing unit. We have implemented some simple high-level scheduling mechanisms and simulated a video conferencing scenario running on topologies inspired by existing systems from Intel, AMD, IBM and nVidia. Our results show the importance of using an efficient high-level scheduler.

This paper focuses on design issues for multimedia distribution architectures and the impact workload characteristics have on architecture design. Our contribution is an analysis of server load and user behavior in a news-on-demand environment, with focus on access patterns, popularity modeling, and the formation of traffic peaks. Finally, we evaluate an existing synthetic workload generator, MediSyn, and suggest some enhancements which will improve its suitability for news-on-demand workload modeling.

Real-time delivery of time-dependent data over the Internet is challenging. UDP has often been used to transport data in a timely manner, but its lack of congestion control is often criticized. This criticism is a reason that the vast majority of applications today use TCP. The downside of this is that TCP has problems with the timely delivery of data. A transport protocol that adds congestion control to an otherwise UDP-like behaviour is DCCP For this protocol, late data choice (LDC) [8] has been proposed to allow adaptive applications control over data packets up to the actual transmission time. We find, however, that application developers appreciate other TCP features as well, such as its reliability. We have therefore implemented and tested the LDC ideas for TCP. It allows the application to modify or drop packets that have been handed to TCP until they are actually transmitted to the network. This is achieved with a shared packet ring and indexes to hold the current status. Our experiments show that we can send more useful data with LDC than without in a streaming scenario. We can therefore claim that we achieve a better utilization of the throughput, giving us a higher goodput with LDC than without.

Distributed interactive applications have become increasingly popular, making it important to address their communication needs, where one of the needs is group communication. In this paper, we consider the applications in which it is at any given time possible to divide its users into groups. The group membership changes over time, and the group division is unrelated to the physical proximity. As a way of enabling the group communication in distributed interactive applications, we choose application layer multicast. We use simulation to evaluate several dynamic algorithms for managing overlay multicast trees. They are compared with respect to four metrics that can be relevant for a distributed interactive application. These are total tree cost, diameter, reconfiguration time and stability. We demonstrate algorithms that perform well for these metrics although they do not consider all users during reconfiguration.

The transport of streaming media data over TCP is hindered by TCP's probing behavior that results in the rapid reduction and slow recovery of the packet rates. On the other side, UDP has been criticized for being unfair against TCP connections, and it is therefore often blocked out in the access networks. In this paper, we try to benefit from a combined approach using a proxy that transparently performs transport protocol translation. We translate HTTP requests by the client transparently into RTSP requests, and translate the corresponding RTP/UDP/AVP stream into the corresponding HTTP response. This enables the server to use UDP on the server side and TCP on the client side. This is beneficial for the server side that scales to a higher load when it doesn't have to deal with TCP. On the client side, streaming over TCP has the advantage that connections can be established from the client side, and data streams are passed through firewalls. Preliminary tests demonstrate that our protocol translation delivers a smoother stream compared to HTTP-streaming where the TCP bandwidth oscillates heavily.

Distributed multimedia streaming systems are increasingly popular due to technological advances, and numerous streaming services are available today. On servers or proxy caches, there is a huge scaling challenge in supporting thousands of concurrent users that request delivery of high-rate, time-dependent data like audio and video, because this requires transfers of large amounts of data through several sub-systems within a streaming node. Unnecessary copy operations in the data path can therefore contribute significantly to the resource consumption of streaming operations. Despite previous research, off-the-shelf operating systems have only limited support for data paths that have been optimized for streaming. Additionally, system call overhead has grown with newer operating systems editions, adding to the cost of data movement. Frequently, it is argued that these issues can be ignored because of the continuing growth of CPU speeds. However, such an argument fails to take problems of modern streaming systems into account. The dissipation of heat generated by disks and high-end CPUs is a major problem of data centers, which would be alleviated if less power-hungry CPUs could be used. The power budget of mobile devices, which are increasingly used for streaming as well, is tight, and reduced power consumption an important issue. In this paper, we prove that these operations consume a large amount of resources, and we therefore revisit the data movement problem and provide a comprehensive evaluation of possible streaming data I/O paths in the Linux 2.6 kernel. We have implemented and evaluated several enhanced mechanisms and show how to provide support for more efficient memory usage and reduction of user/kernel space switches for content download and streaming applications. In particular, we are able to reduce the CPU usage by approximately 27% compared to the best approach without kernel modifications, by removing copy operations and system calls for a streaming scenario in which RTP headers must be added to stored data for sequence numbers and timing.

Abstract — Communication in highly interactive distributed applications, such as massive multiplayer online games, can often be performed efficiently using multicast, i.e., application level multicast. However, in applications with a very dynamic group management, the multicast tree will have frequent changes, and in applications that have stringent latency requirement, this operation needs to be fast. Current multicast approaches either have no notion of reconfiguration, they do not care about tree reconstruction latency or wrongly assume that this is a fast, atomic operation. In this paper, we have focused on dynamic reconfiguration and have tested different ways for a node to join a tree. Our results show that this is an important issue for the class of highly interactive distributed applications. I.

On-demand access to remote multimedia content via the Internet is becoming increasingly popular. Applications like video on demand and news on demand are increasingly based on structured multimedia presentations, which give users more options and freedom to interact with the content than just VCR-like operations. For example, authors provide multiple choices for parts of the presentation and the user can select the most interesting one. Distributing such presentations in hierarchical distribution systems comprised of origin server and proxies can lead to new challenges for proxies since entire presentations are large but only small parts of them are actually consumed by each user. We have analyzed this problem for interactive presentations encoded in MPEG-4 and found that we can use the internal structure of MPEG-4 objects to improve performance in a distribution system based on proxy caching.

This paper investigates several aspects of streaming in news-on-demand services on the Internet by analyzing log files of a news service from Norway's largest online newspaper. We investigate both short and long term effects of client usage of the service. By comparing our results with earlier work, we found that variations in server load depend strongly on local culture. Furthermore, we found that there are slight variations between client usage of the audio and video material

The popularity of distributed interactive applications has exploded in the last few years. For example, massive multi-player online games have become a fast growing, multi-million industry with a very high user mass supporting hundreds or thousands of concurrent players. Today, such games are usually client-server applications that use TCP for time-dependent communication. Similar multimedia applications also choose TCP frequently. Very thin data streams are sent over each of these TCP connections, which means that they consume very little bandwidth. TCP has several shortcomings with respect to the latency requirements of such thin streams because of its retransmission handling (C. Griwodz and P. Halvorsen, 2006). An alternative to TCP may be SCTP (R. Stewart, 2000) which was developed to answer the requirements for signaling transport. SCTP has subsequently also been considered more appropriate than TCP for congestion controlled streaming of other time-dependent data. Important reasons are its maintenance of packet boundaries and partial reliability. In this paper, we evaluate the performance of the Linux SCTP implementation for thin streams. Like others before, we identify latency challenges. We also propose some enhancements for reducing the latency compared to the original Linux implementation. We argue for separate handling of thin and thick data streams in SCTP

One of the most difficult tasks when creating an online multi-player game is to provide the players with a consistent view of the virtual world despite the network delays. Most current games use prediction algorithms to achieve this, however measuring the effect of different approaches is difficult. To solve this problem we introduce a simulator called GLS that gives us a fully controlled environment and allows large-scale experiments to evaluate different aspects of the algorithms.

Massive multi-player online games have become a popular, fast growing, multi-million industry with a very high user mass supporting hundreds or thousands of concurrent players. In many cases, these games are centralized and every player communicates with the central server through a time-critical unicast event stream. Funcom's Anarchy Online is one of these; it is based on TCP. We find that its kind of traffic has some interesting properties that inspire changes to protocol or architecture.

One of the most difficult tasks when creating an online multi-player game is to provide the players with a consistent view of the virtual world despite the network delays. Most current games use prediction algorithms to achieve this, but usually it does not go beyond applying the DIS [2] dead reckoning algorithm proposed in the mid-90s. In this paper we introduce a simulator called GLS that allows us to evaluate different aspects of DIS and its variations. We examine the impact of prediction and clock synchronization on game consistency. We also evaluate the convergence algorithm we introduce here. Furthermore we look into ways for compensating increasing delays to keep the player's view of the game state sufficiently consistent with other players.

Quality of service guarantees for multimedia communication systems have been considered on several abstraction levels. In the multimedia networking field it is typical to identify the minimal QoS requirements of an application to save resources by guaranteeing its functionality. Many of these applications can operate in spite of an imperfect delivery of media data, while other applications such as distributed databases or distributed file systems consider perfect QoS necessary but accept delay. The basic problems of the latter is the consistency of their data, while the former require a consistent perception of the content. More generically, both QoS requirements can be interpreted as a problem of maintaining a consistent system state. Consequently we assume that many distributed applications, including most distributed multimedia applications, can fulfil their tasks in spite of imperfect consistency. Since the application requirements differ widely, the elements that make up "consistency" must be separated and classified. This paper introduces Consistency QoS and proposes a classification of elements that determine an application's consistency requirements. The low level QoS requirements that these separate parameters rely on are shown, and example parameter sets for application classes are given.

Replicating data and services at multiple networked computers increases the service availability of distributed systems. This paper presents the design and implementation architecture of a replication mechanism for a distributed multimedia system medianode which is developed as an infrastructure to share multimedia-enhanced teaching materials among lecture groups. With the replication mechanism, medianode provides enhanced access to presentation materials in both connected and disconnected operation modes. The main contribution of this paper is the identification of new replication requirements in distributed media systems and a multicast-based update propagation mechanism by which not only the update events are signaled, but also the updated data are exchanged between replication managers.

In contrast to audio which is often streamed as complete music titles or even as a life feed from a radio station, video in today's Internet is almost only available as small clips and pre-generated programs. Although some of the problems concerning AV streaming are reasonably solved right now, some work in fields like wide area distribution systems need further investigation to make applications like "true video-on-demand" work. Our research and the one of many others is focused on problems that have to be solved to make application like VoD work in the Internet. It is mainly concerned with wide area distribution. We present a platform for experimental VoD research which is thought to support researchers working on VoD and wide area distribution for audio and video content. This platform offers researchers the possibility to implement their ideas without building a complete streaming environment and in addition allows the combination of different implementations. After motivating the development of our platform we present the design of our platform, give an overview of the actual implementation and the existing components that we have already built. Finally, example scenarios for the use of our platform in research are given.

We think that web caches will soon have to better support multimedia demands.In this paper we present a cache server design for internet video on demand (VoD) systems.1.1

S.21-28

Large scale video-on-demand systems require that the servers offering the video retrieval and playback services are arranged as a distributed system in order to support a large number of concurrent streams.If such a system is hierarchical, an end-node server handles the requests from a particular area, the next server in the hierarchy takes the request over for several end-node servers if those can not answer the request and so on.This architecture provides for cost efficiency, reliability and scalability of servers.The end-node servers store only a limited set of the overall available information which changes over time due to user interests.If a video is requested which is not available, this server contacts the next server in the hierarchy.To decide the size and location of the video servers and the location of videos in the hierarchy, the access behaviour of users must be considered.Various models for the simulation of user behavior (and thus, of the load induced on the video servers) have been presented in the literature.Only a few of these models are designed to take long-term effects into account because the basis for most of the models are short-term influences on a single' video server and the load on this single machine.In this paper we describe a new user behavior mode1 and show that various assumptions made within other models are unrealistic.

Multimedia communication deals with the transfer, protocols, services, and mechanisms of discrete media data (such as text and graphics) and continuous media data (like audio and video) in/over digital networks. Such a communication requires all involved components to be capable of handling a well-defined quality of service (QoS). The most important QoS parameters are used to request: (1) the required capacities of the involved resources, (2) compliance to end-to-end delay and jitter as timing restrictions, and (3) restriction of the loss characteristics. In this paper, we describe the necessary issues and study the ability of current networks and communication systems to support distributed multimedia applications. Further, we discuss upcoming approaches and systems that promise to provide the necessary mechanisms and consider which issues are missing for a complete multimedia communication infrastructure.