We present Statesman, a network-state management service that allows multiple network management applications to operate independently, while maintaining network-wide safety and performance invariants. Network state captures various aspects of the network such as which links are alive and how switches are forwarding traffic. Statesman uses three views of the network state. In observed state, it maintains an up-to-date view of the actual network state. Applications read this state and propose state changes based on their individual goals. Using a model of dependencies among state variables, Statesman merges these proposed states into a target state that is guaranteed to maintain the safety and performance invariants. It then updates the network to the target state. Statesman has been deployed in ten Microsoft Azure datacenters for several months, and three distinct applications have been built on it. We use the experience from this deployment to demonstrate how Statesman enables each application to meet its goals, while maintaining network-wide invariants.

Different 3D tele-immersive (3DTI) activities pose different prioritized requirements for application and network-level quality of service (QoS) to ensure a strong quality of experience (QoE) for participants. Some applications put heavy weight on audio quality, some consider higher quality for upper body video streams, and some seek very low end-to-end interactivity delay. In addition, a variation in streaming content may arise in the 3DTI space due to the participants' change in interests (e.g., view change). Therefore, there is a need for an adaptive multistream, multisite 3DTI session management strategy that is not only unobtrusive, but also optimizes prioritized QoS parameters in 3DTI content distribution based on user activity and content variation. To address this next generation session management problem, we revisit the design space of multistream and multisite 3DTI session layer. We present an evolutionary 3DTI session optimization approach using Open Session Management (OSM) architecture that uses a global view of participants and overlays network conditions to optimize QoS parameters. Experimental results with PlanetLab traces show that our optimization process is unobtrusive, and the optimized TI sessions provide higher satisfaction to the participants (in some cases up to 50% higher) compared to the current solutions in the 3DTI space.

3D Teleimmersive (TI) Networks enable collaborative interactive activities in the same virtual space. The geographically distributed multi-site nature of TIs, the heterogeneity and multi-modality of their I/O devices and the diversity of activities in these environments pose new challenges in designing networked I/O device management for TIs not addressed by existing systems. In this paper we describe Decima, a holistic virtualized I/O Management. Decima enables dynamic seamless and universal access and management to asymmetric Teleimmersive activities using a large scale of heterogeneous TI devices. Our experiments show that Decima is lightweight in terms of CPU overhead and end-to-end delay and scales well as the number of devices in the TI network increases.

Multi-tenant data centers are complex environments, running thousands of applications that compete for the same infrastructure resources and whose behavior is guided by (sometimes) divergent configurations. Small workload changes or simple operator tasks may yield unpredictable results and lead to expensive failures and performance degradation. In this paper, we propose a holistic approach for detecting operational problems in data centers. Our framework, FlowDiff, collects information from all entities involved in the operation of a data center -- applications, operators, and infrastructure -- and continually builds behavioral models for the operation. By comparing current models with pre-computed, known-to-be-stable models, FlowDiff is able to detect many operational problems, ranging from host and network failures to unauthorized access. FlowDiff also identifies common system operations (e.g., VM migration, software upgrades) to validate the behavior changes against planned operator tasks. We show that using passive measurements on control traffic from programmable switches to a centralized controller is sufficient to build strong behavior models; FlowDiff does not require active measurements or expensive server instrumentation. Our experimental results using NEC data center testbed, Amazon EC2, and simulations demonstrate that FlowDiff is effective and robust in detecting anomalous behavior. FlowDiff scales well with the number of applications running in the data center and their traffic volume.

Software Defined Networking (SDN) has provided configurable access to remote network layer components from application hosts during application run-time. We have successfully enabled SDN switches to reduce network resource consumption and improve streaming latency in multi-party 3D tele-immersive (3DTI) applications. Instead of forwarding the same streams to multiple participants from the application host, local network switches replicate network packets towards multiple forwarding paths. However, due to the variable length of application frames generated from the 3DTI application, maintaining constant packet rates at the network switches cannot guarantee required application frame rates towards different forwarding paths. In this poster, we investigate the problem of guaranteeing application frame rates from the network layer switches. We formulate the problem and sketch a scalable solution using OpenFlow, which is a standard protocol developed for SDN.

Video conferencing applications pose fundamentally different service requirements than traditional data traffic on the Internet. Strong real-time interactivity is very important among participants unlike video streaming in VoD applications. Requirements are even more stringent in multi-stream and multi-site teleimmersive applications due to strong dependencies across geographically distributed streams. In this paper, we propose OpenSession, a cross-layer session-network control protocol for multi-stream multi-site 3D teleimmersion (3DTI) that improves interactivity, resource utilization and scalability. OpenSession decouples application layer data and control plane functionalities, and partially offloads the data plane functionalities to network layer switches. To control network layer switches during the session run-time, OpenSession leverages support from Software Defined Networking (e.g., OpenFlow). Through extensive evaluation with multi-stream 3D teleimmersion among four distributed sites and PlanetLab-based larger 3DTI setup, we show that OpenSession improves 3DTI interactivity and resource usage at the data plane. Furthermore, OpenSession keeps data plane robust against host failures and frequent route updates.

The usage and number of available video conferencing (VC) applications are rising as the high-bandwidth, low latency networks on which they depend become increasingly prevalent. Since VC applications support real-time human interaction, problems with performance that impair interactivity are social issues. Currently, performance measurements cannot easily be obtained due to the proprietary nature of VC applications, however, such measurements would be useful because they enable researchers to easily evaluate the performance impact of architectural and design decisions, quantitatively compare VC applications, and determine service level agreement (SLA) compliance. In this paper, we present a tool called Av Cloak that is capable of measuring several key performance metrics in proprietary VC applications: mouth-to-ear latency and jitter, capture-to-display latency and jitter, and audio-visual synchronization skew. AvCloak takes these measurements by wrapping ("cloaking") the VC application's audio/video inputs/outputs and transmitting timestamp data through them. At the sender side, AvCloak synthesizes media data encoding timestamps and feeds them to the VC application's media inputs, while at the receiver side, AvCloak decodes timestamps from the VC application's media outputs. Since AvCloak interacts with the target VC application only through its media inputs and outputs, it treats the target application as a black box and is thus applicable to arbitrary VC applications. We provide extensive analyses to measure AvCloak's overhead and show how to improve accuracy in measurements using two popular VC applications: Skype and Google+ Hangouts.

Being able to detect and recognize human activities is essential for 3D collaborative applications for efficient quality of service provisioning and device management. A broad range of research has been devoted to analyze media data to identify human activity, which requires the knowledge of data format, application-specific coding technique and computationally expensive image analysis. In this paper, we propose a human activity detection technique based on application generated metadata and related system metadata. Our approach does not depend on specific data format or coding technique. We evaluate our algorithm with different cyber-physical setups, and show that we can achieve very high accuracy (above 97%) by using a good learning model.

Different 3D tele-immersive (3DTI) activities pose different requirements for application and network level quality of service (QoS) to ensure a strong quality of experience (QoE) for participants. Some applications put heavy weight on audio quality, some consider higher quality for upper body video streams, and some seek very low end-to-end interactivity delay. In addition, a variation in streaming content may arise due to the participants' change of interests (e.g., view change). Therefore, there is a need for an adaptive multi-stream, multi-site 3DTI session management strategy, which is unobtrusive, and optimizes QoS parameters in the 3DTI content distribution based on the user activity and content variation. To address this next generation session management problem, we revisit the design space of multi-stream and multi-site 3DTI session layer. We propose an evolutionary 3DTI session optimization approach using an Open Session Management (OSM) architecture that uses a global view of participants and overlay network conditions to optimize prioritized QoS parameters. Experimental results with PlanetLab traces show that the optimization process is computationally unobtrusive, and the optimized TI sessions meet expectations of the participants up to 50% higher compared to the current solutions in the 3DTI space.

3D Tele-immersive systems create real-time multi-stream and multi-view 3D collaborative contents from multiple sites to allow interactive shared activities in virtual environments. Applications of 3DTI include online sports, tele-health, remote learning and collaborative arts. In addition to interactive participants in 3DTI environments, we envision a large number of passive non-interactive viewers that (a) watch the interactive activities in 3DTI shared environments, and (b) select views of the activities at run time. To achieve this vision, we present 4D Tele Cast, a novel multi-stream 3D content distribution framework for non-interactive viewers providing the functionality of multi-view selection. It addresses the following challenges: (1) supporting a large number of concurrent multi-stream viewers as well as multi-views, (2) preserving the unique nature of 3DTI multi-stream and multi-view dependencies at the viewers, and (3) allowing dynamic viewer behavior such as view changes and large-scale simultaneous viewer arrivals or departures. We divide the problem space into two: (1) multi-stream overlay construction problem that aims to minimize the cost of distribution of multi-stream contents, and maximize the number of concurrent viewers with sufficient viewer dynamism in terms of their resources and availabilities, and (2) effective resource utilization problem that aims to preserve the multi-stream dependencies in a view considering the heterogeneous resource constraints at the viewers. We evaluate 4D Tele Cast using extensive simulations with 3DTI activity data and Planet Lab traces.

ITU-T G.1070 [7] is widely cited for evaluating the subjective quality of the video conferencing, but its findings cannot be applied to the tele-immersion. The reasons are two fold. First, a tele-immersive system offers end users an unmatched realistic and immersive experience by allowing them to collaborate in the joint virtual space. Second, the human activities in the shared space are not limited to the conferencing conversation. In this paper, we conduct a user study with 19 participants to investigate the human perceptions of two tele-immersive shared activities, where media samples of different qualities are evaluated using the comparative category rating method [9] in case of each activity. We compare our subjective results to those presented in G.1070, and demonstrate heterogeneous human perceptual impacts in different activities.

This article presents a psychophysical study that measures the perceptual thresholds of a new factor called Color-plus-Depth Level-of-Details (CZLoD) peculiar to polygon-based 3D tele-immersive video. The results demonstrate the existence of Just Noticeable Degradation and Just Unacceptable Degradation thresholds on the factor. In light of the results, we design and implement a real-time perception-based quality adaptor for 3D tele-immersive video. Our experimental results show that the adaptation scheme can reduce resource usage (e.g., CPU cycles) while considerably enhancing the overall perceived visual quality. Our analysis confirms the potential temporal and spatial performance benefits achievable with CZLoD adaptation.

Quality control and resource optimization are challenging problems in 3D tele-immersive (3DTI) environments due to their large scale, multi-stream dependencies and dynamic peer (viewer) behavior. Such systems are also prone to performance degradation due to undesired behavior in the event of drastic demand changes, such as view change and large-scale simultaneous viewer arrivals or departures. Therefore, it is crucial to localize undesired behavior inside the system and re-organize the streaming overlay structures accordingly. Doing this accurately for a large scale is even more challenging and it requires to capture all events effecting the data plan and control plan of the system. Moreover, to do this, we need to understand the desired behavior of the application first, which is defined by the dependency patterns of performance and configuration metadata at each participating peers. To assist that, we propose a learning framework that discovers metadata dependency patterns from the time series metadata and uses an online profiler to detect undesired behavior of the system during run-time. Such universal protocol also enables the prediction of large scale performance degradation due to irregular dependencies. Finally an adaptation is proposed that reallocates the resources and rearranges overlay structures to overcome the undesired behavior. In summary, our goal is to provide a universal session monitoring and management framework for complex multi-stream 3DTI environments to support large number of concurrent viewers. We consider the difficulty in overlay construction, collecting metadata, answering queries, learning patterns, detecting undesired behavior at the participating peers and finally overlay adaptation considering multi-stream dependencies.

In this demonstration, we present two 3D tele-immersive games: light-saber dual and block fencing that merge 3D video representations of participants in real-time to enable remote interactions in a virtual world. The light-saber dual arranges participants in a symmetric setup where both participants interact with each other in a virtual world with similar goals. On the other hand, the block fencing creates an asymmetric setup where participants interact with virtual objects having different goals. Using these two setups, we address the challenges and novelty of our solutions in portable environment setup, data acquisition, multi-stream synchronization, multi-stream session management, mobile device rendering, and overlay communication in the design and implementation of advanced 3D tele-immersive systems.

This paper presents a psychophysical study that measures the perceptual thresholds of a new factor called Color-plus-Depth Level-of-Detail peculiar to polygon-based 3D tele-immersive video. The results demonstrate the existence of Just Noticeable Degradation and Just Unacceptable Degradation thresholds on the factor. In light of the results, we describe the design and implementation of a real-time perception-based quality adaptor for 3D tele-immersive video. Our experimental results show that the adaptation scheme can reduce resource usage while considerably enhancing the overall perceived visual quality.

This paper presents a psychophysical approach to control a new factor called Color-plus-Depth Level-of-Detail in polygon-based 3D tele-immersive video. Based on our psychophysical study that demonstrates the existence of perceptual thresholds on the factor, we present a real-time perception-based quality adaptor for 3D tele-immersive video. Our experimental results show that the adaptation scheme can reduce resource usage while considerably enhancing the overall perceived visual quality.

An ideal interactive 3D tele-immersion (3DTI) system is expected to disseminate and synchronize multi-streams with a shortest-possible latency among participating sites, achieve inter-stream synchronization, and bound both inter-sender and inter-receiver skews. This is, however, a key challenge because of (1) the coexistence of multi-modal, correlated, bandwidth-savvy streams from multiple source media, (2) the bounded bandwidth resources at each site, (3) the heterogeneous transmission end-to-end delays (EED) between sites and (4) the diversity of 3D views requested by multiple users. Our study of the existing content dissemination topologies reveals their inadequacy of handlings the complication and dynamics present in 3DTI systems.

This paper presents a novel paradigm of Streaming as a Service (SAS) to model correlated multi-streaming in Distributed Interactive Multimedia Environments. We propose SAS Kernel, a generic, distributed, and modular service kernel realizing SAS paradigm. SAS Kernel features high exibility by employing a configurable interface to allow for input of correlated multi-streams (bundle of streams) from diverse types of sensory devices. It is also highly extensible by allowing user-controlled functions to be applied to bundle of streams in runtime. Experiments with real-world applications demonstrate that the SAS Kernel incurs low overhead in delay, CPU, and bandwidth demands.

Cloud computing provides a revolutionary new computing paradigm for deploying enterprise applications and Internet services. Rather than operating their own data centers, today cloud users run their applications on the remote cloud infrastructures that are owned and managed by cloud providers. However, the cloud computing paradigm also introduces some new challenges in system management. Cloud users create virtual machine instances to run their specific application logic without knowing the underlying physical infrastructure. On the other side, cloud providers manage and operate their cloud infrastructures without knowing their customers' applications. Due to the decoupled ownership of applications and infrastructures, if a problem occurs, there is no visibility for either cloud users or providers to understand the whole context of the incident and solve it quickly. To this end, we propose a software solution, Cloud Insight, to provide some visibility through the middle virtualization layer for both cloud users and providers to address their problems quickly. Cloud Insight automatically tracks each VM instance's configuration status and maintains their life-cycle configuration records in a configuration management database (CMDB). When a user reports a problem, our algorithms automatically analyze CMDB to probabilistically determine the root cause and invoke a recovery process by interacting with the cloud user. Experimental results over data from Amazon EC2 online support forum and NEC Labs' research cloud infrastructures demonstrate that our approach can effectively automate the problem troubleshooting process in cloud environments.

With the advent of video sharing websites (e.g. YouTube) and interactive 3D video systems (e.g. Teeve, Cave), multimedia based content delivery networks (CDN) have become ubiquitous. These CDN architectures comprise of several processing and disseminating entities (e.g. proxies and gateways) that require real-time processing and trans-coding of multimedia content that must be delivered before an established deadline. Furthermore, Virtual Machine Monitors (VMMs) emerged as a popular architecture for server consolidation and cost reduction in enterprise systems. CDN proxies and gateways would greatly benefit from the deployment of VMM architectures since VMMs allow virtualization of their resources for virtual private multimedia sessions in a cost-effective manner. However, current VMMs do not provide abstractions to efficiently map processes to CPU resources and they do not provide adequate Quality of Service (QoS) mechanisms for task prioritization and CPU management to guarantee the completion of multimedia sessions by their deadlines.

This paper presents Eureka: a generic methodology of measuring the instantaneous (per second) bandwidth usage of networked games and applications in run time. Eureka starts with constructing a priority queue and sending low priority traffic through it. Then, the application under study is started with its packets marked as high priority. These packets therefore displace the previously existing low priority packets on the priority queue. We can measure this displacement and thus, the bandwidth used by the application. Unlike tcpdump this can be done while the application is running and without knowing any application specific data. In this paper we use Eureka to study the bandwidth usage of Massive Multiplayer Online Games (MMOGs), BZFlag and World of Warcraft-Trial, along with the 3D multi-site collaborative environments. A major strength of our methodology is that it provides run-time estimations of the bandwidth usage without interfering with the application, the environment, or the server/peer data. Moreover, Eureka is independent of the application and new infrastructure or tools are not required.

Distributed Interactive Multimedia Environments (DIMEs) enable geographically distributed people to interact with each other in a joint media-rich virtual environment for a wide range of activities, such as art performance, medical consultation, sport training, etc. The real-time collaboration is made possible by exchanging a set of multi-modal sensory streams over the network in real time. The characterization and evaluation of such multi-stream interactive environments is challenging because the traditional Quality of Service metrics (e.g., delay, jitter) are limited to a per stream basis. In this work, we present a novel cyber-physical, spatio-temporal QoS model that includes new metrics to measure and evaluate QoS for a bundle of streams and across bundles of streams. The new model allows better assessment of performance of any DIME application which employs multiple sensors. We validate our model by comparing the QoS performance of bundle of streams over three transport protocols - UDP, DCCP, and TCP in a 3D tele-immersive test bed.

Distributed Interactive Multimedia Environments (DIMEs) show important dependency constraints between application and underlying system components over time. For example, the video frame rate and the underlying bandwidth usage have a strong performance dependency. Performance dependencies must also be considered among distributed components. These dependencies over a time-span form correlation relationships. Violations of such correlation relationships represent collective anomalies. Users and most specifically DIME application developers face problems of finding (detecting), localizing such anomalies, and adapting against them in real-time. Current practices are to collect joint application-system metadata characterizing behaviors of application and system components while a DIME session is running, and then analyze them offline. Our goal is to provide a framework, called DIAMOND, that allows for real-time and unobtrusive collection and organization of joint application-system metadata in order to assist in finding such correlation violations in the system. DIAMOND works in four steps: (a) real-time metadata collection, (b) metadata processing to allow efficient computation of correlation constraints, (c) metadata distribution for efficient clustering of distributed metadata, and (d) anomaly detection, localization, and evolution monitoring based on violations of correlation relationships. Our results from real implementations and simulations with Planet Lab traces show the effectiveness of DIAMOND in terms of network overhead and anomaly detection time.

In this paper, we present the results from a quantitative and qualitative study of distributed gaming in 3D tele-immersive (3DTI) environments. We explore the Quality of Experience (QoE) of users in the new cyber-physical gaming environment. Guided by a theoretical QoE model, we conducted a case study and evaluated the impact of various Quality of Service (QoS) metrics (e.g., end-to-end delay, visual quality, etc.) on 3DTI gaming experience. We also identified a number of non-technical factors that are not captured by the original theoretical model, such as age, social interaction, and physical setup. Our analysis highlights new implications for the next-generation gaming system design, as well as a more comprehensive conceptual framework that captures non-technical influences for user experience in such environments.

Synchronization is a challenge in the multi-site 3D tele-immersion (3DTI) because it is complicated by the coexistence of highly correlated heterogenous streams from multiple sources, and the need for multi-stream resynchronization when user views change. To address the problems, we present TSync: a new multi-tier synchronization framework for 3DTI which can effectively reduce the multi-stream sync skews. Our contributions are focusing on (1) the use of timed synchronization points for multi-stream synchronization; (2) Internet bandwidth estimation based on machine learning; (3) the cooperative frame rate allocation for correlated multi-streams and (4) the resynchronization protocol used when user views change. Experimental results show that TSync can successfully achieve the synchronization of multi-source heterogeneous streams in 3DTI under Internet dynamics.

Users and administrators of large distributed systems are frequently in need of monitoring and management of its various components, data items and resources. Though there exist several distributed query and aggregation systems, the clustered structure of tele-immersive interactive frameworks and their time-sensitive nature and application requirements represent a new class of systems which poses different challenges on this distributed search. Multi-attribute composite range queries are one of the key features in this class. Queries are given in high level descriptions and then transformed into multi-attribute composite range queries. Designing such a query engine with minimum traffic overhead, low service latency, and with static and dynamic nature of large datasets, is a challenging task. In this paper, we propose a general multi-attribute based range query framework, Q-Tree, that provides efficient support for this class of systems. In order to serve efficient queries, Q-Tree builds a single topology-aware tree overlay by connecting the participating nodes in a bottom-up approach, and assigns range intervals on each node in a hierarchical manner. We show the relative strength of Q-Tree by analytically comparing it against P-Tree, P-Ring, Skip-Graph and Chord. With fine-grained load balancing and overlay maintenance, our simulations with PlanetLab traces show that our approach can answer complex queries within a fraction of a second.

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.

We present MobileTI, a portable tele-immersive system that merges 3D video representations of users in real time to enable remote collaboration across geographical distances. With portability as a main goal, we address the challenges in the camera setup, time synchronization, video acquisition, and networking in the design and implementation of the system. Having been deployed in public performances, MobileTI proves to be effective, efficient, and user-friendly. Our experimental findings in terms of technical performance and user feedback are presented.