Different 3D tele-immersive (3DTI) activities pose different prioritized requirements for application and network-level quality of service (QoS) to ensure a strong quality of experience (QoE) for participants. Some applications put heavy weight on audio quality, some consider higher quality for upper body video streams, and some seek very low end-to-end interactivity delay. In addition, a variation in streaming content may arise in the 3DTI space due to the participants' change in interests (e.g., view change). Therefore, there is a need for an adaptive multistream, multisite 3DTI session management strategy that is not only unobtrusive, but also optimizes prioritized QoS parameters in 3DTI content distribution based on user activity and content variation. To address this next generation session management problem, we revisit the design space of multistream and multisite 3DTI session layer. We present an evolutionary 3DTI session optimization approach using Open Session Management (OSM) architecture that uses a global view of participants and overlays network conditions to optimize QoS parameters. Experimental results with PlanetLab traces show that our optimization process is unobtrusive, and the optimized TI sessions provide higher satisfaction to the participants (in some cases up to 50% higher) compared to the current solutions in the 3DTI space.

3D Teleimmersive (TI) Networks enable collaborative interactive activities in the same virtual space. The geographically distributed multi-site nature of TIs, the heterogeneity and multi-modality of their I/O devices and the diversity of activities in these environments pose new challenges in designing networked I/O device management for TIs not addressed by existing systems. In this paper we describe Decima, a holistic virtualized I/O Management. Decima enables dynamic seamless and universal access and management to asymmetric Teleimmersive activities using a large scale of heterogeneous TI devices. Our experiments show that Decima is lightweight in terms of CPU overhead and end-to-end delay and scales well as the number of devices in the TI network increases.

Video conferencing applications pose fundamentally different service requirements than traditional data traffic on the Internet. Strong real-time interactivity is very important among participants unlike video streaming in VoD applications. Requirements are even more stringent in multi-stream and multi-site teleimmersive applications due to strong dependencies across geographically distributed streams. In this paper, we propose OpenSession, a cross-layer session-network control protocol for multi-stream multi-site 3D teleimmersion (3DTI) that improves interactivity, resource utilization and scalability. OpenSession decouples application layer data and control plane functionalities, and partially offloads the data plane functionalities to network layer switches. To control network layer switches during the session run-time, OpenSession leverages support from Software Defined Networking (e.g., OpenFlow). Through extensive evaluation with multi-stream 3D teleimmersion among four distributed sites and PlanetLab-based larger 3DTI setup, we show that OpenSession improves 3DTI interactivity and resource usage at the data plane. Furthermore, OpenSession keeps data plane robust against host failures and frequent route updates.

Being able to detect and recognize human activities is essential for 3D collaborative applications for efficient quality of service provisioning and device management. A broad range of research has been devoted to analyze media data to identify human activity, which requires the knowledge of data format, application-specific coding technique and computationally expensive image analysis. In this paper, we propose a human activity detection technique based on application generated metadata and related system metadata. Our approach does not depend on specific data format or coding technique. We evaluate our algorithm with different cyber-physical setups, and show that we can achieve very high accuracy (above 97%) by using a good learning model.

Different 3D tele-immersive (3DTI) activities pose different requirements for application and network level quality of service (QoS) to ensure a strong quality of experience (QoE) for participants. Some applications put heavy weight on audio quality, some consider higher quality for upper body video streams, and some seek very low end-to-end interactivity delay. In addition, a variation in streaming content may arise due to the participants' change of interests (e.g., view change). Therefore, there is a need for an adaptive multi-stream, multi-site 3DTI session management strategy, which is unobtrusive, and optimizes QoS parameters in the 3DTI content distribution based on the user activity and content variation. To address this next generation session management problem, we revisit the design space of multi-stream and multi-site 3DTI session layer. We propose an evolutionary 3DTI session optimization approach using an Open Session Management (OSM) architecture that uses a global view of participants and overlay network conditions to optimize prioritized QoS parameters. Experimental results with PlanetLab traces show that the optimization process is computationally unobtrusive, and the optimized TI sessions meet expectations of the participants up to 50% higher compared to the current solutions in the 3DTI space.

In this demonstration, we present two 3D tele-immersive games: light-saber dual and block fencing that merge 3D video representations of participants in real-time to enable remote interactions in a virtual world. The light-saber dual arranges participants in a symmetric setup where both participants interact with each other in a virtual world with similar goals. On the other hand, the block fencing creates an asymmetric setup where participants interact with virtual objects having different goals. Using these two setups, we address the challenges and novelty of our solutions in portable environment setup, data acquisition, multi-stream synchronization, multi-stream session management, mobile device rendering, and overlay communication in the design and implementation of advanced 3D tele-immersive systems.

No abstract available.

An ideal interactive 3D tele-immersion (3DTI) system is expected to disseminate and synchronize multi-streams with a shortest-possible latency among participating sites, achieve inter-stream synchronization, and bound both inter-sender and inter-receiver skews. This is, however, a key challenge because of (1) the coexistence of multi-modal, correlated, bandwidth-savvy streams from multiple source media, (2) the bounded bandwidth resources at each site, (3) the heterogeneous transmission end-to-end delays (EED) between sites and (4) the diversity of 3D views requested by multiple users. Our study of the existing content dissemination topologies reveals their inadequacy of handlings the complication and dynamics present in 3DTI systems.

This paper presents a novel paradigm of Streaming as a Service (SAS) to model correlated multi-streaming in Distributed Interactive Multimedia Environments. We propose SAS Kernel, a generic, distributed, and modular service kernel realizing SAS paradigm. SAS Kernel features high exibility by employing a configurable interface to allow for input of correlated multi-streams (bundle of streams) from diverse types of sensory devices. It is also highly extensible by allowing user-controlled functions to be applied to bundle of streams in runtime. Experiments with real-world applications demonstrate that the SAS Kernel incurs low overhead in delay, CPU, and bandwidth demands.

With the advent of video sharing websites (e.g. YouTube) and interactive 3D video systems (e.g. Teeve, Cave), multimedia based content delivery networks (CDN) have become ubiquitous. These CDN architectures comprise of several processing and disseminating entities (e.g. proxies and gateways) that require real-time processing and trans-coding of multimedia content that must be delivered before an established deadline. Furthermore, Virtual Machine Monitors (VMMs) emerged as a popular architecture for server consolidation and cost reduction in enterprise systems. CDN proxies and gateways would greatly benefit from the deployment of VMM architectures since VMMs allow virtualization of their resources for virtual private multimedia sessions in a cost-effective manner. However, current VMMs do not provide abstractions to efficiently map processes to CPU resources and they do not provide adequate Quality of Service (QoS) mechanisms for task prioritization and CPU management to guarantee the completion of multimedia sessions by their deadlines.

This paper presents Eureka: a generic methodology of measuring the instantaneous (per second) bandwidth usage of networked games and applications in run time. Eureka starts with constructing a priority queue and sending low priority traffic through it. Then, the application under study is started with its packets marked as high priority. These packets therefore displace the previously existing low priority packets on the priority queue. We can measure this displacement and thus, the bandwidth used by the application. Unlike tcpdump this can be done while the application is running and without knowing any application specific data. In this paper we use Eureka to study the bandwidth usage of Massive Multiplayer Online Games (MMOGs), BZFlag and World of Warcraft-Trial, along with the 3D multi-site collaborative environments. A major strength of our methodology is that it provides run-time estimations of the bandwidth usage without interfering with the application, the environment, or the server/peer data. Moreover, Eureka is independent of the application and new infrastructure or tools are not required.

Distributed Interactive Multimedia Environments (DIMEs) enable geographically distributed people to interact with each other in a joint media-rich virtual environment for a wide range of activities, such as art performance, medical consultation, sport training, etc. The real-time collaboration is made possible by exchanging a set of multi-modal sensory streams over the network in real time. The characterization and evaluation of such multi-stream interactive environments is challenging because the traditional Quality of Service metrics (e.g., delay, jitter) are limited to a per stream basis. In this work, we present a novel cyber-physical, spatio-temporal QoS model that includes new metrics to measure and evaluate QoS for a bundle of streams and across bundles of streams. The new model allows better assessment of performance of any DIME application which employs multiple sensors. We validate our model by comparing the QoS performance of bundle of streams over three transport protocols - UDP, DCCP, and TCP in a 3D tele-immersive test bed.

Distributed Interactive Multimedia Environments (DIMEs) show important dependency constraints between application and underlying system components over time. For example, the video frame rate and the underlying bandwidth usage have a strong performance dependency. Performance dependencies must also be considered among distributed components. These dependencies over a time-span form correlation relationships. Violations of such correlation relationships represent collective anomalies. Users and most specifically DIME application developers face problems of finding (detecting), localizing such anomalies, and adapting against them in real-time. Current practices are to collect joint application-system metadata characterizing behaviors of application and system components while a DIME session is running, and then analyze them offline. Our goal is to provide a framework, called DIAMOND, that allows for real-time and unobtrusive collection and organization of joint application-system metadata in order to assist in finding such correlation violations in the system. DIAMOND works in four steps: (a) real-time metadata collection, (b) metadata processing to allow efficient computation of correlation constraints, (c) metadata distribution for efficient clustering of distributed metadata, and (d) anomaly detection, localization, and evolution monitoring based on violations of correlation relationships. Our results from real implementations and simulations with Planet Lab traces show the effectiveness of DIAMOND in terms of network overhead and anomaly detection time.

In this paper, we present the results from a quantitative and qualitative study of distributed gaming in 3D tele-immersive (3DTI) environments. We explore the Quality of Experience (QoE) of users in the new cyber-physical gaming environment. Guided by a theoretical QoE model, we conducted a case study and evaluated the impact of various Quality of Service (QoS) metrics (e.g., end-to-end delay, visual quality, etc.) on 3DTI gaming experience. We also identified a number of non-technical factors that are not captured by the original theoretical model, such as age, social interaction, and physical setup. Our analysis highlights new implications for the next-generation gaming system design, as well as a more comprehensive conceptual framework that captures non-technical influences for user experience in such environments.

Synchronization is a challenge in the multi-site 3D tele-immersion (3DTI) because it is complicated by the coexistence of highly correlated heterogenous streams from multiple sources, and the need for multi-stream resynchronization when user views change. To address the problems, we present TSync: a new multi-tier synchronization framework for 3DTI which can effectively reduce the multi-stream sync skews. Our contributions are focusing on (1) the use of timed synchronization points for multi-stream synchronization; (2) Internet bandwidth estimation based on machine learning; (3) the cooperative frame rate allocation for correlated multi-streams and (4) the resynchronization protocol used when user views change. Experimental results show that TSync can successfully achieve the synchronization of multi-source heterogeneous streams in 3DTI under Internet dynamics.

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.

We present MobileTI, a portable tele-immersive system that merges 3D video representations of users in real time to enable remote collaboration across geographical distances. With portability as a main goal, we address the challenges in the camera setup, time synchronization, video acquisition, and networking in the design and implementation of the system. Having been deployed in public performances, MobileTI proves to be effective, efficient, and user-friendly. Our experimental findings in terms of technical performance and user feedback are presented.

3D Teleimmersive (TI) environments have gained significant attention as they show promising results for a wide number of applications. These environments are usually distributed and multi platform which makes performance analysis and monitoring difficult. We present a Unified Time Service (UTS) and its measurement framework for evaluating components in terms of their reflected QoS (e.g. delay). Then we utilize UTS to analyze the delay introduced by the virtualized rendezvous points (RP) in the H-MEDIA TI system. We find that the delay overhead heavily depends on frame sizes of the 3D video streams.

The Tele-immersive Dance Environment (TED) is a geographically distributed, real-time 3-D virtual room where multiple participants interact independent of physical distance. TED, a highly interactive collaborative environment, offers digital options with multiple viewpoints, enhancing the creative movement composition involved with dance choreography. A symbiotic relationship for creativity and design exists between dance artists and computer scientists as the tele-immersive environment is analyzed as a creativity and learning tool. We introduce the advancements of the interactive digital options, new interface developments, user study results, and the possibility of a computational model for human creativity through Laban Movement Analysis.

We demonstrate the Tele-immersive Dance Environment (TED), a geographically distributed, real-time 3-D virtual room where multiple participants interact independent of physical distance. TED, a highly interactive collaborative environment, offers digital options with multiple viewpoints, enhancing the creative movement composition involved with dance choreography. We present the advancements of the interactive digital options, new interface developments, and user study results.