ACM Special Interest Group on Multimedia (SIGMM) is pleased to present the 2017 SIGMM Outstanding Ph.D. Thesis Award to Dr. Chien-Nan (Shannon) Chen. The award committee considers Dr. Chien-Nan Chen's dissertation entitled "Semantics-Aware Content Delivery Framework For 3D Tele-Immersion" worthy of the recognition as the thesis is the first to consider semantic information in full-aspect with 3DTI systems. Dr. Chen's thesis includes confound semantic factors regarding user (role, preferences, view), activity (range, speed, posture, position, number of participants, repetitiveness), and environment (network capability, computing capability, power limitation) into the design and development of our 3DTI systems. Less than a year after its publication, the methodology advanced by this thesis has been adopted by three real-life multimedia applications. One of them is Facebook's 360 video streaming, which adopted user-objective awareness into its dynamic streaming for 360 content. The second application also comes from Facebook's 360 media. In the latest F8 conference (Facebook's annual developer conference) of 2017, they announced that the new dynamic streaming standard will encompass content-semantics awareness into its new view prediction model, which leads to more effective pre-buffering of 360 video content. The third application is a theatrics social experiment involving hundreds of Shakespeare enthusiasts in The Miracle Theatre, UK, which referenced the 3DTI amphitheater developed in this thesis which adopts semantic awareness on prioritized content dissemination. ....

Free-viewpoint television (FTV) is a visionary application that provides immersive experience to the audience with the freedom of changing viewpoint during the video playout. However, live broadcasting and user interaction do not coexist in existing FTV systems. In this paper, we propose F.Live, a framework of FTV content dissemination that supports user-initiated viewpoint changing for live broadcasting. Simulation result of a large-scale experiment, based on camera array settings of existing Nagoya systems and EyeVision System, shows that F.Live is capable of supporting 100,000 concurrent audiences with free-viewpoint low user interaction latency and feasible bandwidth requirements.

This paper presents Physical Representation Description (PRD), a method for representing the spatial relationship between sources of multimedia streams in physical environments. PRD is an extension for the most widely used video streaming standard DASH. DASH supports the delivery of multiple synchronized streams (e.g., video, audio and sensor data) from a server to multiple clients, however DASH does not provide information about how the multi-modal sensory sources were actually located relative to each other when the data was captured. PRD extends DASH streaming standard by describing how the sources of multi-modal sensory streams are located relative to each other in the physical environment. This enables DASH clients to understand how the scene was captured in a physical space, and use this information to (1) differentiate and prioritize sensory streams according to their physical location, (2) enhance viewing experience, and (3) assist in easier spatial mapping from physical to virtual cyber space and vice versa. In this paper we describe the need for PRD in an example application called CyPhy, representing a multi-modal sensory cyberphysical environment for tele-physiotherapy in patients' homes. We then describe PRD and show how it can improve DASH streaming standard in delivering enhanced viewing experience under different network, and viewing conditions using physical spatial information of sensory sources.

3D video is a new form of video featuring omni-view and immersive properties that are beneficial to many applications like telediagnosis, remote physiotherapy and e-learning. This paper presents OmniViewer, a multi-modal 3D streaming system based DASH, enhancing the versatility of the existing DASH standard and delivery infrastructure. Besides 3D video, OmniViewer supports multi-modal media in the DASH content for real-time media-synchronized interactive services. OmniViewer provides a complete multi-modal 3D video streaming solution from recording to rendering through DASH, which fills the gap between monoscopic 3D systems and 3D streaming systems.

This paper presents OmniViewer, a multi-modal 3D video streaming system based on Dynamic Adaptive Streaming over HTTP (DASH) standard. OmniViewer allows users to view arbitrary side of a performer by choosing the view angle from 0° to 360°. Besides, according to the current available bandwidth, it can also adaptively change the bitrate of rendered 3D video for both smooth and high-quality view rendering. Finally, OmniViewer extends traditional DASH implementation to support multi-modal data streaming besides video and audio.

3DTI Amphitheater is a live broadcasting system for dissemination of 3DTI (3D Tele-immersive) content. The virtual environment constructed by the system mimics an amphitheater in the real world, where performers interact with each other in the central circular stage, and the audience is placed in virtual seats that surround the stage. Users of the Amphitheater can be geographically dispersed and the streams created by the performer sites are disseminated in a P2P network among the participants. To deal with the high bandwidth demand and strict latency bound of the service, we identify the hierarchical priority of streams in construction of the content dissemination forest. Result shows that the Amphitheater outperforms prior 3DTI systems by boosting the application QoS by a factor of 2.8 while sustaining the same hundred-scale audience group.

This paper proposes FreeViewer, a 3D Tele-Immersion view-control system that allows viewers to see arbitrary side of the performer by intelligently choosing the streams of a subset of cameras and changing the point of view in a 3D virtual space. The view changing is actuated by the change of the sensor data from wearable devices (eg. Google Glass, smartphone) on the performer able to monitor the current orientation.

In this paper we verify the insufficiency of Stevens' power law to describe the relationship between QoS and QoE factors. User studies that target different types of application scenarios of 3D Tele-immersion (3DTI) are conducted and the results show no significant power trend in the relationship between packet loss and perceptual quality metrics. We further verify that activity characteristics, activity objectives, and users' roles in the 3DTI session also have profound effects on the service quality aside to the QoS level. Thus, simple one-factor psychophysical laws are inadequate of serving as a QoS-QoE mapping model.

In this paper we present the 3DTI Amphitheater, a live broadcasting system for dissemination of 3DTI (3D Tele-immersive) content. The virtual environment constructed by the system mimics an amphitheater in the real world, where performers interact with each other in the central circular stage, and the audience is placed in virtual seats that surround the stage. Users of the Amphitheater can be geographically dispersed and the streams created by the performer sites are disseminated in a P2P network among the participants. To deal with the high bandwidth demand and strict latency bound of the service, we identify the hierarchical priority of streams in construction of the content dissemination forest. Result shows that the Amphitheater outperforms prior 3DTI systems by boosting the application QoS by a factor of 2.8 while sustaining the same hundred-scale audience group.

The effective end-to-end transport of delay-sensitive voice data has long been a problem in multimedia networking. One of the major issues is determining the sending rate of real-time VoIP streams such that the user experience is maximized per unit network resource consumed. A particularly interesting complication that remains to be addressed is that the available bandwidth is often dynamic. Thus, it is unclear whether a marginal increase warrants better user experience. If a user naively tunes the sending rate to the optimum at any given opportunity, the user experience could fluctuate. To investigate the effects of magnitude and frequency of rate changes on user experience, we recruited 127 human participants to systematically score emulated Skype calls with different combinations of rate changes, including varying magnitude and frequency of rate changes. Results show that: 1) the rate change frequency affects the user experience on a logarithmic scale, echoing Weber-Fechner's Law; 2) the effect of rate change magnitude depends on how users perceive the quality difference; and 3) this study derives a closed-form model of user perception for rate changes for Skype calls.

In view of the high resource demand of 3D Tele-immersion (3DTI), we propose a user activity-based resource adaptation scheme that adjusts the compression ratio of 3DTI videos according to the user activity. The compression technique we designed is based on the frame synthesis via feature-based morphing. This technique is customized for 3DTI video due to the special properties of its scenes and the depth information provided by the system. Via a machine learning approach, our system can classify the user's current activity and choose the most suitable priority among temporal resolution, spatial resolution, and resource consumption in the adaptation. Our results show that the resource saving can reach up to 25% without perceptible degradation of the 3DTI videos.

In view of the resource demanding nature of 3D Tele-immersion (3DTI), we apply Morphing-based Frame Synthesis (MBFS) on delivery of both online and offline 3DTI visual content to decrease the resource consumption without degrading the perceptual quality. We further investigate the relationship between the level-of-motion of the content and the effectiveness of MBFS. In light of the results, we propose an on-the-fly resource adaptor for 3DTI video transmission which utilizes a perceptual model built from data compiled by a series of subjective experiments. Results show that our adaptor achieves 43% to 87% compression ratio for offline compression on 3DTI videos of different baseline user activities, and a 10% on-the-fly bandwidth saving on complex user activity without perceptible degradation.

In view of the different demands on quality of service of different user activities in the 3D Tele-immersive (3DTI) environment, we combine activity recognition and real-time morphing-based compression and present the Activity-Aware Adaptive Compression. We implement this scheme on our 3DTI platform: the TEEVE Endpoint, which is a runtime engine to handle the creation, transmission and rendering of 3DTI data. User study as well as objective evaluation of the scheme show that it can achieve 25% more bandwidth saving compared to conventional 3D data compression as zlib without perceptible degradation in the user experience.

Being able to detect and recognize human activities is essential for 3D collaborative applications for efficient quality of service provisioning and device management. A broad range of research has been devoted to analyze media data to identify human activity, which requires the knowledge of data format, application-specific coding technique and computationally expensive image analysis. In this paper, we propose a human activity detection technique based on application generated metadata and related system metadata. Our approach does not depend on specific data format or coding technique. We evaluate our algorithm with different cyber-physical setups, and show that we can achieve very high accuracy (above 97%) by using a good learning model.

As Skype has become popular and a profitable business, the long-standing problem of how to deliver Skype calls deserves a serious revisit from an economic viewpoint. This study proposes a rate control mechanism for Skype calls that satisfies more users and satisfies users more than the greedy-naïve mechanism, as well as the mechanism implemented in Skype.

The effective end-to-end transport of delay-sensitive voice data has long been a problem in multimedia networking. One of the major issues is determining the sending rate of real-time VoIP streams such that the user experience is maximized per unit network resource consumed. A particularly interesting complication that remains to be addressed is that the available bandwidth is often dynamic. Thus, it is unclear whether a marginal increase warrants better user experience. If a user naively tunes the sending rate to the optimum at any given opportunity, the user experience could fluctuate.