Technological developments in comprehensive video understanding - detecting and identifying visual elements of a scene, combined with audio understanding (music, speech), as well as aligned with textual information such as captions, subtitles, etc. and background knowledge - have been undergoing a significant revolution during recent years. The workshop brings together experts from academia and industry in order to discuss the latest progress in artificial intelligence research in topics related to multimodal information analysis, and in particular, semantic analysis of video, audio, and textual information for smart digital TV content production, access and delivery.

Medical devices are essential for the practice of modern medicine, and the standard open-source integrated clinical environment (OpenICE) has been well designed and widely adopted to improve their interoperability. With OpenICE, it is easy to connect individual devices into the integrated clinical system to provide a coherent patient care. In this paper, we present ICERV, the first online verification approach for the OpenICE, to ensure the dependability (mainly for the safety and security) of the integrated system and the involved patient and clinician. The key idea is to customize runtime verification technique to provide a transparent verifying infrastructure to continually intercept the communication commands and messages of those devices, based on which, we can formalize the safety and security requirements as past time linear temporal logic expressions for verifier generation and online formal verification. If any requirements violate, predefined warnings or exception handling actions will be triggered timely to prevent hazards and threats. We have implemented and seamlessly integrated the approach without any changes to the source code of OpenICE nor the code of the upper-level applications or supervision, and the real device is used for evaluation to demonstrate the effectiveness.

Virtual Reality (VR) devices are becoming accessible to a large public, which is going to increase the demand for 360° VR videos. VR videos are often characterized by a poor quality of experience, due to the high bandwidth required to stream the 360° video. To overcome this issue, we spatially divide the VR video into tiles, so that each temporal segment is composed of several spatial tiles. Only the tiles belonging to the viewport, the region of the video watched by the user, are streamed at the highest quality. The other tiles are instead streamed at a lower quality. We also propose an algorithm to predict the future viewport position and minimize quality transitions during viewport changes. The video is delivered using the server push feature of the HTTP/2 protocol. Instead of retrieving each tile individually, the client issues a single push request to the server, so that all the required tiles are automatically pushed back to back. This approach allows to increase the achieved throughput, especially in mobile, high RTT networks. In this paper, we detail the proposed framework and present a prototype developed to test its performance using real-world 4G bandwidth traces. Particularly, our approach can save bandwidth up to 35% without severely impacting the quality viewed by the user, when compared to a traditional non-tiled VR streaming solution. Moreover, in high RTT conditions, our HTTP/2 approach can reach 3 times the throughput of tiled streaming over HTTP/1.1, and consistently reduce freeze time. These results represent a major improvement for the efficient delivery of 360° VR videos over the Internet.

Use of telecommunication technologies for remote, continuous monitoring of patients can enhance effectiveness of emergency ambulance care during transport from rural areas to a regional center hospital. However, the communication along the various routes in rural areas may have wide bandwidth ranges from 2G to 4G; some regions may have only lower satellite bandwidth available. Bandwidth fluctuation together with real-time communication of various clinical multimedia pose a major challenge during rural patient ambulance transport.; [email protected] availability of a pre-transport route-dependent communication bandwidth database is an important resource in remote monitoring and clinical multimedia transmission in rural ambulance transport. Here, we present a geo-communication dataset from extensive profiling of 4 major US mobile carriers in Illinois, from the rural location of Hoopeston to the central referral hospital center at Urbana. In collaboration with Carle Foundation Hospital, we developed a profiler, and collected various geographical and communication traces for realistic emergency rural ambulance transport scenarios. Our dataset is to support our ongoing work of proposing "physiology-aware DASH", which is particularly useful for adaptive remote monitoring of critically ill patients in emergency rural ambulance transport. It provides insights on ensuring higher Quality of Service (QoS) for most critical clinical multimedia in response to changes in patients' physiological states and bandwidth conditions. Our dataset is available online1 for research community.

The demand for 360° Virtual Reality (VR) videos is expected to grow in the near future, thanks to the diffusion of VR headsets. VR Streaming is however challenged by the high bandwidth requirements of 360° videos. To save bandwidth, we spatially tile the video using the H.265 standard and stream only tiles in view at the highest quality. The video is also temporally segmented, so that each temporal segment is composed of several spatial tiles. In order to minimize quality transitions when the user moves, an algorithm is developed to predict where the user is likely going to watch in the near future. Consequently, predicted tiles are also streamed at the highest quality. Finally, the server push in HTTP/2 is used to deliver the tiled video. Only one request is sent from the client; all the tiles of a segment are automatically pushed from the server. This approach results in a better bandwidth utilization and video quality compared to traditional streaming over HTTP/1.1, where each tile has to be requested independently by the client. We showcase the benefits of our framework using a prototype developed on a Samsung Galaxy S7 and a Gear VR, which supports both tiled and non-tiled videos and streaming over HTTP/1.1 and HTTP/2. Under limited bandwidth conditions, we demonstrate how our framework can improve the quality watched by the user compared to a non-tiled solution where all of the video is streamed at the same quality. This result represents a major improvement for the efficient streaming of VR videos.

We have proposed an adaptive view-aware bandwidth-efficient 360 VR video streaming framework based on the tiling features of MPEG-DASH SRD. We extend MPEG-DASH SRD to the 3D space of 360 VR videos, and showcase a dynamic view-aware adaptation technique to tackle the high bandwidth demands of streaming 360 VR videos to wireless VR headsets. As a part of our contributions, we spatially partition the underlying 3D mesh into multiple 3D sub-meshes, and construct an efficient 3D geometry mesh called hexaface sphere to optimally represent tiled 360 VR videos in the 3D space. We then spatially divide the 360 videos into multiple tiles while encoding and packaging, use MPEG-DASH SRD to describe the spatial relationship of tiles in the 3D space, and prioritize the tiles in the Field of View (FoV) for view-aware adaptation. The initial evaluations that we conducted show that we can save up to 72% of the required bandwidth on 360 VR video streaming with minor negative quality impacts compared to the baseline scenario when no adaptations is applied.

The ultimate objective of medical cyber-physical systems is to enhance the safety and effectiveness of patient care. To ensure safe and effective care during emergency patient transfer from rural areas to center tertiary hospitals, reliable and real-time communication is essential. Unfortunately, real-time monitoring of patients involves transmission of various clinical multimedia data including videos, medical images, and vital signs, which requires use of mobile network with high-fidelity communication bandwidth. However, the wireless networks along the roads in rural areas range from 4G to 2G to low speed satellite links, which poses a significant challenge to transmit critical patient information. In this paper, we present a bandwidth-compliant criticality-aware system for transmission of massive clinical multimedia data adaptive to varying bandwidths during patient transport. Model-based clinical automata are used to determine the criticality of clinical multimedia data. We borrow concepts from DASH, and propose physiology-aware adaptation techniques to transmit more critical clinical data with higher fidelity in response to changes in disease, clinical states, and bandwidth condition. In collaboration with Carle's ambulance service center, we develop a bandwidth profiler, and use it as proof of concept to support our experiments. Our preliminary evaluation results show that our solutions ensure that most critical patient's clinical data are communicated with higher fidelity.

Today, sepsis syndrome is one of the leading cause of death globally, and is of great clinical importance. In this paper, we present a self-adaptively evolutionary sepsis screening system to shorten the time of syndrome detection and improve the positive effect of treatment, with the screening frequency and content can be automatically adjusted according to the current status of the patient. First, we propose a novel graphical computation model named AdapDBN with a clearly defined syntax for the medical knowledge presentation, especially for the presentation of the pathophysiology model of the disease. Then, the semantics of AdapDBN is formally defined for the evolutionary inference of syndrome onset probability. Finally, we demonstrate how to initialize AdapDBN with sepsis-related epidemiologic statics, published clinical research and physician's knowledge and how to incorporate it into existing sepsis screening and decision support flow. We evaluate its effectiveness and superiority with comparisons to existing computation techniques.

While traditional multimedia applications such as games and videos are still popular, there has been a significant interest in the recent years towards new 3D media such as 3D immersion and Virtual Reality (VR) applications, especially 360 VR videos. 360 VR video is an immersive spherical video where the user can look around during playback. Unfortunately, 360 VR videos are extremely bandwidth intensive, and therefore are difficult to stream at acceptable quality levels. In this paper, we propose an adaptive bandwidth-efficient 360 VR video streaming system using a divide and conquer approach. We propose a dynamic view-aware adaptation technique to tackle the huge bandwidth demands of 360 VR video streaming. We spatially divide the videos into multiple tiles while encoding and packaging, use MPEG-DASH SRD to describe the spatial relationship of tiles in the 360-degree space, and prioritize the tiles in the Field of View (FoV). In order to describe such tiled representations, we extend MPEG-DASH SRD to the 3D space of 360 VR videos. We spatially partition the underlying 3D mesh, and construct an efficient 3D geometry mesh called hexaface sphere to optimally represent a tiled 360 VR video in the 3D space. Our initial evaluation results report up to 72% bandwidth savings on 360 VR video streaming with minor negative quality impacts compared to the baseline scenario when no adaptations is applied.

We demonstrate an adaptive bandwidth-efficient 360 VR video streaming system based on MPEG-DASH SRD. We extend MPEG-DASH SRD to the 3D space of 360 VR videos, and showcase a dynamic view-aware adaptation technique to tackle the high bandwidth demands of streaming 360 VR videos to wireless VR headsets. We spatially partition the underlying 3D mesh into multiple 3D sub-meshes, and construct an efficient 3D geometry mesh called hexaface sphere to optimally represent tiled 360 VR videos in the 3D space. We then spatially divide the 360 videos into multiple tiles while encoding and packaging, use MPEG-DASH SRD to describe the spatial relationship of tiles in the 3D space, and prioritize the tiles in the Field of View (FoV) for view-aware adaptation. Our initial evaluation results show that we can save up to 72% of the required bandwidth on 360 VR video streaming with minor negative quality impacts compared to the baseline scenario when no adaptations is applied.

The era of big data has led to the emergence of new systems for real-time distributed stream processing, e.g., Apache Storm is one of the most popular stream processing systems in industry today. However, Storm, like many other stream processing systems lacks an intelligent scheduling mechanism. The default round-robin scheduling currently deployed in Storm disregards resource demands and availability, and can therefore be inefficient at times. We present R-Storm (Resource-Aware Storm), a system that implements resource-aware scheduling within Storm. R-Storm is designed to increase overall throughput by maximizing resource utilization while minimizing network latency. When scheduling tasks, R-Storm can satisfy both soft and hard resource constraints as well as minimizing network distance between components that communicate with each other. We evaluate R-Storm on set of micro-benchmark Storm applications as well as Storm applications used in production at Yahoo! Inc. From our experimental results we conclude that R-Storm achieves 30-47% higher throughput and 69-350% better CPU utilization than default Storm for the micro-benchmarks. For the Yahoo! Storm applications, R-Storm outperforms default Storm by around 50% based on overall throughput. We also demonstrate that R-Storm performs much better when scheduling multiple Storm applications than default Storm.

Advances in computing hardware and novel multimedia applications have urged the development of handheld mobile devices such as smartphones and PDAs. Amongst the most used applications on handheld devices are mobile 3D graphics such as 3D games and 3D virtual environments. With this significant increase of mobile applications and games, one of the challenges is how to efficiently transmit the bulky 3D information to resource-constrained mobile devices. Despite the many attractive features, 3D graphics impose significant demands on the limited battery capacity of mobile devices. Thus the development of efficient approaches to decrease the amount of streamed data with the aim of increasing the battery lifetime has become a key research topic.

Smartphone devices are becoming the de facto personal computing platform, rivaling the desktop, as the number of smartphone users is projected to reach 1.1 billion by 2013. Unlike the desktop, smartphones have a constrained energy budget, which is further challenged by increasingly sophisticated applications. Amongst the most popular applications on smartphone devices are games and virtual environments that rely on 3D graphics. Due to the computational intensity of geometry and rasterization, as well as the perpetually illuminated display, these applications are extremely power-hungry. To prolong the battery life of devices running these applications, we propose two new energy-aware adaptation schemes that can be employed in 3D graphics applications: lighting limitation and textural transformation. Our results show that we can conserve between 20% and 33% of energy with acceptable sacrifices to a user's visual experience.

With the growing demand of mobile applications and games, one of the challenges is how to efficiently transmit the bulky 3D information to bandwidth- and computationally-limited mobile devices. In this paper, we propose two methods for improving the transmission delay of M3G-based 3D mobile game content over unreliable and congested networks. We introduce Object Mesh Similarity as a server side approach, in which we try to detect an alternative object with minimum complexities that is similar to the original object, and then transmit this reconstructed object, as well as Texture Stretching as a client-side approach, which leads to the efficient receipt of object textures. Our results show 35% to 70% improvement in the transmission delay of 3D textures.