Multi-party Mobile Virtual Reality (MMVR) enables multiple mobile users to share virtual scenes for an immersive multimedia experience in scenarios such as gaming, social interaction, and industrial mission collaboration. Dynamic 3D Point Cloud (DPCL) is an emerging representation form of MMVR that can be consumed as a free-viewpoint video with 6 degrees of freedom. With limited on-device resources, it is a challenge to achieve a satisfying frame rate for DPCL rendering, which makes edge-assisted rendering a practical solution. However, repeated loading of DPCL scenes with a substantial amount of metadata introduces a significant redundancy overhead that cannot be overlooked when enabling multiple edge servers to support the rendering requirements of user groups. In this paper, we design PoClVR, an edge-assisted DPCL rendering system for MMVR applications, which introduces an object-level splitting mode to alleviate performance bottlenecks caused by redundant loading. In addition, PoClVR dynamically selects the splitting mode and scheduling decisions to adapt to varying task requirements and available computational resources, thereby improving overall system efficiency. To evaluate the performance of PoClVR, we implement and deploy a realistic prototype system and also conduct large-scale trace-driven simulations. The experimental results show that PoClVR can reduce resource usage by up to approximately 49.3% under different task requirements and resource conditions, while decreasing bottleneck performance degradation by up to 77.3%.

Multi-party Mobile Virtual Reality (MMVR) enables multiple mobile users to share virtual scenes for immersive multimedia experience in scenarios such as gaming, social interaction, and industrial mission collaboration. Dynamic 3D Point Cloud (DPCL) is an emerging representation form of MMVR that can be consumed as a free-viewpoint video with 6 degrees of freedom. Given that it is challenging to render DPCL at a satisfying frame rate with limited on-device resources, offloading rendering tasks to edge servers is recognized as a practical solution. However, repeated loading of DPCL scenes with a substantial amount of metadata introduces a significant redundancy overhead that cannot be overlooked when enabling multiple edge servers to support the rendering requirements of user groups. In this paper, we design PoClVR, an edge-assisted DPCL rendering system for MMVR applications, which breaks down the rendering process of the complete dynamic scene into multiple rendering tasks of dynamic objects. PoClVR significantly reduces the repetitive loading overhead of DPCL scenes on edge servers and periodically adjusts the rendering task allocation during the application running to accommodate rendering requirements. We deploy PoClVR based on a real-world implementation and the experimental evaluation results show that PoClVR can reduce GPU utilization by up to 15.1% and increase rendering frame rate by up to 34.6% compared to other baselines while ensuring that the image quality viewed by the user is virtually unchanged.

With the popularity of mobile devices and the continuous advancement of mobile network technology, running online augmented reality (AR) on lightweight mobile devices is much more desirable than on heavy and expensive head-mounted devices that are difficult to satisfy users. Mobile edge computing can assist in supporting AR applications running on mobile devices, which copes with compute-intensive and delay-sensitive requirements. However, subject to the limited and heterogeneous edge resources, offloading tasks to edge devices is not easy, especially if the application requires multi-party interaction. It is challenging to develop a credible task placement scheme that satisfies user experience with flexible use of edge resources. This article focus on the task offloading placement problem for AR overlay rendering in multi-party mobile augmented reality system. We first present our observations about performance bottlenecks of edge devices and explain the necessity of splitting the AR overlay rendering pipeline. We then formulate a joint optimization problem of task placement decisions, aiming to maximize the user experience of quality and minimize the service cost. We develop a novel decision approach based on deep reinforcement learning (DRL) to address this complex problem. Finally, we verify the effectiveness and superiority of the proposed method through extensive evaluation experiments.

Video streaming plays a critical role in the video analytics pipeline and thus its adaptation scheme has been a focus of optimization. As machine learning algorithms have become main consumers of video contents, the streaming adaptation decision should be made to optimize their inference performance. Existing video streaming adaptation schemes for video analytics are usually designed to adapt to bandwidth and content variations separately, which fail to consider the coordination between transmission and computation. Given the nature of batch transmission in video streaming and batch processing in deep learning-based inference, we observe that the choices of the batch sizes directly affects the bandwidth efficiency, the response delay and the accuracy of the deep learning inference in video analytics. In this work, we investigate the effect of the batch size in transmission and processing, formulate the optimal batch size adaptation problem, and further develop the deep reinforcement learning-based solution. Practical issues are further addressed for Implementation. Extensive simulations are conducted for performance evaluation, whose results demonstrate the superiority of our proposed batch adaptive streaming approach over the baseline streaming approaches.

Nowadays, although the rapidly growing demand for short video sharing has brought enormous commercial value, considerable bandwidth usage becomes a problem for service providers. To save costs of service providers, the short video applications face a critical conflict between maximizing the user quality of experience (QoE) and minimizing the bandwidth usage. Most of existing bitrate adaptation methods are designed for the livecast and video-on-demand instead of short video applications. In this paper, we propose a QoE-aware adaptive download control mechanism to ensure the user QoE and save the bandwidth, which can download the appropriate video according to user retention probabilities and network conditions, and pause the download when the buffered data is enough. The extensive simulation results demonstrate the superiority of our proposed mechanism over the other baseline methods.

Tile-based approach is widely adopted in adaptive 360\textdegree~video streaming systems. Existing QoE-driven streaming approaches usually obtain the tile selection and adjust the bitrate based on the viewport prediction with a fixed tiling, which fail to consider the unstable prediction performance. However, varying the tiling of the video can produce different number of tiles with different sizes, and thus can have distinct impacts on error tolerance for viewport prediction and on decoding complexity for resource-constrained mobile client. In this work, we introduce adaptive tiling into the conventional bitrate adaptation for mobile 360degree~video streaming. We first analyze the impacts of tilings on tile selection and decoding time, which verify the benefit of tiling adaptation in various practical aspects. We then formulate the QoE optimization problem for adaptive tiling and bitrate streaming and discuss the design details of our adaptation algorithm, which can adapt to the performance of viewport prediction and the decoding capabilities of mobile clients in addition to the conventional influencing factors. Finally, the superiority of our proposed approach compared with the state-of-the-art methods is evaluated through extensive trace-driven simulations.

Today’s multimedia applications usually organize the contents into data blocks with different deadlines and priorities. Meeting/missing the deadline for different data blocks may contribute/hurt the user experience to different degrees. With the goal of optimizing real-time multimedia communications, the transmission control scheme needs to make two challenging decisions: the proper sending rate and the best data block to send under dynamic network conditions. In this paper, we propose a delay-sensitive and priority-aware transmission control scheme with two modules, namely, rate control and block selection. The rate control module constantly monitors the network condition and adjusts the sending rate accordingly. The block selection module classifies the blocks based on whether they are estimated to be delivered before deadline and then ranks them according to their effective priority scores. The extensive simulation results demonstrate the superiority of our proposed scheme over the other representative baseline approaches.