A dynamic 3D model is a key component in a Virtual Reality environment. To reduce the processing requirement while preserving the user experience, an adaptive Level of Detail (LoD) of a 3D model based on the viewing distance has been proposed. In this paper, aiming to optimize the generation and selection of LoD versions, we investigate the effects of Level of Detail and viewing distance on user perception of dynamic 3D models in a VR environment. We conducted a subjective experiment with five representative dynamic 3D models under four Level of Detail and five viewing distance settings in a Virtual Reality environment. Experimental results show that the MOS score increases by 0.3 to 2.1 as the viewing distance increases from$d=4 \mathrm{m}$to$d=20 \mathrm{m}$. Moreover, removing up to 50% of a dynamic 3D model's faces has a negligible impact on the user's perception. An evaluation of popular objective quality metrics reveals that video PSNR has the highest correlation with subjective scores.

Re-buffering is one of the most common factors that reduce the user’s Quality of Experience in online streaming video. In this paper, we introduce a novel solution to tackle the rebuffering problem in video streaming using generative Artificial Intelligence (genAI). In our proposed solution, a generative AI model is trained and executed to generate video frames in case of sudden network bandwidth drops. For that purpose, we first formulate the genAI-assisted adaptation problem in adaptive video streaming as an optimization problem. We then present a lightweight adaptation algorithm featuring a network reduction detection module and an AI-based video frame generation module. Experiment results show that the proposed method can effectively reduce the number of re-buffering events under challenging network conditions.

3D meshes are one of the main components of Virtual Reality applications. However, a huge amount of network and computational resources are required to process 3D meshes in real time. A potential solution to this challenge is to dynamically adapt the Level of Detail (LoD) of a 3D mesh based on the object’s position and the user’s viewpoint. In this paper, we conduct a subjective study to investigate users’ quality perception of 3D meshes with dynamic Levels of Detail in a Virtual Reality environment. The subjective experiment is carried out with five 3D meshes of different characteristics, four Levels of Detail, and four distance settings. The results of the experiment show that the impact of the dynamic level of detail depends on both the position of the 3D object in the virtual world and the number of vertices of the original mesh. In addition, we present a quality model that can accurately predict the MOS score of a LoD version of a 3D mesh from the number of vertices and the distance from the viewpoint.

Point cloud video is an effective method to represent moving objects for metaverse applications. Real-time streaming of point cloud video can offer truly immersive experiments for metaverse users. However, point cloud videos have an extremely high data rate and demand significant processing resources for compression and rendering at the user device. To address the above challenges, this paper presents a novel server-driven view-aware point cloud video streaming system that can effectively reduce bandwidth requirements by combining hidden point removal and video-based point cloud encoding. Experiment results show that the proposed method can reduce the bandwidth requirement by up to 24% compared to the baseline method.

Point cloud video streaming over networks is challenging because of the high data rate of uncompressed point cloud data. Adaptive point cloud video streaming has been proposed to deal with this challenge. However, temporal quality variation and stalling might occur under unstable network conditions, potentially degrading users’ Quality of Experience (QoE). This paper aims to evaluate and model the impacts of temporal quality variations and stalling on users’ QoE in adaptive point cloud video streaming. We first conduct a large-scale subjective study to construct a QoE database. Then, based on the constructed database, the effects of individual factors are analyzed, and two novel QoE prediction models are presented. Experiment results show that the proposed QoE models achieve high prediction performance in PLCC, SROCC, and RMSE across various point cloud videos.

Point cloud is the emerging format for representing real-world objects in VR/AR applications. However, real-time streaming of dynamic point clouds presents challenges due to high data rates and low latency requirements. This paper introduces a novel and bandwidth-efficient streaming approach for scenes consisting of multiple dynamic point clouds over networks with limited bandwidth. The proposed approach dynamically adjusts the Level of Detail (LoD) of individual point clouds based on network conditions and user preferences to optimize the user’s Quality of Experience (QoE). The LoD version selection problem is formulated as a QoE optimization problem, and two real-time solutions are presented for deciding the LoD version for each point cloud. Experimental results demonstrate that the proposed method outperforms the existing methods in terms of visual quality while achieving remarkably low processing time, about 0.01 ms. These findings have the potential to advance seamless user experience.

With the ability to provide an “immersive experience”, 360-degree video-based applications are becoming more and more popular nowadays. In this paper, we propose LL-VAS, a novel adaptation method for low-latency 360-degree video streaming over mobile networks. By applying tile-based streaming, the proposed method allows 360-degree video streaming over resource-constrained mobile networks. In addition, by actively monitoring network throughput at the tile level, the proposed method can detect reductions in network throughput, and adapt video content in a timely manner to avoid re-buffering. Trace-driven experiments show that the proposed method can significantly decrease the number of re-buffering and re-buffering time under strong network throughput fluctuations and small buffer size when compared to reference methods.

Recent years have witnessed the rising of short-form video platforms such as TikTok. Apart from conventional videos, short-form videos are much shorter and users frequently change the content to watch. Thus, it is crucial to have an effective streaming method for this new type of video. In this paper, we propose a resource-efficient prefetching method for short-form video streaming. Taking into account network throughput conditions and user viewing behaviors, the proposed method dynamically adapts the amount of prefetched video data. Experiment results show that our method can reduce the data waste by 37$\sim$52% compared to other existing methods.

HTTP Adaptive Streaming has become a popular solution for multimedia delivery nowadays. However, due to network bandwidth fluctuations, video quality strongly varies during streaming. Therefore, a key challenge in HTTP Adaptive Streaming is how to evaluate the overall quality of a streaming session. In this article, a machine learning approach is proposed for overall quality prediction, where each segment in a streaming session is represented by a set of features. Two options of the feature set are investigated. In the first option, we use four features, namely segment quality, content characteristics, stalling duration, and padding. The second option consists of three features, namely bitstream-level parameters, stalling duration, and padding. The features are fed into a Long Short Term Memory (LSTM) network that is capable of exploring temporal relations between impairment events of quality variations and stalling events. The overall quality is predicted from the outputs of the LSTM network using a linear regression module. Through experimental results, it is shown that the proposed approach achieves a high prediction performance and outperforms seven existing approaches. Especially, the second option is found to be both efficient and effective. The source code of the proposed approach has been made available to the public.

HTTP Adaptive Streaming (HAS) has become a popular solution for multimedia delivery nowadays. However, because of throughput fluctuations, video quality may be dramatically varying. Also, stalling events may occur during a streaming session, causing negative impacts on user experience. Therefore, a main challenge in HAS is how to evaluate the overall quality of a session taking into account the impacts of quality variations and stalling events. In this paper, we present an open software, called BiQPS, using a Long-Short Term Memory (LSTM) network to predict the overall quality of HAS sessions. The prediction is based on bitstream-level parameters, so it can be directly applied in practice. Through experiment results, it is found that BiQPS outperforms four existing models. Our software has been made available to the public at https://github.com/TranHuyen1191/BiQPS.

With the deployment of smart homes, we find that human activity recognition (HAR) is essentially important to many applications, e.g., child/senior care, intelligent information push and exercise promotion. Although it is always better to build HAR model for each smart home to resolve the practical problem that homes have different floorplans or adopted sensors, it is intractable to acquire labeled data for each home due to cost and privacy. We thus propose a method to transfer the HAR model from multiple labeled source homes to the unlabeled target home. Specifically, we first generate transferable representations for the sensors of these homes, based on which we build the HAR model using the data of labeled source homes. Then, we employ the built HAR model into the unlabeled target home. Experiment results on CASAS dataset illustrate that our proposed method outperforms baseline methods in general and also avoids potential negative transfer caused by using only one source home.

360-degree video has become increasingly popular nowadays. For effective transmission of bandwidth-intensive 360-degree video over networks, viewport-adaptive streaming has been introduced. In this article, we evaluate, for the first time, ten existing methods to understand the effectiveness of tile-based viewport adaptive streaming of 360-degree video. Experimental results show that tile-based methods can improve the average V-PSNR by up to 4.3 dB compared to a non-tiled method under low delay settings. Here, the V-PSNR is computed as the peak signal-to-noise ratio of the adapted viewport compared to the corresponding origin viewport. Also, different methods show different tradeoffs between average viewport quality and viewport quality variations. Especially, the performances of most tile-based methods decrease quickly as the segment duration and/or buffer size increase for the content with no main focus. Even, under long delay settings like HTTP Adaptive Streaming, it is found that the simple non-tiled method appears to be the best one. For the content with a strong viewing focus, it is found that the tile-based methods are less influenced by the segment duration and the buffer size. In addition, a comparison of the performances of the tile selection methods using two popular viewport estimation methods is conducted. It is interesting that there is only little difference found in performances of tile selection methods. The findings of this study are useful for service providers to make decisions on deployment of streaming solutions.

360-degree video is the main content type of Virtual Reality, providing users with immersive viewing experience. In this paper, we propose a novel adaptation method for 360-degree video streaming over HTTP/2, which can provide high viewing experience to users under time-varying network conditions and time-varying user head movements. The proposed method utilizes Scalable Video Coding to solve the trade-off between network adaptivity and user adaptivity. An optimal tile layer selection algorithm is provided. To cope with sudden throughput drops, the delivery of late layers is terminated using HTTP/2's stream termination feature. Also, a tile layer updating scheme is proposed to deal with viewport estimation errors. Experimental results show that the proposed method can improve the average bitrate of viewport by 16-17% compared to a reference method.

In this paper, we propose a new adaptation approach for viewport-adaptive streaming of 360-degree videos over the Internet. The proposed approach is able to systematically decide quality levels of tiles according to user head movements and network conditions by taking into account not only prediction errors but also user head movements in each adaptation interval. Experimental results show that the proposed approach can effectively adapt 360-degree videos to both varying network conditions and user head movements. Compared to existing approaches, the proposed approach can improve the average viewport quality by up to 3.9dB and reduce the standard deviation of the viewport quality by up to 50%.

HTTP streaming has become a cost effective means for video delivery nowadays. To enable adaptivity to networks and terminals, a provider should generate multiple representations of an original video as well as the related signaling metadata. So far, most previous studies have just focused on the case of CBR (constant bitrate) video. In this paper, we propose a novel adaptation method for VBR (variable bitrate) video streaming. Based on a trellis representation to estimate future buffer levels, the proposed method can provide smooth video quality while avoiding buffer underflows. The experimental results show that our approach can perform effectively under drastic changes of both connection throughput and video bitrate.