Video streaming platforms and existing ABRs traditionally assume uninterrupted sequential playback, yet users frequently skip to points of interest-a fundamental mismatch causing degradation of quality of experience at high-interest segments while wasting bandwidth on skipped content. We address this through our Hotspot-Aware Joint Optimization framework, which reframes video streaming as a non-monotonic optimization problem with discontinuous state transitions caused by navigation events. Our framework jointly optimizes adaptive bitrate decisions and buffer management by leveraging viewer engagement patterns to predict navigation behavior. Our approach combines: (1) a mathematical formulation capturing state discontinuities in non-sequential viewing, (2) self-supervised models predicting navigation targets using only aggregate viewing data, and (3) hotspot-aware ABR and buffer management algorithms implemented through our Streaming Local Search (SLS) technique that dynamically prioritize quality for frequently-watched segments. Evaluation across diverse content and network conditions demonstrates our framework delivers 38.2% higher quality in hotspot regions, 32.5% reduced navigation delays, and 27.1% improved resource efficiency compared to traditional methods. These improvements establish a foundation for streaming systems that adapt to both network conditions and content structure, aligning resource allocation with actual viewing patterns.

Accurate long-term viewport prediction in tile-based 360° video adaptive streaming helps pre-download tiles for a further future, thus establishing a longer buffer to cope with network fluctuations. Long-term viewport motion is mainly influenced by Historical viewpoint Trajectory (HT) and Video Content information (VC). However, HT and VC are difficult to align in space due to their different modalities, and their relative importance in viewport prediction varies across prediction time steps. In this paper, we propose STAR-VP, a model that fuses HT and VC in a Space-aligned and Time-vARying manner for Viewport Prediction. Specifically, we first propose a novel saliency representation salxyz and a Spatial Attention Module to solve the spatial alignment of HT and VC. Then, we propose a two-stage fusion approach based on Transformer and gating mechanisms to capture their time-varying importance. Visualization of attention scores intuitively demonstrates STAR-VP's capability in space-aligned and time-varying fusion. Evaluation on three public datasets shows that STAR-VP achieves state-of-the-art accuracy for long-term (2-5s) viewport prediction without sacrificing short-term (<1s) prediction performance.

In the field of 360-degree video streaming, the optimization of visual quality within the viewer's field of view, alongside the minimization of system resource consumption, is a critical concern. The considerable file sizes associated with high-quality 360-degree videos pose significant challenges for efficient transmission. Current strategies primarily employ a tile-based approach to address these issues. However, these approaches face two core challenges: the methodology for tile segmentation, and the strategy for downloading tiles. To address these challenges, this paper presents a tile-based solution designed specifically for adaptive 360-degree video-on-demand streaming. Our method surpasses baseline methods in extensive simulations, positioning it as one of the leading solutions in the MMSys'24 Grand Challenge1.

As media streaming grows, service providers have to focus more on users’ quality of experience (QoE). Among the metrics that influence QoE, network-oriented and media-oriented factors have been the primary concern of most existing work. Apart from the two factors, however, another crucial aspect of QoE isthe user. Though the analysis and modeling of users is a turning point in the transition from QoS to QoE, the impact of users’ memory and cognitive-psychological effects have not been fully explored. In this work, we analyze, validate and quantify the relationship between multiple cognitive effects and the QoE of users in media streaming. We propose PsyQoE, a new QoE assessment framework that uniquely focuses on user perceptions during media sessions. Besides providing real-time QoE assessments through leveraging machine learning, PsyQoE also predicts the overall QoE by taking into account user biases and long-term effects. Additionally, we have designed a further modified version to cater to the needs of hardware and energy-restricted scenarios. When compared with other existing methods, our approach demonstrates enhanced performance by boosting accuracy by 7% to 16%. It also provides better interpretability and broader applicability.