Live volumetric video streaming provides immersive and interactive experiences. However, state-of-the-art live streaming systems need excessive bandwidth, exceeding the bandwidth capabilities of common mobile networks. A basic solution is to eliminate the redundancy of the original captured data and compress the created volumetric video. However, existing approaches either incorrectly discard normal data or encode the data inefficiently. In this paper, we propose LiVo, a bandwidth-efficient Live Volumetric video streaming system that comprises (i) a compact capture method that efficiently eliminates superfluous points from multiple partial point clouds, and (ii) a compact attribute encoding method that eliminates the spatial-temporal redundancy by proximity-reserved attribute mapping. We implement a prototype of LiVo using commodity devices and extensively evaluate its performance. Our results demonstrate that LiVo achieves bandwidth-efficient live volumetric video streaming and supports transmission under common mobile networks for the first time. Compared with state-of-the-art systems, LiVo reduces bandwidth consumption by at least 47.93%.

ABSTRACT High‐fidelity, speech‐driven 3D facial animation is crucial for immersive applications and virtual avatars. Nevertheless, advancement is impeded by two principal challenges: (1) a lack of high‐quality 3D data, and (2) inadequate modelling of the multi‐scale characteristics of speech signals. In this paper, we present Speech2Face3D, a novel two‐stage transfer‐learning framework that pretrains on large‐scale pseudo‐3D facial data derived from 2D videos and subsequently finetunes on smaller yet high‐fidelity 3D datasets. This design leverages the richness of easily accessible 2D resources while mitigating reconstruction noise through a simple temporal smoothing step. Our approach further introduces a Multi‐Scale Hierarchical Audio Encoder to capture subtle phoneme transitions, mid‐range prosody, and longer‐range emotional cues. Extensive experiments on public 3D benchmarks demonstrate that our method achieves state‐of‐the‐art performance on lip synchronization, expression fidelity, and temporal coherence metrics. Qualitative user evaluations validate these quantitative improvements. Speech2Face3D is a robust and scalable framework for utilizing extensive 2D data to generate precise and realistic 3D facial animations only based on speech.

Volumetric videos offer viewers more immersive experiences, enabling a variety of applications. However, state-of-the-art streaming systems still need hundreds of Mbps, exceeding the common bandwidth capabilities of mobile devices. We find a research gap in reusing inter-frame redundant information to reduce bandwidth consumption, while the existing inter-frame compression methods rely on the so-called explicit correlation, i.e., the redundancy from the same/adjacent locations in the previous frame, which does not apply to highly dynamic frames or dynamic viewports. This work introduces a new concept called implicit correlation, i.e., the consistency of topological structures, which stably exists in dynamic frames and is beneficial for reducing bandwidth consumption. We design a mobile volumetric video streaming system Hermes consisting of an implicit correlation encoder to reduce bandwidth consumption and a hybrid streaming method that adapts to dynamic viewports. Experiments show that Hermes achieves a frame rate of 30+ FPS over daily networks and on commodity smartphones, with at least 3.37x improvement compared with two baselines.