Free-viewpoint videos (FVVs) streaming plays a pivotal role in driving the progress of immersive AR/VR applications. To be streamable, FVVs should simultaneously satisfy perframe decoding, compactness, and real-time rendering. Neural radiance based works suffer from low rendering speed, while previous 3D Gaussians (3DGs) based works necessitate large data volumes and pose challenges for compact representation. To support FVVs streaming, we present 3DGCoding, a novel framework for efficient reconstruction and coding of 3DGs for dynamic scenes. We design a learnable 3D mask to partition 3DGs into static and dynamic components, capturing all types of changes and making 3DGs quantitatively and attributively consistent for coding. Then, we propose a 2D-based coding process to further address inter-frame redundancy, achieving independent decoding and smaller data volumes. Experiments confirm that 3DGCoding could stream with 0.49MB per frame, and achieves competitive performance compared with state-of-the-art FVVs streaming methods.

Volumetric videos afford comprehensive and immer-sive viewing experiences with six degrees of freedom (6DoF) for navigation, allowing users to move freely within a three-dimensional space. Radiance fields (RF) is emerging to reproduce photorealistic 3D scenes, which achieves lighting consistency and realistic transfer between the real and virtual worlds. In this paper, we investigate how to deliver photorealistic volumetric video under the network bandwidth constraints and low-power device. We design a novel Scalable Radiance Fields Video Streaming, SRFC, to enable streaming RF video with planar codec. An Orthogonal Layered Depth Image (OLDI) mapping is introduced to map 3D to view-dependent 2D plane images, in order to reduce bitrates and decoding complexity by the commercial 2D codec. Moreover, to address the issue of deviation of the user viewpoint, we propose a dual-tiered structure in radiance fields and optimizes user's perceptual quality by adaptive bitrate and viewpoint decision. The evaluations demonstrate that SRFC is capable of reducing bandwidth requirements by 4 × and increasing decoding frame rate by nearly 3 × while maintaining satisfactory user perception of video quality.

Thanks to the high-quality reconstruction and photorealistic rendering, the Neural Radiance Field (NeRF) has garnered extensive attention and has been continuously improved. Despite its high visual quality, the prohibitive training time limits its practical application. Although significant acceleration has been achieved, it is still far from real-time training, due to the need for tens of thousands of iterations. In this paper, a feasible solution is to reduce the number of required iterations by always training the rays with the highest loss values, instead of the traditional method of training each ray with a uniform probability. To this end, we propose an online active ray sampling strategy, ActRay. Specifically, to avoid the substantial overhead of calculating the actual loss values for all rays in each iteration, a rendering-gradient-based loss propagation algorithm is presented to efficiently estimate the loss values. To further narrow the gap between the estimated loss and the actual loss, an online learning algorithm based on the Upper Confidence Bound (UCB) is proposed to control the sampling probability of the rays, thereby compensating for the bias in loss estimation. We evaluate ActRay on both real-world and synthetic scenes, and the promising results show that it accelerates radiance field training by 6.5x. Besides, we test ActRay under all kinds of radiance field representations (implicit, explicit, and hybrid), proving that it is general and effective to different representations. We believe this work will contribute to the practical application of radiance fields, because it has taken a step closer to real-time radiance field training. ActRay is open-source at: https://pku-netvideo.github.io/actray/.