Recent advances in Gaussian splatting have enabled real-time and photorealistic rendering of volumetric scenes, showing potential to become a mainstream 3D representation. Through efficient learning, Gaussian splatting transforms multiview images into a 3D representation, which can be interpreted as a point cloud with additional attributes. One of the challenges of Gaussian splatting is the large file size needed for storing its raw representation, which may be in the order of 1GB to 10GB for objects and scenes. Fast and mass market adoption of this technology requires a standard that offers interoperability and allows to efficiently reduce the size of the Gaussian splat representation to manageable bitrates at high visual quality. In this paper, we present how the ISO/IEC 23090-5 visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC) standard can be utilized for coding scenes and objects represented with Gaussian splats. We show that a new profile of V-PCC provides better compression gains compared to other existing methods. Moreover, as V-PCC leverages video coding at its core, the video decoding hardware acceleration ecosystem paves the way towards streaming, decoding and realtime rendering of static Gaussian Splat scenes on mobile devices.

Current networking solutions are unable to satisfy the low-latency requirements of real-time volumetric video conferencing when faced with heavy congestion scenarios. Traditional congestion controllers use packet loss or the change in round-trip time (RTT) to estimate the bandwidth. Commonly, this method is too slow as congestion has already occurred and the receiving user has already experienced a latency spike. Low latency, low loss and scalable throughput (L4S), recently published as RFC 9330, wants to alleviate this problem by aiming for sub 1 ms queuing delay for low-latency traffic by using accurate explicit congestion notification (AccECN) packet marking to notify applications of early congestion. We propose an L4S-based pipeline for volumetric video delivery, which achieves a more consistent latency under congestion compared to web real-time communication (WebRTC). In addition, L4S bandwidth estimation achieves a 45% faster convergence compared to Google congestion control (GCC) estimation, commonly used in WebRTC. Furthermore, in our detailed evaluation setup the L4S application experiences no packet loss, while the WebRTC-based version suffers from irrecoverable packet loss, resulting in 3% of frames being undecodable.

The video-based dynamic mesh coding standard is built on the concepts of mesh decimation to form a base-mesh, subdivision surfaces, displacement data, and a lifting transform. The displacement data describes how higher level of detail geometry can be reconstructed from the subdivided base-mesh. It is hierarchically predicted through each subdivision iteration and then coded using traditional 2D video codecs or arithmetically.Because the base-mesh is smaller in size, it allows for faster decoding and transfer from CPU to GPU. Applications should therefore perform the V-DMC reconstruction process on the GPU to take advantage of this.Currently, the standard specifies a step-by-step iterative reconstruction process, but this is not optimized for real-time rendering. In this paper, we describe a rendering method for V-DMC compressed mesh sequences that enables tessellation-based reconstruction of the base-mesh.

This paper presents the compression of mesh displacement data based on lossy video coding that has been recently adopted into the emerging ISO/IEC 23090-29 Video-based dynamic mesh coding (V-DMC) standard. As its name indicates, the emerging V-DMC standard relies on video coding for the compression of dynamic meshes, specifies metadata and the decoding processes. More specifically, video coding is used in V-DMC to code texture (attribute) data and displacement (geometry) data. V-DMC has been designed on the principles of subdivision surfaces and the lifting scheme; the displacement data consists in residual information that is hierarchically predicted through Levels-of-Details (LoD). Due to this nature of the data, previous versions of V-DMC relied on lossless video coding of displacement data after quantization of the raw displacement signal, with the intuition that this leads to a better control of the quantization error. We show in this paper that signaling a linear scale factor while using a video codec in lossy mode provides better coding gains and other benefits to V-DMC.

We present an end-to-end system for streaming Cinematic Virtual Reality (VR) content (also called 360 or omnidirectional content). Content is captured and ingested at a resolution of 16K at 25Hz and streamed towards untethered mobile VR devices. Besides the usual navigation interactions such as panning and tilting offered by common VR systems, we also provide a zooming interactivity. This allows the VR client to fetch high quality pixels captured at a spatial resolution of 16K that greatly increase perceived quality compared to a 4K VR streaming solution. Since current client devices are not capable of receiving and decoding a 16K video, several optimizations are provided to only stream the required pixels for the current viewport of the user, while meeting strict latency and bandwidth requirements for a qualitative VR immersive experience.

HTTP Adaptive Streaming (HAS) is the de facto standard for video streaming services over the Internet. In HAS, each video is temporally segmented and stored in different qualities. The client selects the quality level for every video segment based on network conditions, allowing a smooth playback with the best possible Quality of Experience (QoE). Although results are promising, current solutions suffer from two problems. First, a low quality and large end-to-end latency are often observed in live streaming scenarios. Second, freezes in the video playout may occur in case of sudden drops of the available bandwidth. We reduced these issues using two complementary approaches. First, we reduced the live latency using the new HTTP/2 server push in combination with super-short segments. Second, we designed an OpenFlow-based network controller that prioritizes the delivery of particular segments to avoid freezes at the clients. The proof-of-concept shows the results obtained when two clients stream a video under varying network conditions. By monitoring the clients' behavior, it is possible to understand the gains brought by the proposed approaches. Particularly, we demonstrate how our solutions consistently reduce the live latency in high round-trip time networks and video freezes caused by network congestion. These results represent a major improvement for the QoE of the final users.

HTTP Adaptive Streaming (HAS) is today the number one video technology for over-the-top video distribution. In HAS, video content is temporally divided into multiple segments and encoded at different quality levels. A client selects and retrieves per segment the most suited quality version to create a seamless playout. Despite the ability of HAS to deal with changing network conditions, HAS-based live streaming often suffers from freezes in the playout due to buffer under-run, low average quality, large camera-to-display delay, and large initial/channel-change delay. Recently, IETF has standardized HTTP/2, a new version of the HTTP protocol that provides new features for reducing the page load time in Web browsing. In this paper, we present ten novel HTTP/2-based methods to improve the quality of experience of HAS. Our main contribution is the design and evaluation of a push-based approach for live streaming in which super-short segments are pushed from server to client as soon as they become available. We show that with an RTT of 300 ms, this approach can reduce the average server-to-display delay by 90.1% and the average start-up delay by 40.1%.

This paper presents an encoder-based prediction model to study the impact of packet loss on H.264 scalable video coding (SVC). A Markov Chain (MC) with 2Nstates is developed to describe the error propagation process inside a group of pictures (GOP). Using the packet loss rate reported by RTCP (RTP Control Protocol) and analyzing the inter-frame prediction rules, we evaluate packet-loss-induced quality degradation in SVC as the probability that a given scalable layer can be correctly received. Based on the proposed model, the performance of the SVC hierarchical B-frame structure, the zero delay encoding/decoding structure, and the advance video coding (AVC) IPPP structure (compatible base layer in SVC) are evaluated and compared under both random and burst packet loss events.

This paper studies the impact of packet loss on H.264 scalable video coding (SVC). A Markov Chain (MC) with 2Nstates is developed to describe the error propagation process inside a group of pictures (GOP). The characteristic of different packet loss events is captured by the initial state vector of the MC. Based on the proposed model, the performance of different GOP structures can be evaluated under both random and burst packet losses.

Spherical Video is nowadays popular, in particular for interactive on-line entertainment multimedia applications such as Google Street View, as well as for Geographical Information Systems (GIS), video surveillance applications, etc. The user is able to explore and navigate audio/visual scenes by freely choosing viewpoint and viewing direction. Spherical Video is typically represented as a classical 2D rectangular panorama video that is then mapped on a sphere mesh and finally rendered on the user's screen. Existing transmission models of spherical video encode the full panorama video with uniform quality while the sphere mesh is kept implicit. However, in general, the user only watches a restricted field-of-view of the spherical content, which can be interactively modified with pan-tilt-zoom commands. In this paper we evaluate the relevance and optimality of a personalized transmission where the panorama video is decomposed into tiles, which are transmitted at a quality modulated in the spherical regions depending on their likelihood to be visualized from the user interactions.

Travel Depth, introduced by Coleman and Sharp in 2006, is a physical interpretation of molecular depth, a term frequently used to describe the shape of a molecular active site or binding site. Travel Depth can be seen as the physical distance a solvent molecule would have to travel from a point of the surface, i.e., the Solvent-Excluded Surface (SES), to its convex hull. Existing algorithms providing an estimation of the Travel Depth are based on a regular sampling of the molecule volume and the use of the Dijkstra's shortest path algorithm. Since Travel Depth is only defined on the molecular surface, this volume-based approach is characterized by a large computational complexity due to the processing of unnecessary samples lying inside or outside the molecule. In this paper, we propose a surface-based approach that restricts the processing to data defined on the SES. This algorithm significantly reduces the complexity of Travel Depth estimation and makes possible the analysis of large macromolecule surface shape description with high resolution. Experimental results show that compared to existing methods, the proposed algorithm achieves accurate estimations with considerably reduced processing times.

In this paper, we propose a blind and robust watermarking method for 3D polygonal meshes by minimising the mean square error between the original mesh and the watermarked mesh under several constraints. We have formulated the problem of assigning distortions to points in a 3D mesh to a quadratic programming problem, so it can be solved reliably and efficiently. Comparing with similar approaches in [1], experiments indicate the advantages of our method in resisting Gaussian noise.

State-of-the-art blind and robust 3D watermarking schemes already withstand combinations of a wide variety of attacks (e.g. noise addition, simplification, smoothing, ...) except cropping. This attack is however very common and should be dealt with in a copyright protection framework. In this paper, we propose a technique which enables to extend the robustness of such schemes to cropping. Our algorithm proceeds by the automatic detection of robust shape feature points which are then used for the embedding of a watermark in a local neighborhood. We show that robustness against cropping and other common attacks is achieved provided that at least one feature point as well as its corresponding local neighborhood are retrieved.

We present a new blind robust watermarking scheme for 3D meshes. Feature points are used to build a partition of the mesh shape that resists to common 3D watermarking attacks. These points are automatically selected through a multi-scale estimation of the curvature tensor field. The automatic capture of robust feature points and its use for blind detection in a robust watermarking scheme are the contribution of this paper. Our watermarking scheme proceeds by first partitioning the mesh shape using a geodesic Delaunay triangulation of the detected feature points. Each of these geodesic triangle patches is then parameterized and remeshed by a subdivision strategy to obtain a robust base meshing. Then remeshed patches are watermarked in the spectral domain and original mesh points are finally projected on the corresponding watermarked patches. This strategy shows good preliminary results as it resists to affine transforms, white noise addition, smoothing, crop and sampling changes such as decimation, subdivision or remeshing.

Spectral decomposition of mesh geometry has first been introduced by Taubin for geometry processing purposes. It has been extended to address transmission issues by Kami & Gotsman. Such decompositions give rise to pseudofrequential information of the geometry defined over the mesh connectivity. For large meshes a piecewise decomposition has to be applied in order to restrict the complexity of the transform. In this paper, we propose to introduce overlap for its spectral representation. We show visual gains obtained in compression and progressive transmission of mesh geometry.