Recently, 3D GANs based on 3D Gaussian splatting have been proposed for high quality synthesis of human heads. However, existing methods stabilize training and enhance rendering quality from steep viewpoints by conditioning the random latent vector on the current camera position. This compromises 3D consistency, as we observe significant identity changes when re-synthesizing the 3D head with each camera shift. Conversely, fixing the camera to a single viewpoint yields high-quality renderings for that perspective but results in poor performance for novel views. Removing view-conditioning typically destabilizes GAN training, often causing the training to collapse. In response to these challenges, we introduce CGS-GAN, a novel 3D Gaussian Splatting GAN framework that enables stable training and high-quality 3D-consistent synthesis of human heads without relying on view-conditioning. To ensure training stability, we introduce a multi-view regularization technique that enhances generator convergence with minimal computational overhead. Additionally, we adapt the conditional loss used in existing 3D Gaussian splatting GANs and propose a generator architecture designed to not only stabilize training but also facilitate efficient rendering and straightforward scaling, enabling output resolutions up to $2048^2$. To evaluate the capabilities of CGS-GAN, we curate a new dataset derived from FFHQ. This dataset enables very high resolutions, focuses on larger portions of the human head, reduces view-dependent artifacts for improved 3D consistency, and excludes images where subjects are obscured by hands or other objects. As a result, our approach achieves very high rendering quality, supported by competitive FID scores, while ensuring consistent 3D scene generation.

Abstract 3D Gaussian Splatting (3DGS) has emerged as a cutting‐edge technique for real‐time radiance field rendering, offering state‐of‐the‐art performance in terms of both quality and speed. 3DGS models a scene as a collection of three‐dimensional Gaussians, with additional attributes optimized to conform to the scene's geometric and visual properties. Despite its advantages in rendering speed and image fidelity, 3DGS is limited by its significant storage and memory demands. These high demands make 3DGS impractical for mobile devices or headsets, reducing its applicability in important areas of computer graphics. To address these challenges and advance the practicality of 3DGS, this state‐of‐the‐art report (STAR) provides a comprehensive and detailed examination of two complementary yet fundamentally distinct strategies: compression and compaction. Compression techniques focus on reducing the file size by encoding Gaussian attributes more efficiently. In contrast, compaction methods directly optimize the scene's structure by optimizing the number of Gaussian primitives. Notably, while methods in both categories aim to maintain or improve quality, each while minimizing its respective attributes—file size for compression and the number of Gaussians for compaction—compaction does not necessarily lead to smaller file sizes; it specifically targets improved efficiency during rendering, making it distinct from compression. We introduce the basic mathematical concepts underlying the analyzed methods, as well as key implementation details and design choices. Our report thoroughly discusses similarities and differences among the methods, as well as their respective advantages and disadvantages. We establish a consistent framework for comparing the surveyed methods based on key performance metrics and datasets. Specifically, since these methods have been developed in parallel and over a short period of time, currently, no comprehensive comparison exists. This survey, for the first time, presents a unified framework to evaluate 3DGS compression techniques. To facilitate the continuous monitoring of emerging methodologies, we maintain a dedicated website that will be regularly updated with new techniques and revisions of existing findings. Overall, this STAR provides an intuitive starting point for researchers interested in exploring the rapidly growing field of 3DGS compression. By comprehensively categorizing and evaluating existing compression and compaction strategies, our work advances the understanding and practical application of 3DGS in computationally constrained environments.

We propose a novel representation of virtual humans for highly realistic real-time animation and rendering in 3D applications. We learn pose dependent appearance and geometry from highly accurate dynamic mesh sequences obtained from state-of-the-art multiview-video reconstruction. Learning pose-dependent appearance and geometry from mesh sequences poses significant challenges, as it requires the network to learn the intricate shape and articulated motion of a human body. However, statistical body models like SMPL provide valuable a-priori knowledge which we leverage in order to constrain the dimension of the search space, enabling more efficient and targeted learning and to define pose-dependency. Instead of directly learning absolute pose-dependent geometry, we learn the difference between the observed geometry and the fitted SMPL model. This allows us to encode both pose-dependent appearance and geometry in the consistent UV space of the SMPL model. This approach not only ensures a high level of realism but also facilitates streamlined processing and rendering of virtual humans in real-time scenarios.

Accurate and temporally consistent modeling of human bodies is essential for a wide range of applications, including character animation, understanding human social behavior, and AR/VR interfaces. Capturing human motion accurately from a monocular image sequence remains challenging; modeling quality is strongly influenced by temporal consistency of the captured body motion. Our work presents an elegant solution to integrating temporal constraints during fitting. This increases both temporal consistency and robustness during optimization. In detail, we derive parameters of a sequence of body models, representing shape and motion of a person. We optimize these parameters over the complete image sequence, fitting a single consistent body shape while imposing temporal consistency on the body motion, assuming body joint trajectories to be linear over short time. Our approach enables the derivation of realistic 3D body models from image sequences, including jaw pose, facial expression, and articulated hands. Our experiments show that our approach accurately estimates body shape and motion, even for challenging movements and poses. Further, we apply it to the particular application of sign language analysis, where accurate and temporally consistent motion modelling is essential, and show that the approach is well-suited to this kind of application.

An end‐to‐end pipeline for the creation of high‐quality animatable volumetric video of human performances is presented. Going beyond the application of free‐viewpoint video, the authors allow re‐animation and alteration of an actor's performance through the enrichment of the captured data with semantics and animation properties. Hybrid geometry‐ and video‐based animation methods are applied that allow a direct animation of the high‐quality data itself instead of creating a CG model that resembles the captured data. Semantic enrichment and animation are achieved by establishing temporal consistency followed by automatic rigging of each 3D frame using a parametric human body model. The hybrid approach combines the flexibility of classical CG animation with the realism of real captured data. For the face, coarse movements are modelled in the geometry only, while very fine and subtle details, often lacking in purely geometric methods, are captured in video textures, which can interactively be combined to form new facial expressions. On top of that, regions that are challenging to synthesise, such as the teeth or the eyes, are learned and filled in realistically in an autoencoder‐based approach. This study covers the full pipeline from capturing, volumetric video production, and enrichment with semantics for the final hybrid animation.