Abstract Although variable‐rate compressed image formats such as JPEG are widely used to efficiently encode images, they have not found their way into real‐time rendering due to special requirements such as random access to individual texels. In this paper, we investigate the feasibility of variable‐rate texture compression on modern GPUs using the JPEG format, and how it compares to the GPU‐friendly fixed‐rate compression approaches BC1 and ASTC. Using a deferred rendering pipeline, we are able to identify the subset of blocks that are needed for a given frame, decode these, and colorize the framebuffer's pixels. Despite the additional ∼0.17 bit per pixel that we require for our approach, JPEG maintains significantly better quality and compression rates compared to BC1, and depending on the type of image, outperforms or competes with ASTC. The JPEG rendering pipeline increases rendering duration by less than 0.3 ms on an NVIDIA RTX 4090, demonstrating that sophisticated variable‐rate compression schemes are feasible on modern GPUs, even in VR. Source code and data sets are available at: https://github.com/elias1518693/jpeg_textures

Abstract This paper presents a CUDA‐based software rasterizer capable of rendering up to a billion unique triangles, or up to 4 billion instanced triangles, in real time at 60 fps on an RTX 5090. By specifically targeting dense, opaque meshes, our approach is able to outperform the native GPU rasterization pipeline in these scenarios. The resulting performance enables rapid loading and visualization of massive triangle datasets without requiring precomputed spatial acceleration or level‐of‐detail structures, and supports applications such as efficient editing of large‐scale geometry. While the method is primarily designed for dense meshes that generate pixel‐sized triangles, we additionally introduce a three‐stage pipeline to handle larger primitives. The source code is available at: https://github.com/m-schuetz/CuRast

Parametric functions are an extremely efficient representation for 3D geometry, capable of compactly modelling highly complex objects. Once specified, parametric 3D objects allow for visualization at arbitrary levels of detail (LOD), at no additional memory cost, limited only by the amount of evaluated samples. However, mapping the sample evaluation to the hardware rendering pipelines of modern graphics processing units (GPUs) is not trivial. In this article, we propose a general method for efficient rendering of parametrically-defined 3D objects on modern hardware architectures. Our method adaptively analyzes, allocates and evaluates parametric function samples to produce high-quality renderings. Geometric precision can be modulated from few pixels down to sub-pixel level, enabling real-time frame rates of several 100 frames per second (FPS) for various parametric functions. We propose a dedicated LOD stage, which outputs patches of similar geometric detail to a subsequent rendering stage that uses either a hardware tessellation-based approach or performs point-based software rasterization. Our method requires neither preprocessing nor caching, and the proposed LOD mechanism is fast enough to run each frame. Hence, our approach also lends itself to animated parametric objects. We demonstrate the benefits of our method over a state-of-the-art spherical harmonics (SH) glyph rendering method and over classical LOD approaches, while showing its flexibility on a range of other demanding shapes.

Abstract We present Splatshop, a highly optimized toolbox for interactive editing (selection, deletion, painting, transformation, …) of 3D Gaussian Splatting models. Utilizing a comprehensive collection of heuristic approaches, we carefully balance between exact and fast rendering to enable precise editing without sacrificing real‐time performance. Our experiments confirm that Splatshop achieves these goals for scenes with up to 100 million primitives. We also show how our proposed pipeline can be extended for use with head‐mounted displays. As such, Splatshop is the first VR‐capable editor for large‐scale 3D Gaussian Splatting models and a step towards a “Photoshop for Gaussian Splatting.”

Abstract About: We introduce a GPU‐accelerated LOD construction process that creates a hybrid voxel‐point‐based variation of the widely used layered point cloud (LPC) structure for LOD rendering and streaming. The massive performance improvements provided by the GPU allow us to improve the quality of lower LODs via color filtering while still increasing construction speed compared to the non‐filtered, CPU‐based state of the art. Background: LOD structures are required to render hundreds of millions to trillions of points, but constructing them takes time. Results: LOD structures suitable for rendering and streaming are constructed at rates of about 1 billion points per second (with color filtering) to 4 billion points per second (sample‐picking/random sampling, state of the art) on an RTX 3090 – an improvement of a factor of 80 to 400 times over the CPU‐based state of the art (12 million points per second). Due to being in‐core, model sizes are limited to about 500 million points per 24GB memory. Discussion: Our method currently focuses on maximizing in‐core construction speed on the GPU. Issues such as out‐of‐core construction of arbitrarily large data sets are not addressed, but we expect it to be suitable as a component of bottom‐up out‐of‐core LOD construction schemes.

Abstract In this paper, we present several compute‐based point cloud rendering approaches that outperform the hardware pipeline by up to an order of magnitude and achieve significantly better frame times than previous compute‐based methods. Beyond basic closest‐point rendering, we also introduce a fast, high‐quality variant to reduce aliasing. We present and evaluate several variants of our proposed methods with different flavors of optimization, in order to ensure their applicability and achieve optimal performance on a range of platforms and architectures with varying support for novel GPU hardware features. During our experiments, the observed peak performance was reached rendering 796 million points (12.7GB) at rates of 62 to 64 frames per second (50 billion points per second, 802GB/s) on an RTX 3090 without the use of level‐of‐detail structures. We further introduce an optimized vertex order for point clouds to boost the efficiency of GL_POINTS by a factor of 5× in cases where hardware rendering is compulsory. We compare different orderings and show that Morton sorted buffers are faster for some viewpoints, while shuffled vertex buffers are faster in others. In contrast, combining both approaches by first sorting according to Morton‐code and shuffling the resulting sequence in batches of 128 points leads to a vertex buffer layout with high rendering performance and low sensitivity to viewpoint changes.

Abstract Research in rendering large point clouds traditionally focused on the generation and use of hierarchical acceleration structures that allow systems to load and render the smallest fraction of the data with the largest impact on the output. The generation of these structures is slow and time consuming, however, and therefore ill‐suited for tasks such as quickly looking at scan data stored in widely used unstructured file formats, or to immediately display the results of point‐cloud processing tasks. We propose a progressive method that is capable of rendering any point cloud that fits in GPU memory in real time, without the need to generate hierarchical acceleration structures in advance. Our method supports data sets with a large amount of attributes per point, achieves a load performance of up to 100 million points per second, displays already loaded data in real time while remaining data is still being loaded, and is capable of rendering up to one billion points using an on‐the‐fly generated shuffled vertex buffer as its data structure, instead of slow‐to‐generate hierarchical structures. Shuffling is done during loading in order to allow efficiently filling holes with random subsets, which leads to a higher quality convergence behavior.

Abstract We propose an efficient out‐of‐core octree generation method for arbitrarily large point clouds. It utilizes a hierarchical counting sort to quickly split the point cloud into small chunks, which are then processed in parallel. Levels of detail are generated by subsampling the full data set bottom up using one of multiple exchangeable sampling strategies. We introduce a fast hierarchical approximate blue‐noise strategy and compare it to a uniform random sampling strategy. The throughput, including out‐of‐core access to disk, generating the octree, and writing the final result to disk, is about an order of magnitude faster than the state of the art, and reaches up to around 6 million points per second for the blue‐noise approach and up to around 9 million points per second for the uniform random approach on modern SSDs.

Real-time rendering of large point clouds requires acceleration structures that reduce the number of points drawn on screen. State-of-the art algorithms group and render points in hierarchically organized chunks with varying extent and density, which results in sudden changes of density from one level of detail to another, as well as noticeable popping artifacts when additional chunks are blended in or out. These popping artifacts are especially noticeable at lower levels of detail, and consequently in virtual reality, where high performance requirements impose a reduction in detail. We propose a continuous level-of-detail method that exhibits gradual rather than sudden changes in density. Our method continuously recreates a down-sampled vertex buffer from the full point cloud, based on camera orientation, position, and distance to the camera, in a point-wise rather than chunk-wise fashion and at speeds up to 17 million points per millisecond. As a result, additional details are blended in or out in a less noticeable and significantly less irritating manner as compared to the state of the art. The improved acceptance of our method was successfully evaluated in a user study.

Live coding in virtual reality allows users to create and modify their surroundings through code without the need to leave the virtual reality environment. Previous work focuses on modifying the scene. We propose an application that allows developers to modify virtually everything at runtime, including the scene but also the render engine, shader code and input handling, using standard desktop IDEs through a desktop mirror.

For autonomous parking applications to become possible, knowledge about the parking environment is required. Therefore, a real-time algorithm for detecting parked vehicles from radar data is presented. These data are first accumulated in an occupancy grid from which objects are detected by applying techniques borrowed from the computer vision field. Two random forest classifiers are trained to recognize two categories of objects: parallel-parked vehicles and cross-parked vehicles. Performances of the classifiers are evaluated as well as the capacity of the complete system to detect parked vehicles in real world scenarios.

Robust tracking of extended objects plays a major role in research on highly automated driving applications and advanced driver assistance systems. This paper proposes a new approach to estimate the extension of dynamic objects based on object-local occupancy grid maps. This enables estimating free-formed object shapes while being robust against errors coming from, e.g., incorrect segmentation or association. Its benefit is shown based on 4-layer laser scanner sensor data and is evaluated against a ground truth based on a D-GPS system fused with a highly accurate IMU.

This paper describes a region merging method for joint motion estimation and segmentation of digital video sequences. The region merging criterion is based on the measure of the matching error for a region when applying a previously estimated motion to it. A region adjacency graph is used for data representation, which allows a scan independent processing and gives a high-level view. The method is simple-shape object-oriented and starts from a block-based segmentation. The aim of the proposed technique is to define simple shaped objects in a scene using motion information and a simple test.

Block matching motion estimation techniques have shown their efficiency to reduce the temporal redundancy for video coding. However, they tend to introduce annoying block artifacts in the displaced frame difference due to their block-based model. In this paper, we propose a motion field refinement technique in order to overcome these artifacts. The method relies on vector quantization. The patterns to segment the motion field are derived by the LBG algorithm, the training being performed on segmented patterns obtained from natural images. Simulation results show a significant improvement due to the proposed method, in particular moving edges are much better motion compensated and the visual quality is greatly enhanced.>