In medical image visualization, path tracing of volumetric medical data like computed tomography (CT) scans produces lifelike three-dimensional visualizations. Immersive virtual reality (VR) displays can further enhance the understanding of complex anatomies. Going beyond the diagnostic quality of traditional 2D slices, they enable interactive 3D evaluation of anatomies, supporting medical education and planning. Rendering high-quality visualizations in real-time, however, is computationally intensive and impractical for compute-constrained devices like mobile headsets. We propose a novel approach utilizing Gaussian Splatting (GS) to create an efficient but static intermediate representation of CT scans. We introduce a layered GS representation, incrementally including different anatomical structures while minimizing overlap and extending the GS training to remove inactive Gaussians. We further compress the created model with clustering across layers. Our approach achieves interactive frame rates while preserving anatomical structures, with quality adjustable to the target hardware. Compared to standard GS, our representation retains some of the explorative qualities initially enabled by immersive path tracing. Selective activation and clipping of layers are possible at rendering time, adding a degree of interactivity to otherwise static GS models. This could enable scenarios where high computational demands would otherwise prohibit using path-traced medical volumes.

Abstract Monte‐Carlo path tracing techniques can generate stunning visualizations of medical volumetric data. In a clinical context, such renderings turned out to be valuable for communication, education, and diagnosis. Because a large number of computationally expensive lighting samples is required to converge to a smooth result, progressive rendering is the only option for interactive settings: Low‐sampled, noisy images are shown while the user explores the data, and as soon as the camera is at rest the view is progressively refined. During interaction, the visual quality is low, which strongly impedes the user's experience. Even worse, when a data set is explored in virtual reality, the camera is never at rest, leading to constantly low image quality and strong flickering. In this work we present an approach to bring volumetric Monte‐Carlo path tracing to the interactive domain by reusing samples over time. To this end, we transfer the idea of temporal antialiasing from surface rendering to volume rendering. We show how to reproject volumetric ray samples even though they cannot be pinned to a particular 3D position, present an improved weighting scheme that makes longer history trails possible, and define an error accumulation method that downweights less appropriate older samples. Furthermore, we exploit reprojection information to adaptively determine the number of newly generated path tracing samples for each individual pixel. Our approach is designed for static, medical data with both volumetric and surface‐like structures. It achieves good‐quality volumetric Monte‐Carlo renderings with only little noise, and is also usable in a VR context.

We propose clipping methods that are capable of using complex geometries for volume clipping. The clipping tests exploit per-fragment operations on the graphics hardware to achieve high frame rates. In combination with texture-based volume rendering, these techniques enable the user to interactively select and explore regions of the data set. We present depth-based clipping techniques that analyze the depth structure of the boundary representation of the clip geometry to decide which parts of the volume have to be clipped. In another approach, a voxelized clip object is used to identify the clipped regions. Furthermore, the combination of volume clipping and volume shading is considered. An optical model is introduced to merge aspects of surface-based and volume-based illumination in order to achieve a consistent shading of the clipping surface. It is demonstrated how this model can be efficiently incorporated in the aforementioned clipping techniques.

We propose new clipping methods that are capable of using complex geometries for volume clipping. The clipping tests exploit per-fragment operations on the graphics hardware to achieve high frame rates. In combination with texture-based volume rendering, these techniques enable the user to interactively select and explore regions of the data set. We present depth-based clipping techniques that analyze the depth structure of the boundary representation of the clip geometry to decide which parts of the volume have to be clipped. In another approach, a voxelized clip object is used to identify the clipped regions.

For a comprehensive understanding of tomographic image data in medicine, interactive and high-quality direct volume rendering is an essential prerequisite. This is provided by visualization using 3D texture mapping which is still limited to high-end graphics hardware. In order to make it available in a clinical environment, we present a system which uniquely combines local desktop computers and remote high-end graphics hardware. In this context, we exploit the standard visualization capabilities to a maximum which are available in the clinical environment. For 3D representations of high resolution and quality we access the remote specialized hardware. Various tools for 2D and 3D visualization are provided which meet the requirements of a medical diagnosis. This is demonstrated with examples from the field of neuroradiology which show the value of our strategy in practice.

The reconstruction of isosurfaces from scalar volume data has positioned itself as a fundamental visualization technique in many different applications. But the dramatically increasing size of volumetric data sets often prohibits the handling of these models on affordable low-end single processor architectures. Distributed client-server systems integrating high-bandwidth transmission channels and Web based visualization tools are one alternative to attack this particular problem, but therefore new approaches to reduce the load of numerical processing and the number of generated primitives are required. We outline different scenarios for distributed isosurface reconstruction from large scale volumetric data sets. We demonstrate how to directly generate stripped surface representations and we introduce adaptive and hierarchical concepts to minimize the number of vertices that have to be reconstructed, transmitted and rendered. Furthermore, we propose a novel computation scheme, which allows the user to flexibly exploit locally available resources. The proposed algorithms have been merged together in order to build a platform-independent Web based application. Extensive use of VRML and Java OpenGL bindings allows for the exploration of large scale volume data quite efficiently.