We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recent rasterization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering, rather than alpha compositing 3D Gaussian billboards. As such, unlike 3DGS our formulation does not suffer from popping artifacts and view dependent density, but still achieves frame rates of $\sim\!30$ FPS at 720p on an NVIDIA RTX4090. Since our approach is built upon ray tracing it enables effects such as defocus blur and camera distortion (e.g. such as from fisheye cameras), which are difficult to achieve by rasterization. We show that our method is more accurate with fewer blending issues than 3DGS and follow-up work on view-consistent rendering, especially on the challenging large-scale scenes from the Zip-NeRF dataset where it achieves sharpest results among real-time techniques.

We present Neural Microfacet Fields, a method for recovering materials, geometry, and environment illumination from images of a scene. Our method uses a microfacet reflectance model within a volumetric setting by treating each sample along the ray as a (potentially non-opaque) surface. Using surface-based Monte Carlo rendering in a volumetric setting enables our method to perform inverse rendering efficiently by combining decades of research in surface-based light transport with recent advances in volume rendering for view synthesis. Our approach outperforms prior work in inverse rendering, capturing high fidelity geometry and high frequency illumination details; its novel view synthesis results are on par with state-of-the-art methods that do not recover illumination or materials.

Image demosaicing and rectification are key tasks that are frequently used in many computer vision systems. To date, however, their implementations have been plagued with large memory requirements and inconvenient dataflow, making it difficult to scale them to real-time, high resolution settings. This has motivated the development of joint demo-saicing and rectification algorithms that resolve the back-ward mapping dataflow for improved hardware implementation. Towards this purpose, we propose Splatty: an algorithmic solution to pipelined image stream demosaicing and rectification for memory bound applications requiring computational efficiency. We begin by introducing a polynomial Look-up-Table (LUT) compression scheme that can encode any arbitrarily complex lens model for rectification while keeping the remapping errors below 1E-10 pixels, and reducing the memory footprint to O(min(m, n)) from O(mn) for an m × n sized image. The core contribution leverages this LUT for a unified, forward-only splatting algorithm for simultaneous demosaicing and rectification. We demonstrate that merging these two steps into a single, forward-only splatting pass with interpolation, provides distinctive dataflow and performance efficiency benefits while maintaining quality standards when compared to state-of-the-art demosaicing and rectification algorithms.

This paper introduces a distributed approach for playback of video content at resolutions of 4K (digital cinema) and well beyond. This approach is designed for scalable, high-resolution, multi-tile display environments, which are controlled by a cluster of machines, with each node driving one or multiple displays. A preparatory tiling pass separates the original video into a user definable n-by-m array of equally sized video tiles, each of which is individually compressed. By only reading and rendering the video tiles that correspond to a given node's viewpoint, the computation power required for video playback can be distributed over multiple machines, resulting in a highly scalable video playback system. This approach exploits the computational parallelism of the display cluster while only using minimal network resources in order to maintain software-level synchronization of the video playback. While network constraints limit the maximum resolution of other high-resolution video playback approaches, this algorithm is able to scale to video at resolutions of tens of millions of pixels and beyond. Furthermore the system allows for flexible control of the video characteristics, allowing content to be interactively reorganized while maintaining smooth playback. This approach scales well for concurrent playback of multiple videos and does not require any specialized video decoding hardware to achieve ultra-high resolution video playback.

A modern field science such as archaeology is heavily data-driven using various kinds of state-of-the-art measurement instruments. It requires sophisticated computer infrastructure to manage large amounts of heterogeneous data. The concept of cloud computing provides a flexible cyber infrastructure for large-scale data management, which is being deployed at university campuses. A problem unique to field research is that researchers often work at remote field sites with limited computer and network resources. For a data management system that has to work in the campus cloud and under vastly different field conditions, portability of computer infrastructure and common data access methods are essential requirements. This paper explores the portability of cloud infrastructure and illustrates the portable data management system that we used in a recent archaeological expedition.

To ensure the preservation of cultural heritage, artifacts such as paintings must be analyzed to diagnose physical frailties that could result in permanent damage. Advancements in digital imaging techniques and computer-aided analysis have greatly aided in such diagnoses but can limit the ability to work directly with the artifact in the field. This paper presents the implementation and application of ARtifact, a tablet-based augmented reality system that enables on-site visual analysis of the artifact in question. Utilizing real-time tracking of the artifact under observation, a user interacting with the tablet can study various layers of data registered with the physical object in situ. Theses layers, representing data acquired through various imaging modalities such as infrared thermography and ultraviolet fluorescence, provide the user with an augmented view of the artifact to aid in on-site diagnosis and restoration. Intuitive interaction techniques further enable targeted analysis of artifact-related data. We present a case study utilizing our tablet system to analyze a 16th century Italian hall and highlight the benefits of our approach.

The broad adoption of diagnostic and analytical techniques in the field of archaeology, presents a unique opportunity for e-Science in the form of scientific explanation, drawing from methodologies aimed at recording, archiving, analyzing, and disseminating, rich data collections. This paper presents a needed stepping stone towards cyber-archaeology, creating synergy between information technology and archeology, by introducing a data acquisition, tagging and characterization pipeline.

The Cross Platform Cluster Graphics Library (CGLX) is a flexible and transparent OpenGL-based graphics framework for distributed, high-performance visualization systems. CGLX allows OpenGL based applications to utilize massively scalable visualization clusters such as multiprojector or high-resolution tiled display environments and to maximize the achievable performance and resolution. The framework features a programming interface for hardware-accelerated rendering of OpenGL applications on visualization clusters, mimicking a GLUT-like (OpenGL-Utility-Toolkit) interface to enable smooth translation of single-node applications to distributed parallel rendering applications. CGLX provides a unified, scalable, distributed OpenGL context to the user by intercepting and manipulating certain OpenGL directives. CGLX's interception mechanism, in combination with the core functionality for users to register callbacks, enables this framework to manage a visualization grid without additional implementation requirements to the user. Although CGLX grants access to its core engine, allowing users to change its default behavior, general development can occur in the context of a standalone desktop. The framework provides an easy-to-use graphical user interface (GUI) and tools to test, setup, and configure a visualization cluster. This paper describes CGLX's architecture, tools, and systems components. We present performance and scalability tests with different types of applications, and we compare the results with a Chromium-based approach.

VirtualizeMe introduces a new design for a fully immersive Tele-Immersion system for remote collaboration and virtual world interaction. This system introduces a new avatar creation approach full-filling four main attributes: high resolution, scalability, flexibility and affordability. This is achieved by a total separation of reconstruction and rendering and exploiting the capabilities of modern graphic cards. The high resolution is achieved by using as much of the input information as possible through lossless compression of the input data and introducing a focused volumetric visual hull reconstruction. The resulting avatar allows eye-to-eye collaboration for remote users. The interaction with the virtual world is facilitated by the volumetric avatar model and allows a fully immersive system. This paper shows a proof of concept based on publicly available pre-recorded data to allow easier comparison.

This paper introduces a region-of-interest visual hull refinement technique, based on flexible voxel grids for volumetric visual hull reconstructions. Region-of-interest refinement is based on a multi-pass process, beginning with a focussed visual hull reconstruction, resulting in a first 3D approximation of the target, followed by a region-of-interest estimation, tasked with identifying features of interest, which in turn are used to locally refine the voxel grid and extract a higher-resolution surface representation for those regions. This approach is illustrated for the reconstruction of avatars for use in tele-immersion environments, where head and hand regions are of higher interest. To allow reproducability and direct comparison a publicly available data set for human visual hull reconstruction is used. This paper shows that region-of-interest reconstruction of the target is faster and visually comparable to higher resolution focused visual nhull reconstructions. This approach reduces the amount of data generated through the reconstruction, allowing faster post processing, as rendering or networking of the surface voxels. Reconstruction speeds support smooth interactions between the avatar and the virtual environment, while the improved resolution of its facial region and hands creates a higher-degree of immersion and potentially impacts the perception of body language, facial expressions and eye-to-eye contact.

Tiled display environments present opportune workspaces for displaying multimedia content. Often times, displaying this kind of audio/visual information requires a substantial amount of network bandwidth. In this paper we present a method for distributing the decoding over the entire display environment, allowing for substantially less network overhead. This method also allows for interactive configuration of the visual workspace. The method described scales independently of video frame size and produces lower latency compared to streaming approaches.

Abstract A visual analytics technique for the intuitive, hands‐on analysis of massive, multi‐dimensional and multi‐variate data is presented. This multi‐touch‐based technique introduces a set of metaphors such as wiping, scratching, sandblasting, squeezing and drilling, which allow for rapid analysis of global and local characteristics in the data set, accounting for factors such as gesture size, pressure and speed. A case study is provided for the analysis of multi‐spectral image data of cultural artefacts. By aligning multi‐spectral layers in a stack, users can apply different multi‐touch metaphors to investigate features across different wavelengths. With this technique, flexibly definable regions can be interrogated concurrently without affecting surrounding data.

Tele-Immersion depends on face-to-face, viewpoint corrected, stereoscopic, virtual environments, allowing users to naturally interact with each other and the digital environment surrounding them, via realistic avatars. This paper presents an extraction, modeling and communication methodology for the creation of the needed avatars, based on a scalable focused disparity map (FDM) approach. The FDM enables variable distance reconstruction of dynamic target objects despite restrictions in disparity range. Combined with a pixel-based disparity cost interpolation, a sub-pixel disparity refinement is achieved, providing a high depth-resolution and smooth reconstruction of the target objects. This technique approaches real-time characteristics, extracting avatar models and communicating them to remote render nodes at 15 frames per second.

The ability to explore, conceptualize and correlate spatial and temporal changes of topographical records, is needed for the development of new analytical models that capture the mechanisms contributing towards sea cliff erosion. This paper presents a VR-centric approach for cliff erosion assessment from light detection and ranging (LIDAR) data, including visualization techniques for the delineation, segmentation, and classification of features, change detection and annotation. Research findings are described in the context of a sea cliff failure observed in Solana Beach in San Diego county.

Diffusion Tensor Imaging (DTI) of the human brain, coupled with tractography techniques, enable the extraction of large-collections of three-dimensional tract pathways per subject. These pathways and pathway bundles represent the connectivity between different brain regions and are critical for the understanding of brain related diseases. A flexible and efficient GPU-based rendering technique for DTI tractography data is presented that addresses common performance bottlenecks and image-quality issues, allowing interactive render rates to be achieved on commodity hardware. An occlusion query-based pathway LoD management system for streamlines/streamtubes/tuboids is introduced that optimizes input geometry, vertex processing, and fragment processing loads, and helps reduce overdraw. The tuboid, a fully-shaded streamtube impostor constructed entirely on the GPU from streamline vertices, is also introduced. Unlike full streamtubes and other impostor constructs, tuboids require little to no preprocessing or extra space over the original streamline data. The supported fragment processing levels of detail range from texture-based draft shading to full raycast normal computation, Phong shading, environment mapping, and curvature-correct text labeling. The presented text labeling technique for tuboids provides adaptive, aesthetically pleasing labels that appear attached to the surface of the tubes. Furthermore, an occlusion query aggregating and scheduling scheme for tuboids is described that reduces the query overhead. Results for a tractography dataset are presented, and demonstrate that LoD-managed tuboids offer benefits over traditional streamtubes both in performance and appearance.

One of the challenges in the simulation of human motion, either applied to humanoid robots or avatars in virtual environments, is to design a kinematics structure and a set of joint trajectories that move a robot or avatar in a human-like manner. In this paper, a technique is introduced to create accurate human-like motion with a simplified topology as a reference. Using an optical motion capture system, a finite number of key poses are captured from different subjects performing full body articulated movements. Motion is modeled using the Clifford algebra of dual quaternions and dimensional synthesis techniques are applied to generate the kinematic skeleton of a 3D avatar or robot. The synthesized kinematic skeleton provides location of joints and dimensions of the links forming the limbs, as well as the joint trajectories. Five serial chains constitute our approximation to the human skeleton. Revolute, universal and spherical joints are employed, although other topologies can be used in a similar fashion. Several real datasets are evaluated and results demonstrate that good accuracy can be obtained at interactive rates using the presented methodology. The results show that using simple serial chains in combination with dimensional synthesis suffices to generate the mechanical structure and trajectories of a humanoid robot or 3D avatar mimicking human motion

This paper presents a GPU based implementation of an imagebased rendering method for reducing the cost of stereoscopic rendering to the cost of rendering a single monoscopic image plus a smaller fixed cost. Our approach is to use the color and depth information from the rendered image of one eye to produce a reconstructed depthsprite which is rendered for the other eye. A GPU hardware accelerated technique for producing and rendering this depthsprite at rates above 60 Hz is presented. Our technique enables the real time stereoscopic display of complex and data intensive objects, which are currently constrained to monoscopic rendering technology.

Hazard detection and prevention of natural and man-made disasters at critical civil infrastructure is becoming increasingly important. Recent events, such as earthquakes and terrorist attacks, clearly demonstrated the societal and economical impact stemming from the limitations and uncertainty within currently deployed emergency response systems. We present a new data visualization and simulation platform that facilitates risk detection, emergency response and assessment in hazardous situations. The platform is based on the acquisition, modeling and analysis of sensor data, to capture objects and temporal changes in the observed spaces. Avatars are acquired, inserted and tracked in a virtual environment, enabling the simulation of multiple perilous situations and assisting in determining possible risk mitigation strategies. While the initial research focus is on algorithms and techniques in the field of hazard detection and prevention using path planning, the results can also be applied to diverse fields such as media contents development, cultural heritage, e-learning, teleconferencing, animation and video gaming.

A new hand- or finger-mounted data input device, called KITTY, is presented. KITTY is designed for keyboard independent touch typing and supports traditional touch-typing skills as a method for alphanumeric data input. This glove-type device provides an ultra-portable solution for ``quiet'' data input into portable computer systems and full freedom of movement in mobile VR and AR environments. The KITTY design follows the concept of the column and row layout found on traditional keyboards, allowing users to draw from existing touch typing skills, easing the required training time.

VizClass is a university classroom environment designed to offer students in computer graphics and engineering courses up-to-date visualization technologies. Three digital whiteboards and a three-dimensional stereoscopic display provide complementary display surfaces. Input devices include touchscreens on the digital whiteboards, remote keyboards, data gloves, and hand-position sensors.We use observations, interviews, and surveys to examine the pedagogical impacts of VizClass for teaching and learning computer graphics and virtual reality. Preliminary findings include positive student and teacher attitudes and greater learner engagement in after-class collaborations.

We present a method for the efficient construction, optimization, and storage of linear approximations to image data. A general simulated annealing-based optimization technique is adapted to the specific problem of optimizing image approximations. A linear approximation is defined by a triangulated subset of the source image pixels and the quality of an approximation may be judged with reference to a computed mean or RMS per-pixel difference between the source image and the approximation. A greedy algorithm for the construction of an initial configuration is described for which the spatial distribution of approximation points rejects the distribution of detail in the image. The approximation is optimized by a sequence of reconfiguration steps chosen according to a simulated annealing process. We outline an efficient implementation of the optimization procedure, explore fast error-recalculation methods, and describe an approximation-compression technique that allows efficient storing, transmission, and viewing of the final approximations. Test results are provided to illustrate the performance of the approximation construction pipeline.

Real-time visualization of particle traces in virtual environments can aid in the exploration and analysis of complex three dimensional vector fields. This paper introduces a scalable method suitable for the interactive visualization of large time-varying vector fields on commodity hardware. A real-time data streaming and visualization approach and its out-of-core scheme for the pre-processing and rendering of data are described. The presented approach yields low-latency application start-up times and small memory footprints. A proof of concept systems was implemented on a low-cost Linux workstation equipped with spatial tracking hardware, data gloves and shutter glasses. The system was used to implement a virtual wind tunnel in which a volumetric particle injector can introduce up to 60000 particles into the flow field while an interactive rendering performance of 60 frames per second is maintained.

A first step towards a semi-immersive virtual reality (VR) interface for finite element analysis (FEA) is presented in this paper. During recent years, user interfaces of FEA solvers have matured from character-based command-line driven implementations into easy-to-use graphical user interfaces (GUIs). This new generation of GUIs provides access to intuitive and productive tools for the management and analysis of structural problems. Many pre- and post-processors have been implemented targeting the simplification of the man-machine interface in order to increase the ease of use and provide better visual analysis of FEA solver results. Nevertheless, none of these packages provides a real 3D-enabled interface. The main objective of this project is to join state-of-the-art visualization technology, VR devices, and FEA solvers into the integrated development environment VRFEA.