The extended reality market has rapidly grown with a wide range of products for not only Virtual Reality applications, but also for advanced and multiple forms of Augmented Reality. In this paper we review the currently available complete solutions for Augmented Reality, divided into the primary display techniques used: Video See-through, using cameras to capture the real world subsequently presented with virtual overlays on a handheld or headworn display, Optical See-through, using semi-transparent display to allow real world view together with the virtual augmentations, and Projection-based AR or Spatial AR, the use of projectors to display augmentations directly on top of surfaces in the room. First potential products were found using popular Internet search engines, after which products that are not complete solutions or not commercially available were filtered out. We present the different products together with a description of their presented or studied characteristics, and of their accompanying software solutions.

With the emergence of multi-vehicular autonomous systems, such as AI controlled multiple fully autonomous vehicles, we need novel systems that provide tools for planning, executing, and reviewing of missions and keeping humans in the loop during all phases. We therefore present an immersive visualization system for interacting with these systems at a higher cognitive level than piloting of individual vehicles. Our system provides both desktop and VR modes for visual interaction with the robotic multi-vehicle AI system.

Results of planetary mapping are often shared openly for use in scientific research and mission planning. In its raw format, however, the data is not accessible to non-experts due to the difficulty in grasping the context and the intricate acquisition process. We present work on tailoring and integration of multiple data processing and visualization methods to interactively contextualize geospatial surface data of celestial bodies for use in science communication. As our approach handles dynamic data sources, streamed from online repositories, we are significantly shortening the time between discovery and dissemination of data and results. We describe the image acquisition pipeline, the pre-processing steps to derive a 2.5D terrain, and a chunked level-of-detail, out-of-core rendering approach to enable interactive exploration of global maps and high-resolution digital terrain models. The results are demonstrated for three different celestial bodies. The first case addresses high-resolution map data on the surface of Mars. A second case is showing dynamic processes, such as concurrent weather conditions on Earth that require temporal datasets. As a final example we use data from the New Horizons spacecraft which acquired images during a single flyby of Pluto. We visualize the acquisition process as well as the resulting surface data. Our work has been implemented in the OpenSpace software [8], which enables interactive presentations in a range of environments such as immersive dome theaters, interactive touch tables, and virtual reality headsets.

In this paper, we tackle the challenge of effective Transfer Function (TF) design for Direct Volume Rendering (DVR) of full-color datasets. We propose a novel TF design toolbox based on color similarity which is used to adjust opacity as well as replacing colors. We show that both CIE L*u*v* chromaticity and the chroma component of YCbCr are equally suited as underlying color space for the TF widgets. In order to maximize the area utilized in the TF editor, we renormalize the color space based on the histogram of the dataset. Thereby, colors representing a higher share of the dataset are depicted more prominently, thus providing a higher sensitivity for fine-tuning TF widgets. The applicability of our TF design toolbox is demonstrated by volume ray casting challenging full-color volume data including the visible male cryosection dataset and examples from 3D histology.

Abstract A central topic in scientific visualization is the transfer function (TF) for volume rendering. The TF serves a fundamental role in translating scalar and multivariate data into color and opacity to express and reveal the relevant features present in the data studied. Beyond this core functionality, TFs also serve as a tool for encoding and utilizing domain knowledge and as an expression for visual design of material appearances. TFs also enable interactive volumetric exploration of complex data. The purpose of this state‐of‐the‐art report (STAR) is to provide an overview of research into the various aspects of TFs, which lead to interpretation of the underlying data through the use of meaningful visual representations. The STAR classifies TF research into the following aspects: dimensionality, derived attributes, aggregated attributes, rendering aspects, automation, and user interfaces. The STAR concludes with some interesting research challenges that form the basis of an agenda for the development of next generation TF tools and methodologies.

We present an algorithm that enables real-time dynamic shading in direct volume rendering using general lighting, including directional lights, point lights, and environment maps. Real-time performance is achieved by encoding local and global volumetric visibility using spherical harmonic (SH) basis functions stored in an efficient multiresolution grid over the extent of the volume. Our method enables high-frequency shadows in the spatial domain, but is limited to a low-frequency approximation of visibility and illumination in the angular domain. In a first pass, level of detail (LOD) selection in the grid is based on the current transfer function setting. This enables rapid online computation and SH projection of the local spherical distribution of visibility information. Using a piecewise integration of the SH coefficients over the local regions, the global visibility within the volume is then computed. By representing the light sources using their SH projections, the integral over lighting, visibility, and isotropic phase functions can be efficiently computed during rendering. The utility of our method is demonstrated in several examples showing the generality and interactive performance of the approach.

This paper presents a novel technique to efficiently compute illumination for Direct Volume Rendering using a local approximation of ambient occlusion to integrate the intensity of incident light for each voxel. An advantage with this local approach is that fully shadowed regions are avoided, a desirable feature in many applications of volume rendering such as medical visualization. Additional transfer function interactions are also presented, for instance, to highlight specific structures with luminous tissue effects and create an improved context for semitransparent tissues with a separate absorption control for the illumination settings. Multiresolution volume management and GPU-based computation are used to accelerate the calculations and support large data sets. The scheme yields interactive frame rates with an adaptive sampling approach for incrementally refined illumination under arbitrary transfer function changes. The illumination effects can give a better understanding of the shape and density of tissues and so has the potential to increase the diagnostic value of medical volume rendering. Since the proposed method is gradient-free, it is especially beneficial at the borders of clip planes, where gradients are undefined, and for noisy data sets.

In many applications of Direct Volume Rendering (DVR) the importance of a certain material or feature is highly dependent on its relative spatial location. For instance, in the medical diagnostic procedure, the patient's symptoms often lead to specification of features, tissues and organs of particular interest. One such example is pockets of gas which, if found inside the body at abnormal locations, are a crucial part of a diagnostic visualization. This paper presents an approach that enhances DVR transfer function design with spatial localization based on user specified material dependencies. Semantic expressions are used to define conditions based on relations between different materials, such as only render iodine uptake when close to liver. The underlying methods rely on estimations of material distributions which are acquired by weighing local neighborhoods of the data against approximations of material likelihood functions. This information is encoded and used to influence rendering according to the user's specifications. The result is improved focus on important features by allowing the user to suppress spatially less-important data. In line with requirements from actual clinical DVR practice, the methods do not require explicit material segmentation that would be impossible or prohibitively time-consuming to achieve in most real cases. The scheme scales well to higher dimensions which accounts for multi-dimensional transfer functions and multivariate data. Dual-Energy Computed Tomography, an important new modality in radiology, is used to demonstrate this scalability. In several examples we show significantly improved focus on clinically important aspects in the rendered images.

Abstract Multiple‐volume visualization is a growing field in medical imaging providing simultaneous exploration of volumes acquired from varying modalities. However, high complexity results in an increased strain on performance compared to single volume rendering as scenes may consist of volumes with arbitrary orientations and rendering is performed with varying sample densities. Expensive image order techniques such as depth peeling have previously been used to perform the necessary calculations. In this work we present a view‐independentregion based scene descriptionfor multi‐volume pipelines. Using Binary Space Partitioning we are able to create a simple interface providing all required information for advanced multi‐volume renderings while introducing a minimal overhead for scenes with few volumes. The modularity of our solution is demonstrated by the use of visual development and performance is documented with benchmarks and real‐time simulations.

Volume raycasting techniques are important for both visual arts and visualization. They allow an efficient generation of visual effects and the visualization of scientific data obtained by tomography or numerical simulation. Thanks to their flexibility, experts agree that GPU-based raycasting is the state-of-the art technique for interactive volume rendering. It will most likely replace existing slice-based techniques in the near future. Volume rendering techniques are also effective for the direct rendering of implicit surfaces used for soft body animation and constructive solid geometry.

This paper introduces Temporal Density Parallel Coordinates (TDPC) and Depth Cue Parallel Coordinates (DCPC) which extend the standard 2D parallel coordinates technique to capture time-varying dynamics. The proposed techniques can be used to analyse temporal positions of data items as well as temporal positions of changes occurring using 2D displays. To represent temporal changes, polygons (instead of traditional lines) are rendered in parallel coordinates. The results presented show that rendering polygons is superior at revealing large temporal changes. Both TDPC and DCPC have been efficiently implemented on the GPU allowing the visualization of thousands of data items over thousands of time steps at interactive frame rates.

Direct Volume Rendering has proved to be an effective visualization method for medical data sets and has reached wide-spread clinical use. The diagnostic exploration, in essence, corresponds to a tissue classification task, which is often complex and time-consuming. Moreover, a major problem is the lack of information on the uncertainty of the classification, which can have dramatic consequences for the diagnosis. In this paper this problem is addressed by proposing animation methods to convey uncertainty in the rendering. The foundation is a probabilistic Transfer Function model which allows for direct user interaction with the classification. The rendering is animated by sampling the probability domain over time, which results in varying appearance for uncertain regions. A particularly promising application of this technique is a "sensitivity lens" applied to focus regions in the data set. The methods have been evaluated by radiologists in a study simulating the clinical task of stenosis assessment, in which the animation technique is shown to outperform traditional rendering in terms of assessment accuracy.

We present a direct interblock interpolation technique that enables direct volume rendering of blocked, multiresolution volumes. The proposed method smoothly interpolates between blocks of arbitrary block-wise level-of-detail (LOD) without sample replication or padding. This permits extreme changes in resolution across block boundaries and removes the interblock dependency for the LOD creation process. In addition the full data reduction from the LOD selection can be maintained throughout the rendering pipeline. Our rendering pipeline employs a flat block subdivision followed by a transfer function based adaptive LOD scheme. We demonstrate the effectiveness of our method by rendering volumes of the order of gigabytes using consumer graphics cards on desktop PC systems.

The high complexity of Transfer Function (TF) design is a major obstacle to widespread routine use of Direct Volume Rendering, particularly in the case of medical imaging. Both manual and automatic TF design schemes would benefit greatly from a fast and simple method for detection of tissue value ranges. To this end, we introduce the a-histogram, an enhancement that amplifies ranges corresponding to spatially coherent materials. The properties of the a-histogram have been explored for synthetic data sets and then successfully used to detect vessels in 20 Magnetic Resonance angiographies, proving the potential of this approach as a fast and simple technique for histogram enhancement in general and for TF construction in particular.

This paper presents a procedure for virtual autopsies based on interactive 3D visualizations of large scale, high resolution data from CT-scans of human cadavers. The procedure is described using examples from forensic medicine and the added value and future potential of virtual autopsies is shown from a medical and forensic perspective. Based on the technical demands of the procedure state-of-the-art volume rendering techniques are applied and refined to enable real-time, full body virtual autopsies involving gigabyte sized data on standard GPUs. The techniques applied include transfer function based data reduction using level-of-detail selection and multi-resolution rendering techniques. The paper also describes a data management component for large, out-of-core data sets and an extension to the GPU-based raycaster for efficient dual TF rendering. Detailed benchmarks of the pipeline are presented using data sets from forensic cases.

Direct Volume Rendering (DVR) is of increasing diagnostic value in the analysis of data sets captured using the latest medical imaging modalities. The deployment of DVR in everyday clinical work, however, has so far been limited. One contributing factor is that current Transfer Function (TF) models can encode only a small fraction of the user's domain knowledge. In this paper, we use histograms of local neighborhoods to capture tissue characteristics. This allows domain knowledge on spatial relations in the data set to be integrated into the TF. As a first example, we introduce Partial Range Histograms in an automatic tissue detection scheme and present its effectiveness in a clinical evaluation. We then use local histogram analysis to perform a classification where the tissue-type certainty is treated as a second TF dimension. The result is an enhanced rendering where tissues with overlapping intensity ranges can be discerned without requiring the user to explicitly define a complex, multidimensional TF.

Direct Volume Rendering (DVR) is known to be of diagnostic value in the analysis of medical data sets. However, its deployment in everyday clinical use has so far been limited. Two major challenges are that the current methods for Transfer Function (TF) construction are too complex and that the tissue separation abilities of the TF need to be extended. In this paper we propose the use of histogram analysis in local neighborhoods to address both these conflicting problems. To reduce TF construction difficulty, we introduce Partial Range Histograms in an automatic tissue detection scheme, which in connection with Adaptive Trapezoids enable efficient TF design. To separate tissues with overlapping intensity ranges, we propose a fuzzy classification based on local histograms as a second TF dimension. This increases the power of the TF, while retaining intuitive presentation and interaction.

The authors present a visualization system for interactive real time animation and visualization of simulation results from a parallel Particle-in-Cell code. The system was designed and implemented for the Onyx2 Infinite Reality hardware. A number of different visual objects, such as volume rendered particle density functionals were implemented. To provide sufficient frame rates for interactive visualization, the system was designed to provide performance close to the hardware specifications both in terms of the I/O and graphics subsystems. The presented case study applies the developed system to the evolution of an instability that gives rise to a plasma surfatron, a mechanism which rapidly can accelerate particles to very high velocities and thus be of great importance in the context of electron acceleration in astrophysical shocks, in the solar corona and in particle accelerators. The produced visualizations have allowed us to identify a previously unknown saturation mechanism for the surfatron and direct research efforts into new areas of interest.