Speaking aloud to a wearable AR assistant in public can be socially awkward, and re-articulating the same requests every day creates unnecessary effort. We present SpeechLess, a wearable AR assistant that introduces a speech-based intent granularity control paradigm grounded in personalized spatial memory. SpeechLess helps users "speak less," while still obtaining the information they need, and supports gradual explicitation of intent when more complex expression is required. SpeechLess binds prior interactions to multimodal personal context–space, time, activity, and referents–to form spatial memories, and leverages them to extrapolate missing intent dimensions from under-specified user queries. This enables users to dynamically adjust how explicitly they express their informational needs, from full-utterance to micro/zero-utterance interaction. We motivate our design through a week-long formative study using a commercial smart glasses platform, revealing discomfort with public voice use, frustration with repetitive speech, and hardware constraints. Building on these insights, we design SpeechLess, and evaluate it through controlled lab and in-the-wild studies. Our results indicate that regulated speech-based interaction, can improve everyday information access, reduce articulation effort, and support socially acceptable use without substantially degrading perceived usability or intent resolution accuracy across diverse everyday environments.

We introduce Speech-to-Spatial, a referent disambiguation framework that converts verbal remote-assistance instructions into spatially grounded AR guidance. Unlike prior systems that rely on additional cues (e.g., gesture, gaze) or manual expert annotations, Speech-to-Spatial infers the intended target solely from spoken references (speech input). Motivated by our formative study of speech referencing patterns, we characterize recurring ways people specify targets (Direct Attribute, Relational, Remembrance, and Chained) and ground them to our object-centric relational graph. Given an utterance, referent cues are parsed and rendered as persistent in-situ AR visual guidance, reducing iterative micro-guidance ("a bit more to the right", "now, stop.") during remote guidance. We demonstrate the use cases of our system with remote guided assistance and intent disambiguation scenarios. Our evaluation shows that Speech-to-Spatial improves task efficiency, reduces cognitive load, and enhances usability compared to a conventional voice-only baseline, transforming disembodied verbal instruction into visually explainable, actionable guidance on a live shared view.

We present VolMoVis, a method for dynamic tomographic reconstruction that supports real-time volume generation and volumetric motion visualization from 2D projections. Visualizing the motion of 3D anatomical structures, such as organs and tumors, is critical for computer-aided interventions. However, conventional 4D volumetric reconstruction methods typically produce a limited set of volumes at discrete phases, suffering from low temporal resolution. Moreover, it often requires extensive segmentation of 3D structures or regions for visualizing volumetric data, making it challenging to segment and visualize dynamic volumes in real-time. To address these challenges, VolMoVis framework employs a continuous implicit neural representation that decomposes the dynamic volumetric data into a static reference volume and a continuous deformation field. This decomposition, along with an efficient deformation network, enables our framework to achieve real-time volume generation and volumetric visualization of continuous anatomical motions. We evaluate VolMoVis on both 4D digital phantoms and real patient datasets, demonstrating its effectiveness for accurate anatomical reconstruction and motion tracking. Furthermore, we highlight its capabilities in real-time simultaneous volume generation and tumor segmentation for visualizing dynamic volumes and 4D tumor tracking, showcasing its potential in image-guided radiation therapy.

We present AuxiScope, a novel AR-based system designed to enhance personalized data exploration on large wall displays (LWDs) by integrating handheld tablets as auxiliary visualization interfaces. While LWDs offer expanded visual real estate and intuitive embodied interaction, they pose challenges related to effective interfaces for data exploration and analysis, specifically in multi-user settings. AuxiScope addresses these by overlaying supplementary visualizations onto corresponding LWD content, enabling individualized exploration without interference with the visual data displayed on the LWD. To achieve this, we have designed a geometric alignment pipeline that synchronizes the auxiliary visualizations atop the virtual scene. Specifically, by leveraging AR technology, AuxiScope resolves the tablet physical localization, viewpoint computation, and user interaction translation into the virtual space. Subsequently, based on a client-server architecture, it employs remote rendering and delegates computational tasks to the LWD compute nodes in order to minimize memory load on portable devices. We demonstrate the potential of AuxiScope through multiple AR-based interaction techniques across information and scientific visualization scenarios, for both 2D and 3D contexts.

We present the design and construction of the Silo, a fully immersive stereoscopic cylindrical tiled-display visualization facility. Comprising 168 high-density LCD displays, the facility provides an ultra-high-resolution image of 619 million pixels, and close to 360 horizontal field-of-regards (FoR), aiming to maximize visual acuity and completely engage the human visual sensorium and its periphery. In this article, we outline the motivations, design principles, hardware selection and software systems, and interaction modalities used in constructing the Silo. To address missing visual information due to the absence of a ceiling and floor, we have designed a method that utilizes conformal mapping and optimal mass transport to reproject the entire 360 volumetric FoR of the virtual scene to the available display real estate. We showcase several applications demonstrating the utility of the Silo and report the findings of our user studies that highlight the effectiveness of the Silo layout compared to curved mono and flat powerwall display facilities. Our user evaluations and studies have shown that the Silo supports natural exploration and enhanced visualization due to its capability to render surround ultra-high-resolution stereoscopic views.

Understanding visual attention is key to designing efficient human-computer interaction, especially for virtual reality (VR) and augmented reality (AR) applications. However, the relationship between 3D spatial attributes of visual stimuli and visual attention is still underexplored. Thus, we design an experiment to collect a gaze dataset in VR, and use it to quantitatively model the probability of first attention between two stimuli. First, we construct the dataset by presenting subjects with a synthetic VR scene containing varying spatial configurations of two spheres. Second, we formulate their selective attention based on a probability model that takes as input two view-specific stimuli attributes: their eccentricities in the field of view and their sizes as visual angles. Third, we train two models using our gaze dataset to predict the probability distribution of a user's preferences of visual stimuli within the scene. We evaluate our method by comparing model performance across two challenging synthetic scenes in VR. Our application case study demonstrates that VR designers can utilize our models for attention prediction in two-foreground-object scenarios, which are common when designing 3D content for storytelling or scene guidance. We make the dataset and the source code to visualize it available alongside this work.

We present Explainable XR, an end-to-end framework for analyzing user behavior in diverse eXtended Reality (XR) environments by leveraging Large Language Models (LLMs) for data interpretation assistance. Existing XR user analytics frameworks face challenges in handling cross-virtuality - AR, VR, MR - transitions, multi-user collaborative application scenarios, and the complexity of multimodal data. Explainable XR addresses these challenges by providing a virtuality-agnostic solution for the collection, analysis, and visualization of immersive sessions. We propose three main components in our framework: (1) A novel user data recording schema, called User Action Descriptor (UAD), that can capture the users' multimodal actions, along with their intents and the contexts; (2) a platform-agnostic XR session recorder, and (3) a visual analytics interface that offers LLM-assisted insights tailored to the analysts' perspectives, facilitating the exploration and analysis of the recorded XR session data. We demonstrate the versatility of Explainable XR by demonstrating five use-case scenarios, in both individual and collaborative XR applications across virtualities. Our technical evaluation and user studies show that Explainable XR provides a highly usable analytics solution for understanding user actions and delivering multifaceted, actionable insights into user behaviors in immersive environments.

Multi-view diffusion models, obtained by applying Su-pervised Finetuning (SFT) to text-to-image diffusion mod-els, have driven recent breakthroughs in text-to-3D re-search. However, due to the limited size and quality of ex-isting 3D datasets, they still suffer from multi-view incon-sistencies and Neural Radiance Field (NeRF) reconstruction artifacts. We argue that multi-view diffusion models can benefit from further Reinforcement Learning Finetuning (RLFT), which allows models to learn from the data generated by themselves and improve beyond their dataset limitations during SFT. To this end, we introduce Carve3D, an improved RLFT algorithm coupled with a novel Multi-view Reconstruction Consistency (MRC) metric, to enhance the consistency of multi-view diffusion models. To mea-sure the MRC metric on a set of multi-view images, we compare them with their corresponding NeRF renderings at the same camera viewpoints. The resulting model, which we denote as Carve3DM, demonstrates superior multi-view consistency and NeRF reconstruction quality than existing models. Our results suggest that pairing SFT with Carve3D's RLFT is essential for developing multi-view-consistent diffusion models, mirroring the standard Large Language Model (LLM) alignment pipeline. Our code, training and testing data, and video results are available at: https://desaixie.github.io/carve-3d.

We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is automatically synthesized from simple primitive shapes with random texturing and augmentations (e.g., height fields, boolean differences, and wireframes). Unlike previous 3D datasets (e.g., Objaverse) which are often captured or crafted by humans to approximate real 3D data, Zeroverse completely ignores realistic global semantics but is rich in complex geometric and texture details that are locally similar to or even more intricate than real objects. We demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse, can achieve high visual quality in the reconstruction of real-world objects, competitive with models trained on Objaverse. We also analyze several critical design choices of Zeroverse that contribute to LRM-Zero's capability and training stability. Our work demonstrates that 3D reconstruction, one of the core tasks in 3D vision, can potentially be addressed without the semantics of real-world objects. The Zeroverse's procedural synthesis code and interactive visualization are available at: https://desaixie.github.io/lrm-zero/.

We present VoxAR, a method to facilitate an effective visualization of volume-rendered objects in optical see-through head-mounted displays (OST-HMDs). The potential of augmented reality (AR) to integrate digital information into the physical world provides new opportunities for visualizing and interpreting scientific data. However, a limitation of OST-HMD technology is that rendered pixels of a virtual object can interfere with the colors of the real-world, making it challenging to perceive the augmented virtual information accurately. We address this challenge in a two-step approach. First, VoxAR determines an appropriate placement of the volume-rendered object in the real-world scene by evaluating a set of spatial and environmental objectives, managed as user-selected preferences and pre-defined constraints. We achieve a real-time solution by implementing the objectives using a GPU shader language. Next, VoxAR adjusts the colors of the input transfer function (TF) based on the real-world placement region. Specifically, we introduce a novel optimization method that adjusts the TF colors such that the resulting volume-rendered pixels are discernible against the background and the TF maintains the perceptual mapping between the colors and data intensity values. Finally, we present an assessment of our approach through objective evaluations and subjective user studies.

We present Submerse, an end-to-end framework for visualizing flooding scenarios on large and immersive display ecologies. Specifically, we reconstruct a surface mesh from input flood simulation data and generate a to-scale 3D virtual scene by incorporating geographical data such as terrain, textures, buildings, and additional scene objects. To optimize computation and memory performance for large simulation datasets, we discretize the data on an adaptive grid using dynamic quadtrees and support level-of-detail based rendering. Moreover, to provide a perception of flooding direction for a time instance, we animate the surface mesh by synthesizing water waves. As interaction is key for effective decision-making and analysis, we introduce two novel techniques for flood visualization in immersive systems: (1) an automatic scene-navigation method using optimal camera viewpoints generated for marked points-of-interest based on the display layout, and (2) an AR-based focus+context technique using an aux display system. Submerse is developed in collaboration between computer scientists and atmospheric scientists. We evaluate the effectiveness of our system and application by conducting workshops with emergency managers, domain experts, and concerned stakeholders in the Stony Brook Reality Deck, an immersive gigapixel facility, to visualize a superstorm flooding scenario in New York City.

Placing and orienting a camera to compose aesthetically meaningful shots of a scene is not only a key objective in real-world photography and cinematography but also for virtual content creation. The framing of a camera often significantly contributes to the story telling in movies, games, and mixed reality applications. Generating single camera poses or even contiguous trajectories either requires a significant amount of manual labor or requires solving high-dimensional optimization problems, which can be computationally demanding and error-prone. In this paper, we introduce GAIT, a framework for training a Deep Reinforcement Learning (DRL) agent, that learns to automatically control a camera to generate a sequence of aesthetically meaningful views for synthetic 3D indoor scenes. To generate sequences of frames with high aesthetic value, GAIT relies on a neural aesthetics estimator, which is trained on a crowed-sourced dataset. Additionally, we introduce regularization techniques for diversity and smoothness to generate visually interesting trajectories for a 3D environment, and to constrain agent acceleration in the reward function to generate a smooth sequence of camera frames. We validated our method by comparing it to baseline algorithms, based on a perceptual user study, and through ablation studies. Code and visual results are available on the project website: https://desaixie.github.io/gait-rl

Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections over time is limited to observations gathered using population analysis. In this article, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject across specified age-timepoints. To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (GAN) that translates features of neuronal structures across age-timepoints for large brain microscopy volumes. We improve the reconstruction quality of the predicted neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss. Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. Finally, to visualize the change in projections, predicted using neuReGANerator, NeuRegenerate offers two modes: (i) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, from two age domains (using structural view and bounded view), and (ii) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes within the cholinergic system of the mouse brain between a young and old specimen.

The growing complexity of spatial and structural information in 3D data makes data inspection and visualization a challenging task. We describe a method to create a planar embedding of 3D treelike structures using their skeleton representations. Our method maintains the original geometry, without overlaps, to the best extent possible, allowing exploration of the topology within a single view. We present a novel camera view generation method which maximizes the visible geometric attributes (segment shape and relative placement between segments). Camera views are created for individual segments and are used to determine local bending angles at each node by projecting them to 2D. The final embedding is generated by minimizing an energy function (the weights of which are user adjustable) based on branch length and the 2D angles, while avoiding intersections. The user can also interactively modify segment placement within the 2D embedding, and the overall embedding will update accordingly. A global to local interactive exploration is provided using hierarchical camera views that are created for subtrees within the structure. We evaluate our method both qualitatively and quantitatively and demonstrate our results by constructing planar visualizations of line data (traced neurons) and volume data (CT vascular and bronchial data).

The MD-Cave is an immersive analytics system that provides enhanced stereoscopic visualizations to support visual diagnoses performed by radiologists. The system harnesses contemporary paradigms in immersive visualization and 3D interaction, which are better suited for investigating 3D volumetric data. We retain practicality through efficient utilization of desk space and comfort for radiologists in terms of frequent long duration use. MD-Cave is general and incorporates: (1) high resolution stereoscopic visualizations through a surround triple-monitor setup, (2) 3D interactions through head and hand tracking, (3) and a general framework that supports 3D visualization of deep-seated anatomical structures without the need for explicit segmentation algorithms. Such a general framework expands the utility of our system to many diagnostic scenarios. We have developed MD-Cave through close collaboration and feedback from two expert radiologists who evaluated the utility of MD-Cave and the 3D interactions in the context of radiological examinations. We also provide evaluation of MD-Cave through case studies performed by an expert radiologist and concrete examples on multiple real-world diagnostic scenarios, such as pancreatic cancer, shoulder-CT, and COVID-19 Chest CT examination.

We present a novel approach for volume exploration that is versatile yet effective in isolating semantic structures in both noisy and clean data. Specifically, we describe a hierarchical active contours approach based on Bhattacharyya gradient flow which is easier to control, robust to noise, and can incorporate various types of statistical information to drive an edge-agnostic exploration process. To facilitate a time-bound user-driven volume exploration process that is applicable to a wide variety of data sources, we present an efficient multi-GPU implementation that (1) is approximately 400 times faster than a single thread CPU implementation, (2) allows hierarchical exploration of 2D and 3D images, (3) supports customization through multidimensional attribute spaces, and (4) is applicable to a variety of data sources and semantic structures. The exploration system follows a 2-step process. It first applies active contours to isolate semantically meaningful subsets of the volume. It then applies transfer functions to the isolated regions locally to produce clear and clutter-free visualizations. We show the effectiveness of our approach in isolating and visualizing structures-of-interest without needing any specialized segmentation methods on a variety of data sources, including 3D optical microscopy, multi-channel optical volumes, abdominal and chest CT, micro-CT, MRI, simulation, and synthetic data. We also gathered feedback from a medical trainee regarding the usefulness of our approach and discussion on potential applications in clinical workflows.

Abstract The field of connectomics aims to reconstruct the wiring diagram of Neurons and synapses to enable new insights into the workings of the brain. Reconstructing and analyzing the Neuronal connectivity, however, relies on many individual steps, starting from high‐resolution data acquisition to automated segmentation, proofreading, interactive data exploration, and circuit analysis. All of these steps have to handle large and complex datasets and rely on or benefit from integrated visualization methods. In this state‐of‐the‐art report, we describe visualization methods that can be applied throughout the connectomics pipeline, from data acquisition to circuit analysis. We first define the different steps of the pipeline and focus on how visualization is currently integrated into these steps. We also survey open science initiatives in connectomics, including usable open‐source tools and publicly available datasets. Finally, we discuss open challenges and possible future directions of this exciting research field.

We introduce NeuroConstruct, a novel end-to-end application for the segmentation, registration, and visualization of brain volumes imaged using wide-field microscopy. NeuroConstruct offers a Segmentation Toolbox with various annotation helper functions that aid experts to effectively and precisely annotate micrometer resolution neurites. It also offers an automatic neurites segmentation using convolutional neuronal networks (CNN) trained by the Toolbox annotations and somas segmentation using thresholding. To visualize neurites in a given volume, NeuroConstruct offers a hybrid rendering by combining iso-surface rendering of high-confidence classified neurites, along with real-time rendering of raw volume using a 2D transfer function for voxel classification score versus voxel intensity value. For a complete reconstruction of the 3D neurites, we introduce a Registration Toolbox that provides automatic coarse-to-fine alignment of serially sectioned samples. The quantitative and qualitative analysis show that NeuroConstruct outperforms the state-of-the-art in all design aspects. NeuroConstruct was developed as a collaboration between computer scientists and neuroscientists, with an application to the study of cholinergic neurons, which are severely affected in Alzheimer's disease.

We present 3D virtual pancreatography (VP), a novel visualization procedure and application for non-invasive diagnosis and classification of pancreatic lesions, the precursors of pancreatic cancer. Currently, non-invasive screening of patients is performed through visual inspection of 2D axis-aligned CT images, though the relevant features are often not clearly visible nor automatically detected. VP is an end-to-end visual diagnosis system that includes: A machine learning based automatic segmentation of the pancreatic gland and the lesions, a semi-automatic approach to extract the primary pancreatic duct, a machine learning based automatic classification of lesions into four prominent types, and specialized 3D and 2D exploratory visualizations of the pancreas, lesions and surrounding anatomy. We combine volume rendering with pancreas- and lesion-centric visualizations and measurements for effective diagnosis. We designed VP through close collaboration and feedback from expert radiologists, and evaluated it on multiple real-world CT datasets with various pancreatic lesions and case studies examined by the expert radiologists.

Significant work has been done towards deep learning (DL) models for automatic lung and lesion segmentation and classification of COVID-19 on chest CT data. However, comprehensive visualization systems focused on supporting the dual visual+DL diagnosis of COVID-19 are non-existent. We presentCOVID-view, a visualization application specially tailored for radiologists to diagnose COVID-19 from chest CT data. The system incorporates a complete pipeline of automatic lungs segmentation, localization/isolation of lung abnormalities, followed by visualization, visual and DL analysis, and measurement/quantification tools. Our system combines the traditional 2D workflow of radiologists with newer 2D and 3D visualization techniques with DL support for a more comprehensive diagnosis.COVID-viewincorporates a novel DL model for classifying the patients into positive/negative COVID-19 cases, which acts as a reading aid for the radiologist usingCOVID-viewand provides the attention heatmap as an explainable DL for the model output. We designed and evaluatedCOVID-viewthrough suggestions, close feedback and conducting case studies of real-world patient data by expert radiologists who have substantial experience diagnosing chest CT scans for COVID-19, pulmonary embolism, and other forms of lung infections. We present requirements and task analysis for the diagnosis of COVID-19 that motivate our design choices and results in a practical system which is capable of handling real-world patient cases.

Machine learning is a powerful and effective tool for medical image analysis to perform computer-aided diagnosis (CAD). Having great potential in improving the accuracy of a diagnosis, CAD systems are often analyzed in terms of the final accuracy, leading to a limited understanding of the internal decision process, impossibility to gain insights, and ultimately to skepticism from clinicians. We present a visual analytics approach to uncover the decision-making process of a CAD system for classifying pancreatic cystic lesions. This CAD algorithm consists of two distinct components: random forest (RF), which classifies a set of predefined features, including demographic features, and a convolutional neural network (CNN), which analyzes radiological (imaging) features of the lesions. We study the class probabilities generated by the RF and the semantical meaning of the features learned by the CNN. We also use an eye tracker to better understand which radiological features are particularly useful for a radiologist to make a diagnosis and to quantitatively compare with the features that lead the CNN to its final classification decision. Additionally, we evaluate the effects and benefits of supplying the CAD system with a case-based visual aid in a second-reader setting.

We present a visual analytics framework, CMed, for exploring medical image data annotations acquired from crowdsourcing. CMed can be used to visualize, classify, and filter crowdsourced clinical data based on a number of different metrics such as detection rate, logged events, and clustering of the annotations. CMed provides several interactive linked visualization components to analyze the crowd annotation results for a particular video and the associated workers. Additionally, all results of an individual worker can be inspected using multiple linked views in our CMed framework. We allow a crowdsourcing application analyst to observe patterns and gather insights into the crowdsourced medical data, helping him/her design future crowdsourcing applications for optimal output from the workers. We demonstrate the efficacy of our framework with two medical crowdsourcing studies: polyp detection in virtual colonoscopy videos and lung nodule detection in CT thin-slab maximum intensity projection videos. We also provide experts' feedback to show the effectiveness of our framework. Lastly, we share the lessons we learned from our framework with suggestions for integrating our framework into a clinical workflow.

Colorectal cancer screening modalities, such as optical colonoscopy (OC) and virtual colonoscopy (VC), are critical for diagnosing and ultimately removing polyps (precursors of colon cancer). The non-invasive VC is normally used to inspect a 3D reconstructed colon (from CT scans) for polyps and if found, the OC procedure is performed to physically traverse the colon via endoscope and remove these polyps. In this paper, we present a deep learning framework, Extended and Directional CycleGAN, for lossy unpaired image-to-image translation between OC and VC to augment OC video sequences with scale-consistent depth information from VC, and augment VC with patient-specific textures, color and specular highlights from OC (e.g, for realistic polyp synthesis). Both OC and VC contain structural information, but it is obscured in OC by additional patient-specific texture and specular highlights, hence making the translation from OC to VC lossy. The existing CycleGAN approaches do not handle lossy transformations. To address this shortcoming, we introduce an extended cycle consistency loss, which compares the geometric structures from OC in the VC domain. This loss removes the need for the CycleGAN to embed OC information in the VC domain. To handle a stronger removal of the textures and lighting, a Directional Discriminator is introduced to differentiate the direction of translation (by creating paired information for the discriminator), as opposed to the standard CycleGAN which is direction-agnostic. Combining the extended cycle consistency loss and the Directional Discriminator, we show state-of-the-art results on scale-consistent depth inference for phantom, textured VC and for real polyp and normal colon video sequences. We also present results for realistic pendunculated and flat polyp synthesis from bumps introduced in 3D VC models.

We introduce a transfer-function-guided three-dimensional (3-D) block-based saliency-aware compression scheme for volumetric data that is both content and spatially scalable. Salient 3-D volumetric blocks are identified and weighted with the help of a transfer function which is used to render the data. We describe our method in the form of a framework for processing, progressive transmission, and visualization of volumetric data on a target device, such as a mobile device with limited computational resources. In particular, we address the transmission bottleneck incurred when transferring 3-D volumetric data. Identified salient regions are progressively transmitted to the target device. The received data are rendered progressively in the respective order with a predefined or user-defined transfer function. Our method is developed with medical applications in mind, where preservation of all information is essential for clinical diagnosis. Because our method is integrated into a resolution scalable coding scheme with an integer wavelet transform of the image, it allows the rendering of each significant region at a different resolution up to fully lossless reconstruction. We perform a thorough qualitative and quantitative evaluation of the saliency detection method and the resulting saliency-aware compression schemes. Our results show reduced error in representation of the volumetric data with our method.

Multi-class segmentation has recently achieved significant performance in natural images and videos. This achievement is due primarily to the public availability of large multi-class datasets. However, there are certain domains, such as biomedical images, where obtaining sufficient multi-class annotations is a laborious and often impossible task and only single-class datasets are available. While existing segmentation research in such domains use private multi-class datasets or focus on single-class segmentations, we propose a unified highly efficient framework for robust simultaneous learning of multi-class segmentations by combining single-class datasets and utilizing a novel way of conditioning a convolutional network for the purpose of segmentation. We demonstrate various ways of incorporating the conditional information, perform an extensive evaluation, and show compelling multi-class segmentation performance on biomedical images, which outperforms current state-of-the-art solutions (up to 2.7%). Unlike current solutions, which are meticulously tailored for particular single-class datasets, we utilize datasets from a variety of sources. Furthermore, we show the applicability of our method also to natural images and evaluate it on the Cityscapes dataset. We further discuss other possible applications of our proposed framework.

Virtual-reality provides an immersive environment but can induce cybersickness due to the discrepancy between visual and vestibular cues. To avoid this problem, the movement of the virtual camera needs to match the motion of the user in the real world. Unfortunately, this is usually difficult due to the mismatch between the size of the virtual environments and the space available to the users in the physical domain. The resulting constraints on the camera movement significantly hamper the adoption of virtual-reality headsets in many scenarios and make the design of the virtual environments very challenging. In this work, we study how the characteristics of the virtual camera movement (e.g., translational acceleration and rotational velocity) and the composition of the virtual environment (e.g., scene depth) contribute to perceived discomfort. Based on the results from our user experiments, we devise a computational model for predicting the magnitude of the discomfort for a given scene and camera trajectory. We further apply our model to a new path planning method which optimizes the input motion trajectory to reduce perceptual sickness. We evaluate the effectiveness of our method in improving perceptual comfort in a series of user studies targeting different applications. The results indicate that our method can reduce the perceived discomfort while maintaining the fidelity of the original navigation, and perform better than simpler alternatives.

Wide-field microscopes are commonly used in neurobiology for experimental studies of brain samples. Available visualization tools are limited to electron, two-photon, and confocal microscopy datasets, and current volume rendering techniques do not yield effective results when used with wide-field data. We present a workflow for the visualization of neuronal structures in wide-field microscopy images of brain samples. We introduce a novel gradient-based distance transform that overcomes the out-of-focus blur caused by the inherent design of wide-field microscopes. This is followed by the extraction of the 3D structure of neurites using a multi-scale curvilinear filter and cell-bodies using a Hessian-based enhancement filter. The response from these filters is then applied as an opacity map to the raw data. Based on the visualization challenges faced by domain experts, our workflow provides multiple rendering modes to enable qualitative analysis of neuronal structures, which includes separation of cell-bodies from neurites and an intensity-based classification of the structures. Additionally, we evaluate our visualization results against both a standard image processing deconvolution technique and a confocal microscopy image of the same specimen. We show that our method is significantly faster and requires less computational resources, while producing high quality visualizations. We deploy our workflow in an immersive gigapixel facility as a paradigm for the processing and visualization of large, high-resolution, wide-field microscopy brain datasets.

We present a volume exploration framework, FeatureLego, that uses a novel voxel clustering approach for efficient selection of semantic features. We partition the input volume into a set of compact super-voxels that represent the finest selection granularity. We then perform an exhaustive clustering of these super-voxels using a graph-based clustering method. Unlike the prevalent brute-force parameter sampling approaches, we propose an efficient algorithm to perform this exhaustive clustering. By computing an exhaustive set of clusters, we aim to capture as many boundaries as possible and ensure that the user has sufficient options for efficiently selecting semantically relevant features. Furthermore, we merge all the computed clusters into a single tree of meta-clusters that can be used for hierarchical exploration. We implement an intuitive user-interface to interactively explore volumes using our clustering approach. Finally, we show the effectiveness of our framework on multiple real-world datasets of different modalities.

Virtual colonoscopy (VC) is a non-invasive screening tool for colorectal polyps which employs volume visualization of a colon model reconstructed from a CT scan of the patient's abdomen. We present an immersive analytics system for VC which enhances and improves the traditional desktop VC through the use of VR technologies. Our system, using a head-mounted display (HMD), includes all of the standard VC features, such as the volume rendered endoluminal fly-through, measurement tool, bookmark modes, electronic biopsy, and slice views. The use of VR immersion, stereo, and wider field of view and field of regard has a positive effect on polyp search and analysis tasks in our immersive VC system, a volumetric-based immersive analytics application. Navigation includes enhanced automatic speed and direction controls, based on the user's head orientation, in conjunction with physical navigation for exploration of local proximity. In order to accommodate the resolution and frame rate requirements for HMDs, new rendering techniques have been developed, including mesh-assisted volume raycasting and a novel lighting paradigm. Feedback and further suggestions from expert radiologists show the promise of our system for immersive analysis for VC and encourage new avenues for exploring the use of VR in visualization systems for medical diagnosis.

In order to improve the perception of the core part of a virtual soft object when a user squeezes it with his/her bare hand, we apply multisensory integration of visual and auditory stimuli to our proposed AR system. The visual stimuli are real-time stereoscopic images in which the user's bare hand is squeezing the virtual soft object in the actual scene. As the auditory stimuli, we focus on the sound volume change linked to the movements of the user's thumb and index finger. The present paper reports the effects of sound volume change in enhancing the pseudo-softness and the perception of the core part of a virtual soft object. The experimental results of our study statistically show that when a user squeezes a virtual soft object with his/her bare hand, multisensory integration of the visual and auditory stimuli effectively increases the feeling of grasping and facilitates handling of the virtual soft object. In addition, auditory stimuli that have a clearly audible sound volume change at the core part naturally enhance the perception of the core part of the virtual soft object.

Redirected walking techniques can enhance the immersion and visual-vestibular comfort of virtual reality (VR) navigation, but are often limited by the size, shape, and content of the physical environments. We propose a redirected walking technique that can apply to small physical environments with static or dynamic obstacles. Via a head- and eye-tracking VR headset, our method detects saccadic suppression and redirects the users during the resulting temporary blindness. Our dynamic path planning runs in real-time on a GPU, and thus can avoid static and dynamic obstacles, including walls, furniture, and other VR users sharing the same physical space. To further enhance saccadic redirection, we propose subtle gaze direction methods tailored for VR perception. We demonstrate that saccades can significantly increase the rotation gains during redirection without introducing visual distortions or simulator sickness. This allows our method to apply to large open virtual spaces and small physical environments for room-scale VR. We evaluate our system via numerical simulations and real user studies.

Visualization of medical organs and biological structures is a challenging task because of their complex geometry and the resultant occlusions. Global spherical and planar mapping techniques simplify the complex geometry and resolve the occlusions to aid in visualization. However, while resolving the occlusions these techniques do not preserve the geometric context, making them less suitable for mission-critical biomedical visualization tasks. In this paper, we present a shape-preserving local mapping technique for resolving occlusions locally while preserving the overall geometric context. More specifically, we present a novel visualization algorithm, LMap, for conformally parameterizing and deforming a selected local region-of-interest (ROI) on an arbitrary surface. The resultant shape-preserving local mappings help to visualize complex surfaces while preserving the overall geometric context. The algorithm is based on the robust and efficient extrinsic Ricci flow technique, and uses the dynamic Ricci flow algorithm to guarantee the existence of a local map for a selected ROI on an arbitrary surface. We show the effectiveness and efficacy of our method in three challenging use cases: (1) multimodal brain visualization, (2) optimal coverage of virtual colonoscopy centerline flythrough, and (3) molecular surface visualization.

We introduce a novel approach for flame volume reconstruction from videos using inexpensive charge-coupled device (CCD) consumer cameras. The approach includes an economical data capture technique using inexpensive CCD cameras. Leveraging the smear feature of the CCD chip, we present a technique for synchronizing CCD cameras while capturing flame videos from different views. Our reconstruction is based on the radiative transport equation which enables complex phenomena such as emission, extinction, and scattering to be used in the rendering process. Both the color intensity and temperature reconstructions are implemented using the CUDA parallel computing framework, which provides real-time performance and allows visualization of reconstruction results after every iteration. We present the results of our approach using real captured data and physically-based simulated data. Finally, we also compare our approach against the other state-of-the-art flame volume reconstruction methods and demonstrate the efficacy and efficiency of our approach in four different applications: (1) rendering of reconstructed flames in virtual environments, (2) rendering of reconstructed flames in augmented reality, (3) flame stylization, and (4) reconstruction of other semitransparent phenomena.

For many virtual reality applications, a pre-calculated fly-through path is the de facto standard navigation method. Such a path is convenient for users and ensures coverage of critical areas throughout the scene. Traditional applications use constant camera speed, allow for fully user-controlled manual speed adjustment, or use automatic speed adjustment based on heuristics from the scene. We introduce two novel methods for constrained path navigation and exploration in virtual environments which rely on the natural orientation of the user's head during scene exploration. Utilizing head tracking to obtain the user's area of focus, we perform automatic camera speed adjustment to allow for natural off-axis scene examination. We expand this to include automatic camera navigation along the pre-computed path, abrogating the need for any navigational inputs from the user. Our techniques are applicable for any scene with a pre-computed navigation path, including medical applications such as virtual colonoscopy, coronary fly-through, or virtual angioscopy, and graph navigation. We compare the traditional methods (constant speed and manual speed adjustment) and our two methods (automatic speed adjustment and automatic speed/direction control) to determine the effect of speed adjustment on system usability, mental load, performance, and user accuracy. Through this evaluation we observe the effect of automatic speed adjustment compared to traditional methods. We observed no negative impact from automatic navigation, and the users performed as well as with the manual navigation.

A variety of applications such as virtual reality and immersive cinema require high image quality, low rendering latency, and consistent depth cues. 4D light field displays support focus accommodation, but are more costly to render than 2D images, resulting in higher latency. The human visual system can resolve higher spatial frequencies in the fovea than in the periphery. This property has been harnessed by recent 2D foveated rendering methods to reduce computation cost while maintaining perceptual quality. Inspired by this, we present foveated 4D light fields by investigating their effects on 3D depth perception. Based on our psychophysical experiments and theoretical analysis on visual and display bandwidths, we formulate a content-adaptive importance model in the 4D ray space. We verify our method by building a prototype light field display that can render only 16% -- 30% rays without compromising perceptual quality.

We present a novel visualization framework, AnaFe, targeted at observing changes in the spleen over time through multiple image-derived features. Accurate monitoring of progressive changes is crucial for diseases that result in enlargement of the organ. Our system is comprised of multiple linked views combining visualization of temporal 3D organ data, related measurements, and features. Thus it enables the observation of progression and allows for simultaneous comparison within and between the subjects. AnaFe offers insights into the overall distribution of robustly extracted and reproducible quantitative imaging features and their changes within the population, and also enables detailed analysis of individual cases. It performs similarity comparison of temporal series of one subject to all other series in both sick and healthy groups. We demonstrate our system through two use case scenarios on a population of 189 spleen datasets from 68 subjects with various conditions observed over time.

We present a method for registration and visualization of corresponding supine and prone virtual colonoscopy scans based on eigenfunction analysis and fold modeling. In virtual colonoscopy, CT scans are acquired with the patient in two positions, and their registration is desirable so that physicians can corroborate findings between scans. Our algorithm performs this registration efficiently through the use of Fiedler vector representation (the second eigenfunction of the Laplace-Beltrami operator). This representation is employed to first perform global registration of the two colon positions. The registration is then locally refined using the haustral folds, which are automatically segmented using the 3D level sets of the Fiedler vector. The use of Fiedler vectors and the segmented folds presents a precise way of visualizing corresponding regions across datasets and visual modalities. We present multiple methods of visualizing the results, including 2D flattened rendering and the corresponding 3D endoluminal views. The precise fold modeling is used to automatically find a suitable cut for the 2D flattening, which provides a less distorted visualization. Our approach is robust, and we demonstrate its efficiency and efficacy by showing matched views on both the 2D flattened colons and in the 3D endoluminal view. We analytically evaluate the results by measuring the distance between features on the registered colons, and we also assess our fold segmentation against 20 manually labeled datasets. We have compared our results analytically to previous methods, and have found our method to achieve superior results. We also prove the hot spots conjecture for modeling cylindrical topology using Fiedler vector representation, which allows our approach to be used for general cylindrical geometry modeling and feature extraction.

This work presents a novel framework for spherical mesh parameterization. An efficient angle-preserving spherical parameterization algorithm is introduced, which is based on dynamic Yamabe flow and the conformal welding method with solid theoretic foundation. An area-preserving spherical parameterization is also discussed, which is based on discrete optimal mass transport theory. Furthermore, a spherical parameterization algorithm, which is based on the polar decomposition method, balancing angle distortion and area distortion is presented. The algorithms are tested on 3D geometric data and the experiments demonstrate the efficiency and efficacy of the proposed methods.

Real walking offers higher immersive presence for virtual reality (VR) applications than alternative locomotive means such as walking-in-place and external control gadgets, but needs to take into consideration different room sizes, wall shapes, and surrounding objects in the virtual and real worlds. Despite perceptual study of impossible spaces and redirected walking, there are no general methods to match a given pair of virtual and real scenes. We propose a system to match a given pair of virtual and physical worlds for immersive VR navigation. We first compute a planar map between the virtual and physical floor plans that minimizes angular and distal distortions while conforming to the virtual environment goals and physical environment constraints. Our key idea is to design maps that are globally surjective to allow proper folding of large virtual scenes into smaller real scenes but locally injective to avoid locomotion ambiguity and intersecting virtual objects. From these maps we derive altered rendering to guide user navigation within the physical environment while retaining visual fidelity to the virtual environment. Our key idea is to properly warp the virtual world appearance into real world geometry with sufficient quality and performance. We evaluate our method through a formative user study, and demonstrate applications in gaming, architecture walkthrough, and medical imaging.

We present a novel method to create planar visualizations of treelike structures (e.g., blood vessels and airway trees) where the shape of the object is well preserved, allowing for easy recognition by users familiar with the structures. Based on the extracted skeleton within the treelike object, a radial planar embedding is first obtained such that there are no self-intersections of the skeleton which would have resulted in occlusions in the final view. An optimization procedure which adjusts the angular positions of the skeleton nodes is then used to reconstruct the shape as closely as possible to the original, according to a specified view plane, which thus preserves the global geometric context of the object. Using this shape recovered embedded skeleton, the object surface is then flattened to the plane without occlusions using harmonic mapping. The boundary of the mesh is adjusted during the flattening step to account for regions where the mesh is stretched over concavities. This parameterized surface can then be used either as a map for guidance during endoluminal navigation or directly for interrogation and decision making. Depth cues are provided with a grayscale border to aid in shape understanding. Examples are presented using bronchial trees, cranial and lower limb blood vessels, and upper aorta datasets, and the results are evaluated quantitatively and with a user study.

Empirical, hypothesis-driven, experimentation is at the heart of the scientific discovery process and has become commonplace in human-factors related fields. To enable the integration of visual analytics in such experiments, we introduce VEEVVIE, the Visual Explorer for Empirical Visualization, VR and Interaction Experiments. VEEVVIE is comprised of a back-end ontology which can model several experimental designs encountered in these fields. This formalization allows VEEVVIE to capture experimental data in a query-able form and makes it accessible through a front-end interface. This front-end offers several multi-dimensional visualization widgets with built-in filtering and highlighting functionality. VEEVVIE is also expandable to support custom experimental measurements and data types through a plug-in visualization widget architecture. We demonstrate VEEVVIE through several case studies of visual analysis, performed on the design and data collected during an experiment on the scalability of high-resolution, immersive, tiled-display walls.

We have developed a novel visualization system based on the reconstruction of high resolution and high frame rate images from a multi-tiered stream of samples that are rendered framelessly. This decoupling of the rendering system from the display system is particularly suitable when dealing with very high resolution displays or expensive rendering algorithms, where the latency of generating complete frames may be prohibitively high for interactive applications. In contrast to the traditional frameless rendering technique, we generate the lowest latency samples on the optimal sampling lattice in the 3D domain. This approach avoids many of the artifacts associated with existing sample caching and reprojection methods during interaction that may not be acceptable in many visualization applications. Advanced visualization effects are generated remotely and streamed into the reconstruction system using tiered samples with varying latencies and quality levels. We demonstrate the use of our visualization system for the exploration of volumetric data at stable guaranteed frame rates on high resolution displays, including a 470 megapixel tiled display as part of the Reality Deck immersive visualization facility.

We present the results of a variable information space experiment, targeted at exploring the scalability limits of immersive highresolution, tiled-display walls under physical navigation. Our work is motivated by a lack of evidence supporting the extension of previously established benefits on substantially large, room-shaped displays. Using the Reality Deck, a gigapixel resolution immersive display, as its apparatus, our study spans four display form-factors, starting at 100 megapixels arranged planarly and up to one gi-gapixel in a horizontally immersive setting. We focus on four core tasks: visual search, attribute search, comparisons and pattern finding. We present a quantitative analysis of per-task user performance across the various display conditions. Our results demonstrate improvements in user performance as the display form-factor changes to 600 megapixels. At the 600 megapixel to 1 gigapixel transition, we observe no tangible performance improvements and the visual search task regressed substantially. Additionally, our analysis of subjective mental effort questionnaire responses indicates that subjective user effort grows as the display size increases, validating previous studies on smaller displays. Our analysis of the participants' physical navigation during the study sessions shows an increase in user movement as the display grew. Finally, by visualizing the participants' movement within the display apparatus space, we discover two main approaches (termed “overview” and “detail”) through which users chose to tackle the various data exploration tasks. The results of our study can inform the design of immersive high-resolution display systems and provide insight into how users navigate within these room-sized visualization spaces.

We introduce a semi-automatic lighting design method that deploys per-voxel accessory lights (fill and detail lights) to enhance local shapes, as well as to increase the perceptibility and visual saliency of an object. Our approach allows the user to manually design arbitrary lights in a scene for creating the desired feeling of emotion. The user designed lights are used as key lights and our approach automatically configures per-voxel accessory lights that preserve the user designed feeling of emotion. Per-voxel fill lights brighten the shadows and thus increase the perceptibility and visual saliency. Per-voxel detail lights enhance the visual cues for the local shape perception. Moreover, the revealed local shapes are controlled by the user employing an importance distribution. Similarly, the perceptibility and visual saliency are also controlled based on an importance distribution. Our perceptual measurement guarantees that the revealed local shapes are independent of the key lights. In addition, our method provides two control parameters, which adjust the fill and detail lights, to provide the user with additional flexibility in designing the expected lighting effect. The major contributions of this paper are the idea of using the importance distribution to control local shapes, the per-voxel accessory lights and the perceptual measurement.

We introduce the concept of the infinite canvas as a metaphor for the immersive visual exploration of very large image datasets using a natural walking interface. The interface allows the user to move along the display surface and to be continuously exposed to new data, essentially exploring the horizontal axis of an arbitrarily long canvas. Our system provides a spiral navigation interface that shows a compressed immersive overview of the data and facilitates the rapid and fluid transition to distant points within the infinite canvas. We demonstrate the implementation of the infinite canvas in the world's first 1.5 billion pixel tiled immersive display.

We introduce a lighting system that enhances the visual cues in a rendered image for the perception of 3D volumetric objects. We divide the lighting effects into global and local effects, and deploy three types of directional lights: the key light and accessory lights (fill and detail lights). The key light provides both lighting effects and carries the visual cues for the perception of local and global shapes and depth. The cues for local shapes are conveyed by gradient; those for global shapes are carried by shadows; and those for depth are provided by shadows and translucent objects. Fill lights produce global effects to increase the perceptibility. Detail lights generate local effects to improve the cues for local shapes. Our method quantifies the perception and uses an exhaustive search to set the lights. It configures accessory lights with the consideration of preserving the global impression conveyed by the key light. It ensures the feeling of smooth light movements in animations. With simplification, it achieves interactive frame rates and produces results that are visually indistinguishable from results using the nonsimplified algorithm. The major contributions of this paper are our lighting system, perception measurement and lighting design algorithm with our indistinguishable simplification.

We propose a new colon flattening algorithm that is efficient, shape-preserving, and robust to topological noise. Unlike previous approaches, which require a mandatory topological denoising to remove fake handles, our algorithm directly flattens the colon surface without any denoising. In our method, we replace the original Euclidean metric of the colon surface with a heat diffusion metric that is insensitive to topological noise. Using this heat diffusion metric, we then solve a Laplacian equation followed by an integration step to compute the final flattening. We demonstrate that our method is shape-preserving and the shape of the polyps are well preserved. The flattened colon also provides an efficient way to enhance the navigation and inspection in virtual colonoscopy. We further show how the existing colon registration pipeline is made more robust by using our colon flattening. We have tested our method on several colon wall surfaces and the experimental results demonstrate the robustness and the efficiency of our method.

We present a framework for acuity-driven visualization of super-high resolution image data on gigapixel displays. Tiled display walls offer a large workspace that can be navigated physically by the user. Based on head tracking information, the physical characteristics of the tiled display and the formulation of visual acuity, we guide an out-of-core gigapixel rendering scheme by delivering high levels of detail only in places where it is perceivable to the user. We apply this principle to gigapixel image rendering through adaptive level of detail selection. Additionally, we have developed an acuity-driven tessellation scheme for high-quality Focus-and-Context (F+C) lenses that significantly reduces visual artifacts while accurately capturing the underlying lens function. We demonstrate this framework on the Reality Deck, an immersive gigapixel display. We present the results of a user study designed to quantify the impact of our acuity-driven rendering optimizations in the visual exploration process. We discovered no evidence suggesting a difference in search task performance between our framework and naive rendering of gigapixel resolution data, while realizing significant benefits in terms of data transfer overhead. Additionally, we show that our acuity-driven tessellation scheme offers substantially increased frame rates when compared to naive pre-tessellation, while providing indistinguishable image quality.

We present a novel area-preservation mapping/flattening method using the optimal mass transport technique, based on the Monge-Brenier theory. Our optimal transport map approach is rigorous and solid in theory, efficient and parallel in computation, yet general for various applications. By comparison with the conventional Monge-Kantorovich approach, our method reduces the number of variables from O(n2) to O(n), and converts the optimal mass transport problem to a convex optimization problem, which can now be efficiently carried out by Newton's method. Furthermore, our framework includes the area weighting strategy that enables users to completely control and adjust the size of areas everywhere in an accurate and quantitative way. Our method significantly reduces the complexity of the problem, and improves the efficiency, flexibility and scalability during visualization. Our framework, by combining conformal mapping and optimal mass transport mapping, serves as a powerful tool for a broad range of applications in visualization and graphics, especially for medical imaging. We provide a variety of experimental results to demonstrate the efficiency, robustness and efficacy of our novel framework.

We introduce NuNav3D, a body-driven 3D navigation interface for large displays and immersive scenarios. While 3D navigation is a core component of VR applications, certain situations, like remote displays in public or large visualization environments, do not allow for using a navigation controller or prop. NuNav3D maps hand motions, obtained from a pose recognition framework which is driven by a depth sensor, to a virtual camera manipulator, allowing for direct control of 4 DOFs of navigation. We present the NuNav3D navigation scheme and our preliminary user study results under two scenarios, a path-following case with tight geometrical constraints and an open space exploration case, while comparing our method against a traditional joypad controller.

Abstract This paper introduces an automatic color design method that is driven by an importance function of the objects within a volumetric dataset. Our method allows the user to intuitively modify the object classification and the importance distribution function in the 2D rendered image. It automatically computes the transfer function, especially the color distribution, to convey the importance of the objects. In our approach, the importance of an object is represented as the attentiveness of a color. In addition, we preserve the color harmony in the rendered image in order to provide a visually pleasing result. In this paper, we propose a set of computational measurements to compute the color attentiveness and color harmony. Our color assignment algorithm supports arbitrary‐dimensional transfer functions and obtains interactive frame rates. Our method involves three color spaces, namely Coloroid system, CIE LChuv, and Adobe RGB color space. It calculates the color attentiveness in CIE LChuvspace, and the color harmony in Coloroid system. It, then, assigns the transfer function in a dual space of Adobe RGB space and renders the resulting image in Adobe RGB space. We conducted a detailed user study, which proves that our method successfully conveys the importance distributions. Our contribution in this paper is not only our importance driven approach, but also our computational measurements and our color assignment algorithm.

We introduce a simple, yet powerful method called the Cumulative Heat Diffusion for shape-based volume analysis, while drastically reducing the computational cost compared to conventional heat diffusion. Unlike the conventional heat diffusion process, where the diffusion is carried out by considering each node separately as the source, we simultaneously consider all the voxels as sources and carry out the diffusion, hence the term cumulative heat diffusion. In addition, we introduce a new operator that is used in the evaluation of cumulative heat diffusion called the Volume Gradient Operator (VGO). VGO is a combination of the LBO and a data-driven operator which is a function of the half gradient. The half gradient is the absolute value of the difference between the voxel intensities. The VGO by its definition captures the local shape information and is used to assign the initial heat values. Furthermore, VGO is also used as the weighting parameter for the heat diffusion process. We demonstrate that our approach can robustly extract shape-based features and thus forms the basis for an improved classification and exploration of features based on shape.

In Virtual Reality, immersive systems such as the CAVE provide an important tool for the collaborative exploration of large 3D data. Unlike head-mounted displays, these systems are often only partially immersive due to space, access, or cost constraints. The resulting loss of visual information becomes a major obstacle for critical tasks that need to utilize the users' entire field of vision. We have developed a conformal visualization technique that establishes a conformal mapping between the full 360° field of view and the display geometry of a given visualization system. The mapping is provably angle-preserving and has the desirable property of preserving shapes locally, which is important for identifying shape-based features in the visual data. We apply the conformal visualization to both forward and backward rendering pipelines in a variety of retargeting scenarios, including CAVEs and angled arrangements of flat panel displays. In contrast to image-based retargeting approaches, our technique constructs accurate stereoscopic images that are free of resampling artifacts. Our user study shows that on the visual polyp detection task in Immersive Virtual Colonoscopy, conformal visualization leads to improved sensitivity at comparable examination times against the traditional rendering approach. We also develop a novel user interface based on the interactive recreation of the conformal mapping and the real-time regeneration of the view direction correspondence.

We introduce a modified dendrogram (MD) (with subtrees to represent clusters) and display it in 2D for multidimensional transfer function design. Such a transfer function for direct volume rendering employs a multidimensional space, termed attribute space. The MD reveals the hierarchical structure information of the attribute space. The user can design a transfer function in an intuitive and informative manner using the MD user interface in 2D instead of multidimensional space, where it is hard to ascertain the relationship of the space. In addition, we provide the capability to interactively modify the granularity of the MD. The coarse-grained MD primarily shows the global information of the attribute space while the fine-grained MD reveals the finer details, and the separation ability of the attribute space is completely preserved in the finest granularity. With this so called multigrained method, the user can efficiently create a transfer function using the coarse-grained MD, and then fine tune it with the fine-grained MDs. Our method is independent on the type of the attributes and supports arbitrary-dimension attribute space.

We present the conformal magnifier, a novel interactive focus+context visualization technique that magnifies a region of interest (ROI) using conformal mapping. Our framework supports the arbitrary shape design of magnifiers for the user to enlarge the ROI while globally deforming the context region without any cropping. By using the mathematically well-defined conformal mapping theory and algorithm, the ROI is magnified with local shape preservation (angle distortion minimization), while the transition area between the focus and context regions is deformed smoothly and continuously. After the selection of a specified magnifier shape, our system can automatically magnify the ROI in real time with full resolution even for large volumetric data sets. These properties are important for many visualization applications, especially for the computer aided detection and diagnosis (CAD). Our framework is suitable for diverse applications, including the map visualization, and volumetric visualization. Experimental results demonstrate the effectiveness, robustness, and efficiency of our framework.

Scientists, engineers and physicians are now confronted with a fire hose of data. Immersive visualization environments provide these users with a novel way of interacting and reasoning with large datasets. They allow them to utilize the entirety of their visual bandwidth, effectively engulfing the user in the data and enabling collaborative interaction. We present a custom-built 5-wall Cave environment, called the Immersive Cabin (IC). It is driven by a GPU cluster for both computation and 3D stereo rendering. We also propose a conformal deformation rendering pipeline for the visualization of datasets on partially-immersive platforms. Combined with a range of interaction and navigation tools, our system can support numerous interactive applications of large datasets. Several demonstrations include architectural visualization, urban planning, medical visualization, simulation and rendering of physical phenomena, and entertainment. Current visualization displays, however, have not kept up with the explosive growth in data size and resolution, which is beginning to match the resolution of the visuals that surround us in daily life. To ameliorate this challenge, we have developed a life-like, realistic immersion into the petascale data to be explored, appropriately called The RealityDeck. It is a one-of-a kind pioneering G-pixel immersive and collaborative display system - a unique assembly of high-res display panels, GPU cluster, sensors, networking, computer vision, and human-computer interaction technologies.

Current immersive VR systems such as the CAVE provide an effective platform for the immersive exploration of large 3D data. A major limitation is that in most cases at least one display surface is missing due to space, access or cost constraints. This partially-immersive visualization results in a substantial loss of visual information that may be acceptable for some applications, however it becomes a major obstacle for critical tasks, such as the analysis of medical data. We propose a conformal deformation rendering pipeline for the visualization of datasets on partially-immersive platforms. The angle-preserving conformal mapping approach is used to map the 360°3D view volume to arbitrary display configurations. It has the desirable property of preserving shapes under distortion, which is important for identifying features, especially in medical data. The conformal mapping is used for rasterization, realtime raytracing and volume rendering of the datasets. Since the technique is applied during the rendering, we can construct stereoscopic images from the data, which is usually not true for image-based distortion approaches. We demonstrate the stereo conformal mapping rendering pipeline in the partially-immersive 5-wall Immersive Cabin (IC) for virtual colonoscopy and architectural review.

Abstract Prostate cancer is one of the most prevalent cancers among males, and the use of magnetic resonance imaging (MRI) has been suggested for its detection. A framework is presented for scoring and visualizing various MR data in an efficient and intuitive manner. A classification method is introduced where a cumulative score volume is created which takes into account each of three acquisition types. This score volume is integrated into a volume rendering framework which allows the user to view the prostate gland, the multi‐modal score values, and the surrounding anatomy. A visibility persistence mode is introduced to automatically avoid full occlusion of a selected score and indicate overlaps. The use of GPU‐accelerated multi‐modal single‐pass ray casting provides an interactive experience. User driven importance rendering allows the user to gain insight into the data and can assist in localization of the disease and treatment planning. We evaluate our results against pathology and radiologists'determinations.

When visualizing tubular 3D structures, external representations are often used for guidance and display, and such views in 2D can often contain occlusions. Virtual dissection methods have been proposed where the entire 3D structure can be mapped to the 2D plane, though these will lose context by straightening curved sections. We present a new method of creating maps of 3D tubular structures that yield a succinct view while preserving the overall geometric structure. Given a dominant view plane for the structure, its curve skeleton is first projected to a 2D skeleton. This 2D skeleton is adjusted to account for distortions in length, modified to remove intersections, and optimized to preserve the shape of the original 3D skeleton. Based on this shaped 2D skeleton, a boundary for the map of the object is obtained based on a slicing path through the structure and the radius around the skeleton. The sliced structure is conformally mapped to a rectangle and then deformed via harmonic mapping to match the boundary placement. This flattened map preserves the general geometric context of a 3D object in a 2D display, and rendering of this flattened map can be accomplished using volumetric ray casting. We have evaluated our method on real datasets of human colon models.

In virtual colonoscopy, CT scans are typically acquired with the patient in both supine (facing up) and prone (facing down) positions. The registration of these two scans is desirable so that the user can clarify situations or confirm polyp findings at a location in one scan with the same location in the other, thereby improving polyp detection rates and reducing false positives. However, this supine-prone registration is challenging because of the substantial distortions in the colon shape due to the patient's change in position. We present an efficient algorithm and framework for performing this registration through the use of conformal geometry to guarantee that the registration is a diffeomorphism (a one-to-one and onto mapping). The taeniae coli and colon flexures are automatically extracted for each supine and prone surface, employing the colon geometry. The two colon surfaces are then divided into several segments using the flexures, and each segment is cut along a taenia coli and conformally flattened to the rectangular domain using holomorphic differentials. The mean curvature is color encoded as texture images, from which feature points are automatically detected using graph cut segmentation, mathematic morphological operations, and principal component analysis. Corresponding feature points are found between supine and prone and are used to adjust the conformal flattening to be quasi-conformal, such that the features become aligned. We present multiple methods of visualizing our results, including 2D flattened rendering, corresponding 3D endoluminal views, and rendering of distortion measurements. We demonstrate the efficiency and efficacy of our registration method by illustrating matched views on both the 2D flattened colon images and in the 3D volume rendered colon endoluminal view. We analytically evaluate the correctness of the results by measuring the distance between features on the registered colons.

Abstract Ray directed volume‐rendering algorithms are well suited for parallel implementation in a distributed cluster environment. For distributed ray casting, the scene must be partitioned between nodes for good load balancing, and a strict view‐dependent priority order is required for image composition. In this paper, we define the load balanced network distribution (LBND) problem and map it to the NP‐complete precedence constrained job‐shop scheduling problem. We introduce a kd‐tree solution and a dynamic programming solution. To process a massive data set, either a parallel or an out‐of‐core approach is required. Parallel preprocessing is performed by render nodes on data, which are allocated using a static data structure. Volumetric data sets often contain a large portion of voxels that will never be rendered, or empty space. Parallel preprocessing fails to take advantage of this. Our slab‐projection slice,introduced in this paper, tracks empty space across consecutive slices of data to reduce the amount of data distributed and rendered. It is used to facilitate out‐of‐core bricking and kd‐tree partitioning. Load balancing using each of our approaches is compared with traditional methods using several segmented regions of the Visible Korean data set.

An editorail on the 15th Anniversary of the IEEE Transactions on Visualization and Computer Graphics.

The Lattice Boltzmann method (LBM) for visual simulation of fluid flow generally employs cubic Cartesian (CC) lattices such as the D3Q13 and D3Q19 lattices for the particle transport. However, the CC lattices lead to suboptimal representation of the simulation space. We introduce the face-centered cubic (FCC) lattice, fD3Q13, for LBM simulations. Compared to the CC lattices, the fD3Q13 lattice creates a more isotropic sampling of the simulation domain and its single lattice speed (i.e., link length) simplifies the computations and data storage. Furthermore, the fD3Q13 lattice can be decomposed into two independent interleaved lattices, one of which can be discarded, which doubles the simulation speed. The resulting LBM simulation can be efficiently mapped to the GPU, further increasing the computational performance. We show the numerical advantages of the FCC lattice on channeled flow in 2D and the flow-past-a-sphere benchmark in 3D. In both cases, the comparison is against the corresponding CC lattices using the analytical solutions for the systems as well as velocity field visualizations. We also demonstrate the performance advantages of the fD3Q13 lattice for interactive simulation and rendering of hot smoke in an urban environment using thermal LBM.

Abstract Due to its high performance/cost ratio, a GPU cluster is an attractive platform for large scale general‐purpose computation and visualization applications. However, the programming model for high performance general‐purpose computation on GPU clusters remains a complex problem. In this paper, we introduce the Zippy frame‐work, a general and scalable solution to this problem. It abstracts the GPU cluster programming with a two‐level parallelism hierarchy and a non‐uniform memory access (NUMA) model. Zippy preserves the advantages of both message passing and shared‐memory models. It employs global arrays (GA) to simplify the communication, synchronization, and collaboration among multiple GPUs. Moreover, it exposes data locality to the programmer for optimal performance and scalability. We present three example applications developed with Zippy: sort‐last volume rendering, Marching Cubes isosurface extraction and rendering, and lattice Boltzmann flow simulation with online visualization. They demonstrate that Zippy can ease the development and integration of parallel visualization, graphics, and computation modules on a GPU cluster.

We simulate 3D fluid flow by a locally-refined lattice Boltzmann method (LBM) on graphics hardware. A low resolution LBM simulation running on a coarse grid models global flow behavior of the entire domain with low consumption of computational resources. For regions of interest where small visual details are desired, LBM simulations are performed on fine grids, which are separate grids superposed on the coarse one. The flow properties on boundaries of the fine grids are determined by the global simulation on the coarse grid. Thus, the locally refined fine-grid simulations follow the global fluid behavior, and model the desired small-scale and turbulent flow motion with their denser numerical discretization. A fine grid can be initiated and terminated at any time while the global simulation is running. It can also move inside the domain with a moving object to capture small-scale vortices caused by the object. Besides the performance improvement due to the adaptive simulation, the locally-refined LBM is suitable for acceleration on contemporary graphics hardware (GPU), since it involves only local and linear computations. Therefore, our approach achieves fast and adaptive 3D flow simulation for computer games and other interactive applications.

We describe a novel volumetric global illumination framework based on the Face-Centered Cubic (FCC) lattice. An FCC lattice has important advantages over a Cartesian lattice. It has higher packing density in the frequency domain, which translates to better sampling efficiency. Furthermore, it has the maximal possible kissing number (equivalent to the number of nearest neighbors of each site), which provides optimal 3D angular discretization among all lattices. We employ a new two-pass (illumination and rendering) global illumination scheme on an FCC lattice. This scheme exploits the angular discretization to greatly simplify the computation in multiple scattering and to minimize illumination information storage. The GPU has been utilized to further accelerate the rendering stage. We demonstrate our new framework with participating media and volume rendering with multiple scattering, where both are significantly faster than traditional techniques with comparable quality.

We provide a physically-based framework for simulating the natural phenomena related to heat interaction between objects and the surrounding air. We introduce a heat transfer model between the heat source objects and the ambient flow environment, which includes conduction, convection, and radiation. The heat distribution of the objects is represented by a novel temperature texture. We simulate the thermal flow dynamics that models the air flow interacting with the heat by a hybrid thermal lattice Boltzmann model (HTLBM). The computational approach couples a multiple-relaxation-time LBM (MRTLBM) with a finite difference discretization of a standard advection-diffusion equation for temperature. In heat shimmering and mirage, the changes in the index of refraction of the surrounding air are attributed to temperature variation. A nonlinear ray tracing method is used for rendering. Interactive performance is achieved by accelerating the computation of both the MRTLBM and the heat transfer, as well as the rendering on contemporary graphics hardware (GPU).

We present an efficient colon flattening algorithm using a conformal structure, which is angle-preserving and minimizes the global distortion. Moreover, our algorithm is general as it can handle high genus surfaces. First, the colon wall is segmented and extracted from the CT data set of the abdomen. The topology noise (i.e., minute handle) is located and removed automatically. The holomorphic 1-form, a pair of orthogonal vector fields, is then computed on the 3D colon surface mesh using the conjugate gradient method. The colon surface is cut along a vertical trajectory traced using the holomorphic 1-form. Consequently, the 3D colon surface is conformally mapped to a 2D rectangle. The flattened 2D mesh is then rendered using a direct volume rendering method accelerated with the GPU. Our algorithm is tested with a number of CT data sets of real pathological cases, and gives consistent results. We demonstrate that the shape of the polyps is well preserved on the flattened colon images, which provides an efficient way to enhance the navigation of a virtual colonoscopy system.

Radial Basis Functions (RBFs) have become a popular rendering primitive, both in surface and in volume rendering. This paper focuses on volume visualization, giving rise to 3D kernels. RBFs are especially convenient for the representation of scattered and irregularly distributed point samples, where the RBF kernel is used as a blending function for the space in between samples. Common representations employ radially symmetric RBFs, and various techniques have been introduced to render these, also with efficient implementations on programmable graphics hardware (GPUs). In this paper, we extend the existing work to more generalized, ellipsoidal RBF kernels, for the rendering of scattered volume data. We devise a post-shaded kernel-centric rendering approach, specifically designed to run efficiently on GPUs, and we demonstrate our renderer using datasets from subdivision volumes and computational science.

We present a novel pipeline for computer-aided detection (CAD) of colonic polyps by integrating texture and shape analysis with volume rendering and conformal colon flattening. Using our automatic method, the 3D polyp detection problem is converted into a 2D pattern recognition problem. The colon surface is first segmented and extracted from the CT data set of the patient's abdomen, which is then mapped to a 2D rectangle using conformal mapping. This flattened image is rendered using a direct volume rendering technique with a translucent electronic biopsy transfer function. The polyps are detected by a 2D clustering method on the flattened image. The false positives are further reduced by analyzing the volumetric shape and texture features. Compared with shape based methods, our method is much more efficient without the need of computing curvature and other shape parameters for the whole colon surface. The final detection results are stored in the 2D image, which can be easily incorporated into a virtual colonoscopy (VC) system to highlight the polyp locations. The extracted colon surface mesh can be used to accelerate the volumetric ray casting algorithm used to generate the VC endoscopic view. The proposed automatic CAD pipeline is incorporated into an interactive VC system, with a goal of helping radiologists detect polyps faster and with higher accuracy.

We present an efficient point-based isosurface exploration system with high quality rendering. Our system incorporates two point-based isosurface extraction and visualization methods: edge splatting and the edge kernel method. In a volume, two neighboring voxels define an edge. The intersection points between the active edges and the isosurface are used for exact isosurface representation. The point generation is incorporated in the GPU-based hardware-accelerated rendering, thus avoiding any overhead when changing the isovalue in the exploration. We call this method edge splatting. In order to generate high quality isosurface rendering regardless of the volume resolution and the view, we introduce an edge kernel method. The edge kernel upsamples the isosurface by subdividing every active cell of the volume data. Enough sample points are generated to preserve the exact shape of the isosurface defined by the trilinear interpolation of the volume data. By employing these two methods, we can achieve interactive isosurface exploration with high quality rendering.

We describe the rendering of massive volumes on a volume visualization cluster. We present our data distribution scheme and introduce an algorithm which reduces the memory requirement with no loss of accuracy. The volume is automatically cropped and partitioned into small volume blocks. The bounding boxes of these volume blocks are used at run-time for flexible partitioning of the volume across the network. We present results of rendering the full visible male color dataset, seismic data, and several large micro-CT scanned fossil and teeth datasets.

We propose point with implicit connectivity (PIC) as a new data structure for representing solid objects using points. In the PIC representation, an object is adaptively sampled into an octree, where each leaf cell contains at most one surface component of the object. Each surface component is represented by a vertex, together with inside/outside classification values of the cell corners. PIC objects are compact, feature-preserving, and supports easy construction of the boundary surfaces. To convert geometric objects into the PIC representation, we propose a sampling algorithm and use quadric error functions as error metrics. For CSG operations between PIC objects, we present a feature-preserving, adaptive CSG algorithm on the octrees. Our experiments show promising results for PIC objects with sharp features and large flat regions.

We introduce the notion of oriented charges to compute distance fields from unorganized point clouds. Unlike traditional implicit function methods that usually require normal information, our approach only uses information about the position of the samples. Our approach is adaptive, using an octree to reorganize the input points, thus avoiding difficulties often associated with the use of local distance fields. The resulting representation is simple, efficient and general in the sense that it handles complex models of arbitrary topology, is robust in the presence of noise and fills holes automatically.

We present a visualization and modeling system for the creation of chemical gardens. A chemical garden is a collection of plant-like structures formed when placing a soluble metal salt in an aqueous solution of sodium silicate. An accurate simulation of the chemical reaction of the metal salt and the physical behavior of the surrounding sodium silicate solution flow are complicated and time-consuming. Instead, in our system we use and propose a simplified 3D cellular automata (CA) approach to model the growing process of a chemical garden and its interaction with the solution flow. The input of our system is the shape and types of various soluble metal salts. Our program supports the visualization of the growth and formation of the chemical garden. The interactive rendering result on a texture mapping hardware also helps the user in observing and creating a desirable chemical garden.

We present a method for simulating front propagation over 3D surfaces with arbitrary topology. Our approach, which can handle the simultaneous spreading of multiple fronts, is implemented by applying a level set method on the surface. A special data structure, called the surrounding grid, assists in advancing the fronts without triangulation or parameterization of the surface. The surrounding grid consists of discretized grid points in the vicinity of the surface. An enhanced distance field provides the bridge between the surrounding grid and the surfaces on which the fronts travel. By manipulating speed functions in the level set method, the front propagation dynamics can be conveniently controlled. We illustrate the applicability and flexibility of our method with three example applications: texture generation, water flooding, and fire propagation.

This paper presents a novel approach for surface reconstruction from point clouds. The proposed technique is general in the sense that it naturally handles both manifold and non-manifold surfaces, providing a consistent way for reconstructing closed surfaces as well as surfaces with boundaries. It is also robust in the presence of noise, irregular sampling and surface gaps. Furthermore, it is fast, parallelizable and easy to implement because it is based on simple local operations. In this approach, surface reconstruction consists of three major steps: first, the space containing the point cloud is subdivided, creating a voxel representation. Then, a voxel surface is computed using gap filling and topological thinning operations. Finally, the resulting voxel surface is converted into a polygonal mesh. We demonstrate the effectiveness of our approach by reconstructing polygonal models from range scans of real objects as well as from synthetic data.

The size and resolution of volume datasets in science and medicine are increasing at a rate much greater than the resolution of the screens used to view them. This limits the amount of data that can be viewed simultaneously, potentially leading to a loss of overall context of the data when the user views or zooms into a particular area of interest. We propose a focus+context framework that uses various standard and advanced magnification lens rendering techniques to magnify the features of interest, while compressing the remaining volume regions without clipping them away completely. Some of these lenses can be interactively configured by the user to specify the desired magnification patterns, while others are feature-adaptive. All our lenses are accelerated on the GPU. They allow the user to interactively manage the available screen area, dedicating more area to the more resolution-important features.

This work presents a powerful variant of the ICP (iterative closest point) algorithm for registering range images using a probability field. The probability field (p-field) represents the probability distribution of the surface position. By capitalizing on the properties of the range image, fast construction, compact representation, and efficient query of the p-field can be achieved. Different sensor models are supported by the p-field according to the properties of the range image. Range images can be precisely aligned by maximizing the probability of overlapping surfaces via the p-field.

Inspired by the attractive Flops/dollar ratio and the incredible growth in the speed of modern graphics processing units (GPUs), we propose to use a cluster of GPUs for high performance scientific computing. As an example application, we have developed a parallel flow simulation using the lattice Boltzmann model (LBM) on a GPU cluster and have simulated the dispersion of airborne contaminants in the Times Square area of New York City. Using 30 GPU nodes, our simulation can compute a 480x400x80 LBM in 0.31 second/step, a speed which is 4.6 times faster than that of our CPU cluster implementation. Besides the LBM, we also discuss other potential applications of the GPU cluster, such as cellular automata, PDE solvers, and FEM.

We present a space leaping technique for accelerating volume rendering with very low space and run-time complexity. Our technique exploits the ray coherence during ray casting by using the distance a ray traverses in empty space to leap its neighboring rays. Our technique works with parallel as well as perspective volume rendering, does not require any preprocessing or 3D data structures, and is independent of the transfer function. Being an image-space technique, it is independent of the complexity of the data being rendered. It can be used to accelerate both time-coherent and noncoherent animation sequences.

We present a system for simulating and visualizing the propagation of dispersive contaminants with an application to urban security. In particular, we simulate airborne contaminant propagation in open environments characterised by sky-scrapers and deep urban canyons. Our approach is based on the multiple relaxation time lattice Boltzmann model (MRTLBM), which can efficiently handle complex boundary conditions such as buildings. In addition, we model thermal effects on the flow field using the hybrid thermal MRTLBM. Our approach can also accommodate readings from various sensors distributed in the environment and adapt the simulation accordingly. We accelerate the computation and efficiently render many buildings with small textures on the GPU. We render streamlines and the contaminant smoke with self-shadowing composited with the textured buildings.

We present a fast, topology-preserving approach for isosurface simplification. The underlying concept behind our approach is to preserve the disconnected surface components in cells during isosurface simplification. We represent isosurface components in a novel representation, called enhanced cell, where each surface component in a cell is represented by a vertex and its connectivity information. A topology-preserving vertex clustering algorithm is applied to build a vertex octree. An enhanced dual contouring algorithm is applied to extract error-bounded multiresolution isosurfaces from the vertex octree while preserving the finest resolution isosurface topology. Cells containing multiple vertices are properly handled during contouring. Our approach demonstrates better results than existing octree-based simplification techniques.

Abstract We present three methods for texturing point models from sample textures. The first method, the point parameterization method, uses a fast distortion‐bounded parameterization algorithm to flatten the point model's surface into one or more 2D patches. The sample texture is mapped onto these patches and alpha blending is used to minimize the discontinuity in the gaps between the patches. The second method is based on neighborhood matching where a color is assigned to each point by searching the best match within an irregular neighborhood. The hybrid method combines the former two methods, capitalizing on the advantages of both. The point parameterization method is used first to color most of the points, and the point neighborhood‐matching method is then applied to the points belonging to the gaps between the parameterized patches to minimize the discontinuity. We opt for fast texture generation, while some discontinuities may appear in the gaps of anisotropic textures.

We study texture projection based on a four region subdivision: magnification, minification, and two mixed regions. We propose improved versions of existing techniques by providing exact filtering methods which reduce both aliasing and overblurring, especially in the mixed regions. We further present a novel texture mapping algorithm called FAST (Footprint Area Sampled Texturing), which not only delivers high quality, but also is efficient. By utilizing coherence between neighboring pixels, performing prefiltering, and applying an area sampling scheme, we guarantee a minimum number of samples sufficient for effective antialiasing. Unlike existing methods (e.g., MIP-map, Feline), our method adapts the sampling rate in each chosen MIP-map level separately to avoid undersampling in the lower level l for effective antialiasing and to avoid oversampling in the higher level l + 1 for efficiency. Our method has been shown to deliver superior image quality to Feline and other methods while retaining the same efficiency. We also provide implementation trade offs to apply a variable degree of accuracy versus speed.

We present an innovative modeling and rendering primitive, called the O-buffer, as a framework for sample-based graphics. The 2D or 3D O-buffer is, in essence, a conventional image or a volume, respectively, except that samples are not restricted to a regular grid. A sample position in the O-buffer is recorded as an offset to the nearest grid point of a regular base grid (hence the name O-buffer). The O-buffer can greatly improve the expressive power of images and volumes. Image quality can be improved by storing more spatial information with samples and by avoiding multiple resamplings. It can be exploited to represent and render unstructured primitives, such as points, particles, and curvilinear or irregular volumes. The O-buffer is therefore a unified representation for a variety of graphics primitives and supports mixing them in the same scene. It is a semiregular structure which lends itself to efficient construction and rendering. O-buffers may assume a variety of forms including 2D O-buffers, 3D O-buffers, uniform O-buffers, nonuniform O-buffers, adaptive O-buffers, layered-depth O-buffers, and O-buffer trees. We demonstrate the effectiveness of the O--buffer in a variety of applications, such as image-based rendering, point sample rendering, and volume rendering.

We present an efficient stereoscopic rendering algorithm supporting interactive navigation through large-scale 3D voxel-based environments. In this algorithm, most of the pixel values of the right image are derived from the left image by a fast 3D warping based on a specific stereoscopic projection geometry. An accelerated volumetric ray casting then fills the remaining gaps in the warped right image. Our algorithm has been parallelized on a multiprocessor by employing effective task partitioning schemes and achieved a high cache coherency and load balancing. We also extend our stereoscopic rendering to include view-dependent shading and transparency effects. We have applied our algorithm in two virtual navigation systems, flythrough over terrain and virtual colonoscopy, and reached interactive stereoscopic rendering rates of more than 10 frames per second on a 16-processor SGI Challenge.

We present a physically-based, yet fast and simple method to simulate gaseous phenomena. In our approach, the incompressible Navier-Stokes (NS) equations governing fluid motion have been modeled in a novel way to achieve a realistic animation. We introduce the Lattice Boltzmann Model (LBM), which simulates the microscopic movement of fluid particles by linear and local rules on a grid of cells so that the macroscopic averaged properties obey the desired NS equations. The LBM is defined on a 2D or 3D discrete lattice, which is used to solve fluid animation based on different boundary conditions. The LBM simulation generates, in real-time, an accurate velocity field and can incorporate an optional temperature field to account for the buoyancy force of hot gas. Because of the linear and regular operations in each local cell of the LBM grid, we implement the computation in commodity texture hardware, further improving the simulation speed. Finally, textured splats are used to add small scale turbulent details, achieving high-quality real-time rendering. Our method can also simulate the physically correct action of stationary or mobile obstacles on gaseous phenomena in real-time, while still maintaining highly plausible visual details.

We present an approach for simulating the natural dynamics that emerge from the interaction between a flow field and immersed objects. We model the flow field using the Lattice Boltzmann Model (LBM) with boundary conditions appropriate for moving objects and accelerate the computation on commodity graphics hardware (GPU) to achieve real-time performance. The boundary conditions mediate the exchange of momentum between the flow field and the moving objects resulting in forces exerted by the flow on the objects as well as the back-coupling on the flow. We demonstrate our approach using soap bubbles and a feather. The soap bubbles illustrate Fresnel reflection, reveal the dynamics of the unseen flow field in which they travel, and display spherical harmonics in their undulations. Our simulation allows the user to directly interact with the flow field to influence the dynamics in real time. The free feather flutters and gyrates in response to lift and drag forces created by its motion relative to the flow. Vortices are created as the free feather falls in an otherwise quiescent flow.

We present a simple, linear 3D cellular automata approach for animating the melting process of solid volumetric models. Accurate modelling of object melting usually requires complicated physical simulations of heat transfer and phase transition from solid to liquid. Instead, we propose a simplified model to describe the melting behaviors of highly viscous objects, such as wax, lava, plastic, metal and chocolate. We simulate the process by which a volumetric solid transforms into a viscous liquid as the amount of heat it accumulates on its surface reaches a certain temperature. We then animate smooth fluid behavior using a cellular automata. The dynamic volume data is rendered directly on texture mapping hardware to achieve an interactive speed.

We explore real time volume rendering of multichannel data for volumes with color and multimodal information. We demonstrate volume rendering of the visible human male color dataset and photorealistic rendering of voxelized terrains, and achieve high quality visualizations. We render multimodal volumes utilizing hardware programmability for accumulation level mixing, and use CT and MRI information as examples. We also use multiboard parallel/distributed rendering schemes for large datasets and investigate scalability issues. We employ the VolumePro 1000 for real time multichannel volume rendering. Our approach, however, is not hardware-specific and can use commodity texture hardware instead.

We present a novel surface reconstruction algorithm which can directly reconstruct surfaces with different levels of smoothness in one framework from height fields using 3D discrete grid ray tracing. Our algorithm exploits the 2.5D nature of the elevation data and the regularity of the rectangular grid from which the height field surface is sampled. Based on this reconstruction method, we also develop a hybrid rendering method which has the features of both rasterization and ray tracing. This hybrid method is designed to take advantage of GPUs newly available flexibility and processing power.

We propose methods to accelerate texture-based volume rendering by skipping invisible voxels. We partition the volume into sub-volumes, each containing voxels with similar properties. Sub-volumes composed of only voxels mapped to empty by the transfer function are skipped. To render the adaptively partitioned sub-volumes in visibility order, we reorganize them into an orthogonal BSP tree. We also present an algorithm that computes incrementally the intersection of the volume with the slicing planes, which avoids the overhead of the intersection and texture coordinates computation introduced by the partitioning. Rendering with empty space skipping is 2 to 5 times faster than without it. To skip occluded voxels, we introduce the concept of orthogonal opacity map, that simplifies the transformation between the volume coordinates and the opacity map coordinates, which is intensively used for occlusion detection. The map is updated efficiently by the GPU. The sub-volumes are then culled and clipped against the opacity map. We also present a method that adaptively adjusts the optimal number of the opacity map updates. With occlusion clipping, about 60% of non-empty voxels can be skipped and an additional 80% speedup on average is gained for iso-surface-like rendering.

We present an innovative modeling and rendering primitive, called the O-buffer, for sample-based graphics, such as images, volumes and points. The 2D or 3D O-buffer is in essence a conventional image or a volume, respectively, except that samples are not restricted to a regular grid. A sample position in the O-buffer is recorded as an offset to the nearest grid point of a regular base grid (hence the name O-buffer). The offset is typically quantized for compact representation and efficient rendering. The O-buffer emancipates pixels and voxels from the regular grids and can greatly improve the modeling power of images and volumes. It is a semi-regular structure which lends itself to efficient construction and rendering. Image quality can be improved by storing more spatial information with samples and by avoiding multiple resamplings and delaying reconstruction to the final rendering stage. Using O-buffers, more accurate multi-resolution representations can be developed for images and volumes. It can also be exploited to represent and render unstructured primitives, such as points, particles, curvilinear or irregular volumes. The O-buffer is therefore a uniform representation for a variety of graphics primitives and supports mixing them in the same scene. We demonstrate the effectiveness of the O-buffer with hierarchical O-buffers, layered depth O-buffers, and hybrid volume rendering with O-buffers.

This paper addresses the problem of surface reconstruction of highly noisy point clouds. The surfaces to be reconstructed are assumed to be 2-manifolds of piecewise C/sup 1/ continuity, with isolated small irregular regions of high curvature, sophisticated local topology or abrupt burst of noise. At each sample point, a quadric field is locally fitted via a modified moving least squares method. These locally fitted quadric fields are then blended together to produce a pseudo-signed distance field using Shepard's method. We introduce a prioritized front growing scheme in the process of local quadrics fitting. Flatter surface areas tend to grow faster. The already fitted regions will subsequently guide the fitting of those irregular regions in their neighborhood.

We introduce a method for the animation of fire propagation and the burning consumption of objects represented as volumetric data sets. Our method uses a volumetric fire propagation model based on an enhanced distance field. It can simulate the spreading of multiple fire fronts over a specified isosurface without actually having to create that isosurface. The distance field is generated from a specific shell volume that rapidly creates narrow spatial bands around the virtual surface of any given isovalue. The complete distance field is then obtained by propagation from the initial bands. At each step multiple fire fronts can evolve simultaneously on the volumetric object. The flames of the fire are constructed from streams of particles whose movement is regulated by a velocity field generated with the hardware-accelerated Lattice Boltzmann Model (LBM). The LBM provides a physically-based simulation of the air flow around the burning object. The object voxels and the splats associated with the flame particles are rendered in the same pipeline so that the volume data with its external and internal structures can be displayed along with the fire.

We present a simple and linear 3D cellular automata approach for animating the behaviors of viscous flow volumes in virtual environments. An accurate modeling of fluid flow usually requires complicated physical simulations. In this paper, we concentrate on the behaviors of highly viscous fluids, such as wax, lava, plastic, metal, chocolate, etc. We animate the smooth behaviors of viscous fluid based on a local 3D cellular automata. The dynamic changing volume data is rendered directly on a VolumePro board or texture mapping hardware to achieve an interactive speed.

This paper introduces the field of volume visualization, volumetric data representations, and volume rendering algorithms. It further discusses volume graphics and its underlying voxelization algorithms. Special-purpose volume rendering architectures have been researched for over two decades. Recently, commercial real-time volume rendering boards have been introduced, most notably the VolumePro board which is based on the Cube-4 architecture developed at Stony Brook University.

We present a fast and reliable space-leaping scheme to accelerate ray casting during interactive navigation in a complex volumetric scene, where we combine innovative space-leaping techniques in a number of ways. First, we derive most of the pixel depths at the current frame by exploiting the temporal coherence during navigation, where we employ a novel fast cell-based reprojection scheme that is more reliable than the traditional intersection-point based reprojection. Next, we exploit the object space coherence to quickly detect the remaining pixel depths, by using a precomputed accurate distance field that stores the Euclidean distance from each empty (background) voxel toward its nearest object boundary. In addition, we propose an effective solution to the challenging new-incoming-objects problem during navigation. Our algorithm has been implemented on a 16-processor SGI Power Challenge and reached interactive rendering rates at more than 10 Hz during the navigation inside 512/sup 3/ volume data sets acquired from both a simulation phantom and actual patients.

We propose the use of textured splats as the basic display primitives for an open surface fire model. The high-detail textures help to achieve a smooth boundary of the fire and gain the small-scale turbulence appearance. We utilize the Lattice Boltzmann Model (LBM) to simulate physically-based equations describing the fire evolution and its interaction with the environment (e.g., obstacles, wind and temperature). The property of fuel and non-burning objects are defined on the lattice of the computation domain. A temperature field is also incorporated to model the generation of smoke from the fire due to incomplete combustion. The linear and local characteristics of the LBM enable us to accelerate the computation with graphics hardware to reach real-time simulation speed, while the texture splat primitives enable interactive rendering frame rates.

In this paper, we introduce a concise and concrete definition of an accurate colon centerline and provide an efficient automatic means to extract the centerline and its associated branches (caused by a forceful touching of colon and small bowel or a deep fold in twisted colon lumen). We further discuss its applications on fly-through path planning and endoscopic simulation, as well as its potential to solve the challenging touching and colon collapse problems in virtual colonoscopy. Experimental results demonstrated its centeredness, robustness, and efficiency.

We present a generic method for rapid flight planning, virtual navigation and effective camera control in a volumetric environment. Directly derived from an accurate distance from boundary (DFB) field, our automatic path planning algorithm rapidly generates centered flight paths, a skeleton, in the navigable region of the virtual environment. Based on precomputed flight paths and the DFB field, our dual-mode physically based camera control model supports a smooth, safe, and sticking-free virtual navigation with six degrees of freedom. By using these techniques, combined with accelerated volume rendering, we have successfully developed a real-time virtual colonoscopy system on low-cost PCs and confirmed the high speed, high accuracy and robustness of our techniques on more than 40 patient datasets.

The Relativistic Heavy Ion Collider (RHIC) experiment at the Brookhaven National Lab is designed to study how the universe came into being. It is believed that after the Big Bang, the universe expanded and cooled, consisting of a soup of quarks, gluons, electrons and neutrinos. As the temperature lowered, electrons combined with protons and formed neutral atoms. Later, clouds of atoms contracted into stars. In this paper, we describe how techniques of volume rendering and information visualization are used to visualize the large particle track data set generated from this high energy physics experiment. The system, called TrackVis, is based on our earlier work of VolVis - Volume Visualization software. Example images of real particle collision data are shown, which are helpful to physicists in investigating the behavior of strongly interacting matter at high energy density.

Introduces a refined general definition of a skeleton that is based on a penalized distance function and that cannot create any of the degenerate cases of the earlier CEASAR (Center-line Extraction Algorithm-Smooth, Accurate and Robust) and TEASAR (Tree-structure Extraction Algorithm for Skeletons-Accurate and Robust) algorithms. Additionally, we provide an algorithm that finds the skeleton accurately and rapidly. Our solution is fully automatic, which frees the user from having to engage in manual data pre-processing. We present the accurate skeletons computed on a number of test data sets. The algorithm is very efficient, as demonstrated by the running times, which were all below seven minutes.

We propose the use of high-degree resampling filters for improved temporal antialiasing, or as the result is often called motion blur. Without temporal antialiasing, strange effects can occur within an animation, for example, wheels can appear to spin backwards at a certain speed. In a typical effort to overcome this, the camera shutter is left open over some period of time during the frame, leading to temporal box filtering. Even with a temporal box filter, aliasing can still occur. We show how a high-degree resampling filter, such as the Gaussian or cubic-spline lead to superior results in all cases, without the need for constant scrutiny and hand tweaking on the part of the animator.

An accurate segmentation is critical for clinical application of medical images. The undesirable partial-volume-effect, which lies on a boundary between a high intensity region and a low intensity region, makes unerring boundary determination a difficult task. A new approach to segmentation is required for removing the adverse effect on the boundary, which is unwanted especially from the point of view of volume rendering. Here, the authors propose a gradient magnitude based region growing algorithm for accurate segmentation. The gradient is useful for enhancing the boundary because it emphasizes the difference among voxel values. By analyzing the gradient magnitude, the authors can see the sufficient contrast which must be presented on the boundary region and they use this contrast to increase the accuracy of their segmentation method. The authors pay attention only to the boundary region, not to the whole large volumetric dataset itself, making it more computationally efficient.

We introduce the TEASAR algorithm which is a treestructure extraction algorithm delivering skeletons that are accurate and robust. Volumetric skeletons are needed for accurate measurements of length along branching and winding structures. Skeletons are also required in automatic virtual navigation, such as traveling through human organs (e.g., the colon) to control movement and orientation of the virtual camera. We introduce a concise but general definition of a skeleton, and provide an algorithm that finds the skeleton accurately and rapidly. Our solution is fully automatic, which frees the user from having to engage in data preprocessing. We present the accurate skeletons computed on a number of test datasets. The algorithm is efficient as demonstrated by the running times on a single 194 MHz MIPS R10000 CPU which were all below five minutes.

We present CEASAR, a centerline extraction algorithm that delivers smooth, accurate and robust results. Centerlines are needed for accurate measurements of length along winding tubular structures. Centerlines are also required in automatic virtual navigation through human organs, such as the colon or the aorta, as they are used to control movement and orientation of the virtual camera. We introduce a concise but general definition of a centerline, and provide an algorithm that finds the centerline accurately and rapidly. Our algorithm is provably correct for general geometries. Our solution is fully automatic, which frees the user from having to engage in data preprocessing. For a number of test datasets, we show the smooth and accurate centerlines computed by our CEASAR algorithm on a single 194 MHz MIPS R10000 CPU within five minutes.

We propose a novel approach for segmentation and digital cleansing of endoscopic organs. Our method can be used for a variety of segmentation needs with little or no modification. It aims at fulfilling the dual and often conflicting requirements of a fast and accurate segmentation and also eliminates the undesirable partial volume effect which contemporary approaches cannot. For segmentation and digital cleansing, we use the peculiar characteristics exhibited by the intersection of any two distinct-intensity regions. To detect these intersections we cast rays through the volume, which we call the segmentation rays as they assist in the segmentation. We then associate a certain task of reconstruction and classification with each intersection the ray detects. We further use volumetric contrast enhancement to reconstruct surface lost by segmentation (if any), which aids in improving the quality of the volume rendering.

Presents an efficient keyframeless image-based rendering technique. An intermediate image is used to exploit the coherences among neighboring frames. The pixels in the intermediate image are first rendered by a ray-casting method and then warped to the intermediate image at the current viewpoint and view direction. We use an offset buffer to record the precise positions of these pixels in the intermediate image. Every frame is generated in three steps: warping the intermediate image onto the frame, filling in holes, and selectively rendering a group of "old" pixels. By dynamically adjusting the number of those "old" pixels in the last step, the workload at every frame can be balanced. The pixels generated by the last two steps make contributions to the new intermediate image. Unlike occasional keyframes in conventional image-based rendering, which need to be totally re-rendered, intermediate images only need to be partially updated at every frame. In this way, we guarantee more stable frame rates and more uniform image qualities. The intermediate image can be warped efficiently by a modified incremental 3D warp algorithm. As a specific application, we demonstrate our technique with a voxel-based terrain rendering system.

Presents an interactive stereoscopic rendering algorithm of voxel-based terrain. It provides unambiguous depth information of a terrain scene by generating perspective images for a pair of eyes with a horizontal parallax. The left-eye image is generated using a fast ray-casting algorithm accelerated by exploiting a specific ray coherence method in the voxel-based terrain scene. The right-eye image is obtained by exploiting the frame coherence between the two views. Most of the pixel values are directly obtained from the left image by re-projection. The remaining pixels are computed by ray casting, which is further accelerated with ray coherence. An A-buffer is employed to reduce the image error caused by re-projection to non-integer pixel locations. Image-based task partitioning schemes are explored to effectively parallelize our algorithm on a multiprocessor.

An automatic method has been developed for segmentation of abdominal computed tomography (CT) images for virtual colonoscopy obtained after a bowel preparation of a low-residue diet with ingested contrast solutions to enhance the image intensities of residual colonic materials. Removal of the enhanced materials was performed electronically by a computer algorithm. The method is a multistage approach that employs a modified self-adaptive on-line vector quantization technique for a low-level image classification and utilizes a region-growing strategy for a high-level feature extraction. The low-level classification labels each voxel based on statistical analysis of its three-dimensional intensity vectors consisting of nearby voxels. The high-level processing extracts the labeled stool, fluid and air voxels within the colon, and eliminates bone and lung voxels which have similar image intensities as the enhanced materials and air, but are physically separated from the colon. This method was evaluated by volunteer studies based on both objective and subjective criteria. The validation demonstrated that the method has a high reproducibility and repeatability and a small error due to partial volume effect. As a result of this electronic colon cleansing, routine physical bowel cleansing prior to virtual colonoscopy may not be necessary.

The paper discusses and experimentally compares distance based acceleration algorithms for ray tracing of volumetric data with an emphasis on the Chessboard Distance (CD) voxel traversal. The acceleration of this class of algorithms is achieved by skipping empty macro regions, which are defined for each background voxel of the volume. Background voxels are labeled in a preprocessing phase by a value, defining the macro region size, which is equal to the voxel distance to the nearest foreground voxel. The CD algorithm exploits the chessboard distance and defines the ray as a nonuniform sequence of samples positioned at voxel faces. This feature assures that no foreground voxels are missed during the scene traversal. Further, due to parallelepipedal shape of the macro region, it supports accelerated visualization of cubic, regular, and rectilinear grids. The CD algorithm is suitable for all modifications of the ray tracing/ray casting techniques being used in volume visualization and volume graphics. However, when used for rendering based on local surface interpolation, it also enables fast search of intersections between rays and the interpolated surface, further improving speed of the process.

Conventional free-form surface design usually require tedious control-point manipulation and/or painstaking constraint specification via unnatural mouse-based interfaces. This paper presents a novel haptic approach for the direct manipulation of physics-based B-spline surfaces. Our method permits users to interactively sculpt virtual yet real material with a standard haptic device, and feel the physically realistic presence of virtual B-spline objects with force feedback throughout the design process. We aim to develop various haptic sculpting tools to expedite the direct manipulation of B-spline surfaces with haptic feedback and constraints. One significant contribution of this paper is that point, normal, and curvature constraints can be specified interactively and modified naturally using forces. We propose and formulate a dual representation for Bspline surfaces in both physical and mathematical space. This massspring model is mathematically constrained by the B-spline surface throughout the sculpting session. The equations of motion controlling the physical behavior of the B-spline surface are solved using a tractable numerical solver in real-time. The integration of haptics with traditional geometric modeling will increase the bandwidth of human-computer interaction, and thus shorten the time-consuming design cycle. We envision that this integrated approach promises a much greater potential in computer-integrated design and manufacturing, haptic interface, interactive graphics, medical applications, and virtual environments.

We present a novel forward image mapping algorithm, which speeds up perspective warping, as in texture mapping. It processes the source image in a special scanline order instead of the normal raster scanline order. This special scanline has the property of preserving parallelism when projecting to the target image. The algorithm reduces the complexity of perspective-correct image warping by eliminating the division per pixel and replacing it with a division per scanline. The method also corrects the perspective distortion in Gouraud shading with negligible overhead. Furthermore, the special scanline order is suitable for antialiasing using a more accurate antialiasing conic filter, with minimum additional cost. The algorithm is highlighted by incremental calculations and optimized memory bandwidth by reading each source pixel only once, suggesting a potential hardware implementation.

We present an algorithm which renders opaque and/or translucent polygons embedded within volumetric data. The processing occurs such that all objects are composited in the correct order, by rendering thin slabs of the translucent polygons between volume slices using slice-order volume rendering. We implemented our algorithm with OpenGL on current general-purpose graphics systems. We discuss our system implementation, speed and image quality, as well as the renderings of several mixed scenes.

We present a novel presence acceleration for volumetric ray casting. A highly accurate estimation for object presence is obtained by projecting all grid cells associated with the object boundary on the image plane. Memory space and access time are reduced by run-length encoding of the boundary cells, while boundary cell projection time is reduced by exploiting projection templates and multiresolution volumes. Efforts have also been made towards a fast perspective projection as well as interactive classification. We further present task partitioning schemes for effective parallelization of both boundary cell projection and ray traversal procedures. Good load balancing has been reached by taking full advantage of both the optimizations in the serial rendering algorithm and shared-memory architecture. Our experimental results on a 16-processor SGI Power Challenge have shown interactive rendering rates for 256/sup 3/ volumetric data sets at 10-30 Hz. We describe the theory and implementation of our algorithm, and shows its superiority over the shear-warp factorization approach.

We present an interactive navigation system for virtual colonoscopy, which is based solely on high performance volume rendering. Previous colonic navigation systems have employed either a surface rendering or a Z-buffer-assisted volume rendering method that depends on the surface rendering results. Our method is a fast direct volume rendering technique that exploits distance information stored in the potential field of the camera control model, and is parallelized on a multiprocessor. Experiments have been conducted on both a simulated pipe and patients' data sets acquired with a CT scanner.

A voxel-based terrain visualization system is presented with real-time performance on general-purpose graphics multiprocessor workstations. Ray casting of antialiased 3D volume terrain and subvoxel sampling in true 3D space produce high quality images. Based on this rendering algorithm, an interactive flythrough system for mission planning and flight simulation has been developed on an SGI Power Challenge and a virtual reality environment using a Responsive Workbench. Arbitrary stereoscopic perspective views over the terrain and a 6D input device are supported.

This paper presents an image-based walkthrough technique where reference images are sparsely sampled along a path. The technique relies on a simple user interface for rapid modeling. Simple meshes are drawn to model and represent the underlying scene in each of the reference images. The meshes, consisting of only few polygons for each image are then registered by drawing a single line on each image, called model registration line, to form an aligned 3D model. To synthesize a novel view, two nearby reference images are mapped back onto their models by projective texture-mapping. Since the simple meshes are a crude approximation to the real model in the scene, image feature lines are drawn and used as aligning anchors to further register and blend two views together and form a final novel view. The simplicity of the technique yields rapid, “home-made” image-based walkthroughs. We have produced walkthroughs from a set of photographs to show the effectiveness of the technique.

We present an efficient and robust ray-casting algorithm for directly rendering a curvilinear volume of arbitrarily-shaped cells. By projecting cell-faces onto the image plane, we have effectively addressed three critical steps of the ray-casting process, namely finding the entry cell-faces for a ray, traversing along the ray from one cell to another, and reconstructing data values at the ray/cell-face intersections. Our algorithm significantly reduces rendering time, alleviates memory space consumption, and overcomes the conventional limitation requiring cells to be convex. Application of this algorithm to several commonly used curvilinear data sets has produced a favorable performance when compared with recently reported algorithms.

Introduces a new concept for alias-free voxelization of geometric objects based on a voxelization model (V-model). The V-model of an object is its representation in 3D continuous space by a trivariate density function. This function is sampled during the voxelization and the resulting values are stored in a volume buffer. This concept enables us to study general issues of sampling and rendering separately from object-specific design issues. It provides us with a possibility to design such V-models, which are correct from the point of view of both the sampling and rendering, thus leading to both alias-free volumetric representation and alias-free rendered images. We performed numerous experiments with different combinations of V-models and reconstruction techniques. We have shown that the V-model with a Gaussian surface density profile combined with tricubic interpolation and Gabor derivative reconstruction outperforms the previously published technique with a linear density profile. This enables higher fidelity of images rendered from volume data due to increased sharpness of edges and thinner surface patches.

The paper presents a voxel-based animation technique which employs either a mass-spring model or a finite element model. Two volumetric objects, a voxelized chair and a scanned muscle, are used as case studies with the two different models. The mass-spring model is used to show an animation sequence of a falling and bouncing chair: wireframe display and volume rendering are used to display a real-time animation of the process. In addition, a real-time simulation is carried out by the finite element method (FEM) of a voxel-based multi-resolution muscle mesh. Two techniques, a direct integration and a simplified modal analysis method are discussed in the context of applying FEM for muscle deformation. Local deformation optimization with modal analysis for higher resolution muscle volumetric animation, which allows accurate prediction of muscle deformation changes, has been used. Physiological muscle force has been considered and a biomechanically-based 3D FEM muscle model has been implemented. Realistic animations have been produced based on the FEM simulation with various graphics techniques.

Recursive algorithms are generally not straightforward to be processed in parallel, particularly when an octree is involved. Some researchers have tried to process an octree in parallel, but this solution is restricted to recursive subdivision methods that guarantee surface containment in an octant. We show a whole class of recursive subdivision algorithms which does not fall into this category. Our approach solves this problem, accepting any kind of spatial recursive subdivision algorithm, by accessing and creating the octree serially, while the subdivision process is done in parallel. Thanks to our efficient octree generation algorithm, its access and creation times are meaningless in comparison to the subdivision process. We further show that our solution gives excellent results to the problem of surface voxelization using spatial recursive subdivision. The method can be also successfully applied to other computer intensive problems using spatial recursive subdivision.

We present an efficient and robust ray-casting algorithm for directly rendering a curvilinear volume of arbitrarily-shaped cells. We designed the algorithm to alleviate the consumption of CPU power and memory space. By incorporating the essence of the projection paradigm into the ray-casting process, we have successfully accelerated the ray traversal through the grid and data interpolations at sample points. Our algorithm also overcomes the conventional limitation requiring the cells to be convex. Application of this algorithm to several commonly-used curvilinear data sets has produced a favorable performance when compared with recently reported algorithms.

This paper presents a voxel‐based biomechanical model for muscle deformation using finite element method (FEM) and volume graphics. Hierarchical voxel meshes are reconstructed from filtered segmented muscle images followed by FEM simulation and volume rendering. Physiological muscle force is considered and linear elastic muscle models for both static and dynamic cases are simulated by FEM. Voxel‐based wireframe, polygon surface rendering, and volume rendering techniques are applied to show real‐time muscle deformation processes as well as realistic animations.

Virtual colonoscopy is a non-invasive computerized medical procedure for examining the entire colon to detect polyps.We present an interactive virtual colonoscopy method, which uses a physicallybased camera control model and a hardware-assisted visibility algorithm.By employing a potential field and rigid body dynamics, our camera control supplies a convenient and intuitive mechanism for examining the colonic surface while avoiding collisions.Our Zbuffer-assisted visibility algorithm culls invisible regions based on their visibility through a chain of portals, thus providing interactive rendering speed.We demonstrate our method with experimental results on a plastic pipe phantom, the Visible Human, and several patients.

Volumetric data sets require enormous storage capacity even at moderate resolution levels. The excessive storage demands not only stress the capacity of the underlying storage and communications systems, but also seriously limit the speed of volume rendering due to data movement and manipulation. A novel volumetric data visualization scheme is proposed and implemented in this work that renders 2D images directly from compressed 3D data sets. The novelty of this algorithm is that rendering is performed on the compressed representation of the volumetric data without pre-decompression. As a result, the overheads associated with both data movement and rendering processing are significantly reduced. The proposed algorithm generalizes previously proposed whole-volume frequency-domain rendering schemes by first dividing the 3D data set into subcubes, transforming each subcube to a frequency-domain representation, and applying the Fourier projection theorem to produce the projected 2D images according to given viewing angles. Compared to the whole-volume approach, the subcube-based scheme not only achieves higher compression efficiency by exploiting local coherency, but also improves the quality of resultant rendering images because it approximates the occlusion effect on a subcube by subcube basis.

We propose a probability model for the handling of complicated interactions between volumetric objects. In our model each volume is associated with a "probability map" that assigns a "surface crossing" probability to each space point according to local volume properties. The interaction between two volumes is then described by finding the intersecting regions between the volumes, and calculating the "collision probabilities" at each intersecting point from the surface crossing probabilities. To enable fast and efficient calculations, we introduce the concept of a distance map and develop two hierarchical collision detection algorithms, taking advantage of the uniform structure of volumetric datasets.

3D virtual colonoscopy has recently been proposed as a non-invasive alternative procedure for the visualization of the human colon. Surface rendering is sufficient for implementing such a procedure to obtain an overview of the interior surface of the colon at interactive rendering speeds. Unfortunately, physicians can not use it to explore tissues beneath the surface to differentiate between benign and malignant structures. In this paper, we present a direct volume rendering approach based on perspective ray casting, as a supplement to the surface navigation. To accelerate the rendering speed, surface-assistant techniques are used to adapt the resampling rates by skipping the empty space inside the colon. In addition, a parallel version of the algorithm has been implemented on a shared-memory multiprocessing architecture. Experiments have been conducted on both simulation and patient data sets.

The authors present a multiresolution framework, called Multi-Tetra framework, that approximates volume data with different levels-of-detail tetrahedra. The framework is generated through a recursive subdivision of the volume data and is represented by binary trees. Instead of using a certain level of the Multi-Tetra framework for approximation, an error-based model (EBM) is generated by recursively fusing a sequence of tetrahedra from different levels of the Multi-Tetra framework. The EBM significantly reduces the number of voxels required to model an object, while preserving the original topology. The approach provides continuous distribution of rendered intensity or generated isosurfaces along boundaries of different levels-of-detail thus solving the crack problem. The model supports typical rendering approaches, such as marching cubes, direct volume projection, and splatting. Experimental results demonstrate the strengths of the approach.

This paper presents a novel approach to assist the user in exploring appropriate transfer functions for the visualization of volumetric datasets. The search for a transfer function is treated as a parameter optimization problem and addressed with stochastic search techniques. Starting from an initial population of (random or pre-defined) transfer functions, the evolution of the stochastic algorithms is controlled by either direct user selection of intermediate images or automatic fitness evaluation using user-specified objective functions. This approach essentially shields the user from the complex and tedious "trial and error" approach, and demonstrates effective and convenient generation of transfer functions.

We present new volume rendering techniques for efficiently generating high-quality stereoscopic images and propose criteria to evaluate stereo volume rendering algorithms. Specifically, we present fast stereo volume ray casting algorithms using segment composition and linearly-interpolated re-projection. A fast stereo shear-warp volume rendering algorithm is also presented and discussed.

We present a modeling technique based on the metaphor of interactively sculpting complex 3D objects from a solid material, such as a block of wood or marble. The 3D model is represented in a 3D raster of voxels where each voxel stores local material property information such as color and texture. Sculpting is done by moving 3D voxel-based tools within the model. The affected regions are indicated directly on the 2D projected image of the 3D model. By reducing the complex operations between the 3D tool volume and the 3D model down to primitive voxel-by-voxel operations, coupled with the utilization of a localized ray casting for image updating, our sculpting tool achieves real-time interaction. Furthermore, volume sampling techniques and volume manipulations are employed to ensure that the process of sculpting does not introduce aliasing into the models.

Presents a simple, robust and practical method for object simplification for applications where gradual elimination of high-frequency details is desired. This is accomplished by sampling and low-pass filtering the object into multi-resolution volume buffers and applying the marching cubes algorithm to generate a multi-resolution triangle-mesh hierarchy. Our method simplifies the genus of objects and can also help existing object simplification algorithms achieve better results. At each level of detail, a multi-layered mesh can be used for an optional and efficient antialiased rendering.

This paper describes an approach for interactive visualization of mixed scalar and vector fields, in which vector icons are generated from pre-voxelized icon templates and volume-rendered together with the volumetric scalar data. This approach displays simultaneously the global structure of the scalar field and the detailed features of the vector field. Interactive visualization is achieved with incremental image update, by re-rendering only a small portion of the image wherever and whenever a change occurs. This technique supports a set of interactive visualization tools, including change of vector field visualization parameters, real-time animation of vector icons advected within the scalar field, a zooming lens, and a local probe.

The paper presents a splatting algorithm for volume rendering of curvilinear grids. A stochastic sampling technique called Poisson sphere/ellipsoid sampling is employed to adaptively resample a curvilinear grid with a set of randomly distributed points whose energy support extents are well approximated by spheres and ellipsoids. Filter kernels corresponding to these spheres and ellipsoids are used to generate the volume rendered image of the curvilinear grid with a conventional footprint evaluation algorithm. Experimental results show that our approach can be regarded as an alternative to existing fast volume rendering techniques of curvilinear grids.

Proposes a new approach to the automatic generation of triangular irregular networks (TINs) from dense terrain models. We have developed and implemented an algorithm based on the greedy principle used to compute minimum-link paths in polygons. Our algorithm works by taking greedy cuts ("bites") out of a simple closed polygon that bounds the yet-to-be triangulated region. The algorithm starts with a large polygon, bounding the whole extent of the terrain to be triangulated, and works its way inward, performing at each step one of three basic operations: ear cutting, greedy biting, and edge splitting. We give experimental evidence that our method is competitive with current algorithms and has the potential to be faster and to generate many fewer triangles. Also, it is able to keep the structural terrain fidelity at almost no extra cost in running time and it requires very little memory beyond that for the input height array.

Abstract Volume rendering is a key technique in scientific visualization that lends itself to significant exploitable parallelism. The high computational demands of real‐time volume rendering and continued technological advances in the area of VLSl give impetus to the development of special‐purpose volume rendering architectures. This paper presents and characterizes three recently developed volume rendering engines which are based on the ray‐casting method. A taxonomy of the algorithmic variants of ray‐casting and details of each ray‐casting architecture are discussed. The paper then compares the machinefeatures and provides an outlook onfuture developments in the area of volume rendering hardware.

We extend a 2D linear skewed memory organization to 3D and introduce the associated de-skewing scheme designed to provide conflict-free access to projection rays of voxels for use in a volume rendering architecture. This is an application of a 3D linear skewing scheme which supports real-time axonometric projection from 26 primary orientations.>

VolVis is a diversified, easy to use, extensible, high performance, and portable volume visualization system for scientists and engineers as well as for visualization developers and researchers. VolVis accepts as input 3D scalar volumetric data as well as 3D volume-sampled and classical geometric models. Interaction with the data is controlled by a variety of 3D input devices in an input device-independent environment. VolVis output includes navigation preview, static images, and animation sequences. A variety of volume rendering algorithms are supported ranging from fast rough approximations, to compression-domain rendering, to accurate volumetric ray tracing and radiosity, and irregular grid rendering.>

This paper presents a technique for performing volume morphing between two volumetric datasets in the wavelet domain. The idea is to decompose the volumetric datasets into a set of frequency bands, apply smooth interpolation to each band, and reconstruct to form the morphed model. In addition, a technique for establishing a suitable correspondence among object voxels is presented. The combination of these two techniques results in a smooth transition between the two datasets and produces morphed volume with fewer high frequency distortions than those obtained from spatial domain volume morphing.>

Describes a technique for achieving fast volume ray-casting on parallel machines, using a load-balancing scheme and an efficient pipelined approach to compositing. We propose a new model for measuring the amount of work one needs to perform in order to render a given volume, and we use this model to obtain a better load-balancing scheme for distributed memory machines. We also discuss in detail the design trade-offs of our technique. In order to validate our model, we have implemented it on the Intel iPSC/860 and the Intel Paragon, and conducted a detailed performance analysis.>

The device unified interface is a generalized and easily expandable protocol for the communication between applications and input devices. The key idea is to unify various device data into the parameters of a so-called "virtual input device." The device information-base, which includes device dependent information, is also incorporated into the virtual input device. Using the device unified interface, system builders are able to design their applications independent of the input devices as well as utilize the capabilities of several devices in the same application.>

We present a 3-D antialiasing algorithm for voxel-based geometric models. The technique band-limits the continuous object before sampling it at the desired 3-D raster resolution. By precomputing tables of filter values for different types and sizes of geometric objects, the algorithm is very efficient and has a complexity that is linear with the number of voxels generated. The algorithm not only creates voxel models which are free from object space aliasing, but it also incorporates the image space antialiasing information as part of the view independent voxel model. The resulting alias-free voxel models have been used to model synthetic scenes, for discrete ray tracing applications. The discrete ray-traced image is superior in quality to the image generated with a conventional surface-based ray tracer, since silhouettes of objects, shadows, and reflections appear smooth (jaggy-less). In addition, the alias-free models are also suitable for intermixing with sampled datasets, since they can be treated uniformly as one common data representation.>

The VolVis system has been developed to satisfy the diverse requirements of the volume visualization community by comfortably housing numerous visualization algorithms and methods within a consistent and well organized framework. The VolVis system is supported by a generalized abstract model which provides for both geometric and volumetric constructs. VolVis contains several rendering algorithms that span the speed versus accuracy continuum. A fast volume rendering algorithm has been developed, which is capable of exploiting existing graphics hardware without placing any viewing restrictions or compromising accuracy. In addition, VolVis includes a volumetric navigation facility, key-frame animation generator, quantitative analysis tools, and a generalized protocol for communicating with 3D input devices.>

Abstract We present an efficient three‐phase algorithm for volume viewing that is based on exploiting coherency between rays in parallel projection. The algorithm starts by building a ray‐template and determining a special plane for projection ‐ the base‐plane. Parallel rays are cast into the volume from within the projected region of the volume on the base‐plane, by repeating the sequence of steps specified in the ray‐template. We carefully choose the type of line to be employed and the way the template is being placed on the base‐plane in order to assure uniform sampling of the volume by the discrete rays. We conclude by describing an optimized software implementation of our algorithm and reporting its performance.

A set of volume visualization tools that are based on the use of recursive ray tracing as the primary vehicle for realistic volume imaging is presented. The tools include shadows, mirrors, specularity, and constructive solid geometry. The underlying representation for the ray tracer is a 3-D raster of voxels that holds the discrete form of the scene. Unlike traditional volume rendering techniques, the discrete recursive ray tracer models many illumination phenomena by traversing discrete rays in voxel space. The approach provides true ray tracing of sampled or computed datasets, as well as ray tracing of hybrid scenes where sampled or computed data are intermixed with geometric models and enhances the understanding of complex biomedical datasets.>

No abstract available.

The authors discuss the special properties of volumetric cell data (e.g., noise, discontinuity, raggedness) and the particular difficulties encountered when trying to visualize them in three dimensions. The authors describe some of the solutions adopted, specifically in surface discrimination and shading. Nerve cells (neuroblastoma) grown in tissue culture were selected as the biological preparation because these cells possess very rich actin structures. The cells were stained with a fluorescent probe specific for actin (rhodamine-phalloidin) and were viewed and optically sectioned using the Bio-Rad MRC 600 confocal fluorescence microscope. The slice dataset was then reconstructed and processed in the BioCube environment, a comprehensive system developed for volume visualization of cellular structures. The actin cytoskeleton of single cells was visualized and manipulated using this system.>

A forest of quadtrees is proposed as an alternative data structure for representing and manipulating 3D and 2.5D graphics. A data representation of a forest offers space savings over common quadtrees by concentrating the vital information and discarding unused pointers. Several properties of the forest of quadtrees and the basic operations for display and elementary transformations like rotation, reflection, enlargement, reduction, and translation are investigated. Specifically, the temporary memory requirements and duplication time of the algorithms are analyzed.

A three-dimensional (3D) scan-conversion algorithm, that scanconverts 3D planar polygons into their discrete voxel-map representation within a Cubic Frame Buffer (CFB), is presented. The algorithm, which is a variation of a 2D scan-line filling algorithm, is incremental and uses only simple operations like additions and testy inside the inner loops. The algorithm performs scan-conversion with computational complexity which is linear in the number of voxels written to the CFB. The paper also presents an algorithm that scan-converts polygons clipped to the CFB boundaries with no added time complexity. An all-integer decision mechanism which makes the inner-most loop of the algorithm more efficient is discussed too. All the algorithms guarantee lack of 6-connected "tunnels" in the converted polygons. The algorithms have been implemented as part of the 3D geometry processor of the CUBE Architecture, which is a voxel-based system for 3D graphics. These algorithms allow the' CUBE system to generate the essential primitive polygon within the CFB from a 3D geometric model.

Three-dimensional (3D) scan-conversion algorithms, that scan-convert 3D parametric objects into their discrete voxelmap representation within a Cubic Frame Buffer (CFB), are presented. The parametric objects that are studied include Bezier form of cubic parametric curves, bicubic parametric surface patches, and tricubic parametric volumes. The converted objects in discrete 3D space maintain pre-defined application-dependent connectivity and fidelity requirements.The algorithms introduced here emply third-order forward difference techniques. Efficient versions of the algorithms based on first-order decision mechanisms, which employ only integer arithmetic, are also discussed. All algorithms are incremental and use only simple operations inside the inner algorithm loops. They perform scan-conversion with computational complexity which is linear in the number of voxels written to the CFB. All the algorithms have been implemented as part of the CUBE Architecture, which is a voxel-based system for 3D graphics.

A special memory organization of a cubic voxel-based frame buffer is presented. The memory of voxels is divided into n modules of voxels each. The k-th module groups together all the principal diagonal planes within the cubic frame buffer, that their sequential index is k mod (n ). This organization guarantees that all the n voxels which constitute a row in any direction parallel to the axes reside in different modules, permitting the retrieval of all the voxels of a row simultaneously. This memory layout is the heart of a voxel-based CUBE Architecture for three-dimensional graphics. The architecture employs three processors which access the memory, to input, manipulate, and view the image within the memory. The unique memory organization enables these processors and the entire system to cope with real-time constraints.

A new architecture for storing and processing of threedimensional (3-D) graphics is described, The 3-D objects are discretized and stored in a full 3-D cellular memory of voxels. A 3-D graphics processor scan-converts 3-D geometric objects into the cellular representation. A 3-D frame buffer processor manipulates 3-D cellular sub-boxes and controls 3-D interaction. Displaying 2-D orthographic projections from a given view position and direction is accomplished by a 3-D viewing processor. With this new architecture, there is neither a need for repeatedly scan-converting modified geometric objects nor a need for removing hidden-surfaces.

Two improved variations of the binary buddy system for dynamic memory management, the tailoredlist buddy system (TLBS) and the recombination-delaying buddy system (RDBS), are introduced.In an attempt to save on execution time, these variations do not recombine free buddies every time recombination is possible.In the TLBS recombination is delayed in such a way as to tailor the available free-space lists to the request-size distribution.In the RDBS time is saved by recombining buddies only when larger blocks are unavailable during allocation.Comparative simulation experiments indicate that for not very heavy loads the TLBS is slightly faster than the RDBS, which, in turn, is significantly faster than the traditional system.Since no significant variation has been found among the memory utilizations of the three systems, the TLBS is preferable to the other systems provided that the load is not very heavy and the expected requestsize distribution is at hand.Otherwise, the RDBS is recommended.

BGRAF2 is a real-time interactive 2D graphics language. Its supporting system contends with an unusual combination of features: timing, events, parallelism, image manipulation, user interaction and procedural structures. This combination creates within the system many unpredictable interrelated tasks competing for execution.

Various approaches to explicit and implicit control of time and motion in procedural real-time graphic packages and languages are discussed and illustrated. The particular approach of independentty defining static objects and attribute is introduced and developed. The procedures which alter and manipulate basic objects at interactive run time, teamed dynamic attributes, may be defined independently of objects as a function of time, input, and other variables. Attributes, once defined, may be associated with and dissociated from objects dynamically. Several objects may simultaneously have the identical attribute. A particular & association exists within its own zero-origin local time frame. Basic objects are static two-or three-dimensional entities in some space. Dynamic objects termed images may be recursively defined in teams oa basic objects and attributes. Objects and attributes are the basic building blocks enabling modular program construction. The notions presented here can be used as the basis for design of interactive graphic languages or oda the implementation of graphics packages. An experimental FORTRAN-callable package has been developed to test these notions.

The BGRAF2 language for interactive real-time 2D graphics was designed as a user-oriented language emphasizing ease of use rather than of implementation. Procedural statements allow straightforward computation while classlike characteristics encourage modular graphics programming. The clock and event data types facilitate creation of parallel and synchronous procedures automatically acting on structures and display files. A graphic procedure, which includes procedural, drawing and event-driven statements, is a graphic prototype. Such procedures may be nested and iteratively or recursively called. An individual invocation of a procedure, called a graphic object, shares code while commanding its own data structures and images.