Stream surfaces are an intuitive approach to represent 3D vector fields. In many cases, however, they are challenging objects to visualize and to understand, due to a high degree of self-occlusion. Despite the need for adequate rendering methods, little work has been done so far in this important research area. In this paper, we present an illustrative rendering strategy for stream surfaces. In our approach, we apply various rendering techniques, which are inspired by the traditional flow illustrations drawn by Dallmann and Abraham \& Shaw in the early 1980s. Among these techniques are contour lines and halftoning to show the overall surface shape. Flow direction as well as singularities on the stream surface are depicted by illustrative surface streamlines. ;To go beyond reproducing static text book images, we provide several interaction features, such as movable cuts and slabs allowing an interactive exploration of the flow and insights into subjacent structures, e.g., the inner windings of vortex breakdown bubbles. These methods take only the parameterized stream surface as input, require no further preprocessing, and can be freely combined by the user. We explain the design, GPU-implementation, and combination of the different illustrative rendering and interaction methods and demonstrate the potential of our approach by applying it to stream surfaces from various flow simulations. ;

An iterative classification method developed for 2D CT head data classification problem and using both statistical and spatial information is introduced in this paper. The method reduces the chance of misclassification, preserving the contiguity of tissue classes. This method is minimally supervised so that it enforces a relation between tissues and classes. In later iterations high-confidence points are used to help classify nearby ambiguous points, based on the assumption that points in close proximity and of comparable intensities are probably representing the same tissue class.

One of the main challenges in external optical tracking device based augmented reality (AR) is the extrinsic calibration. Such an example is calibration of the endoscope camera with the tracking device to estimate the transformation between the endoscope-mounted marker and endoscope sensor. In this paper, we describe two alternative techniques to the Hand-Eye method for online calibration. First, we describe a direct technique based on estimating rigid transformation from corresponding point sets and our idea of improving the calibration efficiency by collecting corresponding points covering broader area of the endoscope and tracker field-of-view (FOV). Then, we describe a technique based on estimation of position and orientation of a planar object from camera image. The main advantage of these techniques is that they are easily repeatable in applications where a change in the relation between camera sensor and camera-mounted marker is possible during the run-time of the AR application.

Abstract Common practice in brain research and brain surgery involves the multi‐modal acquisition of brain anatomy and brain activation data. These highly complex three‐dimensional data have to be displayed simultaneously in order to convey spatial relationships. Unique challenges in information and interaction design have to be solved in order to keep the visualization sufficiently complete and uncluttered at the same time. The visualization method presented in this paper addresses these issues by using a hybrid combination of polygonal rendering of brain structures and direct volume rendering of activation data. Advanced rendering techniques including illustrative display styles and ambient occlusion calculations enhance the clarity of the visual output. The presented rendering pipeline produces real‐time frame rates and offers a high degree of configurability. Newly designed interaction and measurement tools are provided, which enable the user to explore the data at large, but also to inspect specific features closely. We demonstrate the system in the context of a cognitive neurosciences dataset. An initial informal evaluation shows that our visualization method is deemed useful for clinical research.

The goal of stylization is to provide an abstracted representation of an image that highlights specific types of visual information. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently making stylization into a useful technique, not only for artistic, but also for visualization applications. In this paper, we report results from two sets of experiments that aim at characterizing the perceptual impact and effectiveness of three different stylization techniques in the context of dynamic facial expressions. In the first set of experiments, animated facial expressions are stylized using three common techniques (brush, cartoon, and illustrative stylization) and investigated using different experimental measures. Going beyond the usual questionnaire approach, these experiments compare the techniques according to several criteria ranging from subjective preference to task-dependent measures (such as recognizability, intensity) allowing us to compare behavioral and introspective approaches. The second set of experiments use the same stylization techniques on real-world video sequences in order to compare the effect of stylization on natural and artificial stimuli. Our results shed light on how stylization of image contents affects the perception and subjective evaluation of both real and computer-generated facial expressions.

In video see-through augmented reality (AR), virtual objects are overlaid over digital video images. One particular problem of this image mixing process is that the visual appearance of the computer graphics differs strongly from the real background image. The reason for this is that typical AR systems use fast but simple real-time rendering techniques for displaying virtual objects. In this paper, methods for reducing the impact of three effects which make virtual and real objects easily distinguishable are presented. The first effect is camera image noise, which is contained in the data delivered by the image sensor used for capturing the real scene. The second effect considered is edge aliasing, which makes distinguishing virtual objects from real objects simple. Finally, we consider motion blur, which is caused by the temporal integration of color intensities in the image sensor during fast movements of the camera or observed objects. In this paper, we present a system for generating a realistic simulation of image noise based on a new camera calibration step. Additionally, a rendering algorithm is introduced, which performs a smooth blending between the camera image and virtual objects at their boundary in order to reduce aliasing. Lastly, a rendering method is presented, which produces motion blur according to the current camera movement. The implementation of the new rendering techniques utilizes the programmability of modern graphics processing units (GPUs) and delivers real-time frame rates.

Forensic pathology is largely concerned with the determination of the cause and manner of deaths after accidents, or other circumstances in criminal investigations. A major task in that process is the documentation of surface injuries, which is traditionally done by drawing sketches, photography, or more recently by photogrammetry to generate a three-dimensional digital lesion cartography of the body surface. In this paper, we describe a semi-automatic processing pipeline how data from 3D photogrammetry is combined and used to generate a visual surface representation of accident victims. In that course, a number of steps are performed to provide a high-quality interactive, point-based visualization of the acquired data, which can be used in a more routine way than previous forensic surface methods.

Abstract Image space occlusion culling is a powerful approach to reduce the rendering load of large polygonal models. However, occlusion culling is not for free; it trades overhead costs with the rendering costs of the possibly occluded geometry. Meanwhile, occlusion queries based on image space occlusion culling are supported on modern graphics hardware. However, a significant consumption of fillrate bandwidth and latency costs are associated with these queries. In this paper, we propose new techniques to reduce redundant occlusion queries. Our approach uses several "Occupancy Maps" to organize scene traversal. The respective information is accumulated efficiently by hardware‐supported asynchronous occlusion queries. To avoid redundant requests, we arrange these multiple occlusion queries according to the information of the Occupancy Maps. Our presented technique is conservative and benefits from a partial depth order of the geometry.

The demand for high quality and speed in graphical representations of 3D scenes is continuously increasing. An important aspect of visual realism is real-time processing of transparent objects. In order to provide a realistic solution to be considered for future graphics cards, proposed extensions to the hardware need to be small. In this paper, we propose two new hardware oriented proposals for computing order independent transparency for any number of layers. The first proposal is based on the precomputation of the contribution factor of each element to the final colour of the pixel. This permits the evaluation of the transparent objects without requiring their previous sorting. The second proposal is based on the utilization of a pre-sorting stage so that the transparent objects are organized in a back to front order. In both proposals, time requirements are reduced with respect to previous solutions.

Recently, we have proposed a novel approach to generating augmented video streams. The output images are a non-photorealistic reproduction of the augmented environment. Special stylization methods are applied to both the background camera image and the virtual objects. This way, the graphical foreground and the real background images are rendered in a similar style, so that they are less distinguishable from each other. Here, we present a new algorithm for the cartoon-like stylization of augmented reality images, which uses a novel post-processing filter for cartoon-like color segmentation and high-contrast silhouettes. In order to make a fast post-processing of rendered images possible, the programmability of modern graphics hardware is exploited. The system is capable of generating a stylized augmented video stream of high visual quality at real-time frame rates.

Indirect volume rendering is a widespread method for the display of volume datasets. It is based on the extraction of polygonal iso-surfaces from volumetric data, which are then rendered using conventional rasterization methods. Whereas this rendering approach is fast and relatively easy to implement, it cannot easily provide an understandable display of structures occluded by the directly visible iso-surface. Simple approaches like alpha-blending for transparency when drawing the iso-surface often generate a visually complex output, which is difficult to interpret. Moreover, such methods can significantly increase the computational complexity of the rendering process. In this paper, we therefore propose a new approach for the illustrative indirect rendering of volume data in real-time. This algorithm emphasizes the silhouette of objects represented by the iso-surface. Additionally, shading intensities on objects are reproduced with a monochrome hatching technique. Using a specially designed two-pass rendering process, structures behind the front layer of the iso-surface are automatically extracted with a depth peeling method. The shapes of these hidden structures are also displayed as silhouette outlines. As an additional option, the geometry of explicitly specified inner objects can be displayed with constant translucency. Although these inner objects always remain visible, a specific shading and depth attenuation method is used to convey the depth relationships. We describe the implementation of the algorithm, which exploits the programmability of state-of-the-art graphics processing units (GPUs). The algorithm described in this paper does not require any preprocessing of the input data or a manual definition of inner structures. Since the presented method works on iso-surfaces, which are stored as polygonal datasets, it can also be applied to other types of polygonal models.

The ultimate goal of augmented reality is to provide the user with a view of the surroundings enriched by virtual objects. Practically all augmented reality systems rely on standard real-time rendering methods for generating the images of virtual scene elements. Although such conventional computer graphics algorithms are fast, they often fail to produce sufficiently realistic renderings. The use of simple lighting and shading methods, as well as the lack of knowledge about actual lighting conditions in the real surroundings, cause virtual objects to appear artificial. In this paper, we propose an entirely novel approach for generating augmented reality images in video see-through systems. Our method is based on the idea of applying stylization techniques for reducing the visual realism of both the camera image and the virtual graphical objects. A special painterly image filter is applied to the camera video stream. The virtual scene elements are generated using a non-photorealistic rendering method. Since both the camera image and the virtual objects are stylized in a corresponding cartoon-like or sketch-like way, they appear very similar. As a result, the graphical objects seem to be an actual part of the real surroundings. We describe both the new painterly filter for the camera image and the non-photorealistic rendering method for virtual scene elements, which has been adapted for this purpose. Both are fast enough for generating augmented reality images in real-time and are highly customizable. The results obtained using our method, are very promising and show that it improves immersion in augmented reality.

The goal of augmented reality is to provide the user with a view of the surroundings enriched by virtual objects. Practically all augmented reality systems rely on standard real-time rendering methods for displaying graphical objects. Although such conventional computer graphics algorithms are fast, they often fail to produce sufficiently realistic renderings. Therefore, virtual models can easily be distinguished from the real environment. We have recently proposed a novel approach for generating augmented reality images [4]. Our method is based on the idea of applying stylization techniques for adapting the visual realism of both the camera image and the virtual graphical objects. Since both the camera image and the virtual objects are stylized in a corresponding way, they appear very similar. Here, we present a new method for the stylization of augmented reality images. This approach generates a painterly brush stroke rendering. The resulting stylized augmented reality video frames look similar to paintings created in the pointillism style. We describe the implementation of the camera image filter and the non-photorealistic rendererfor virtual objects. These components have been newly designed or adapted for this purpose. They are fast enough for generating augmented reality images in near real-time (more than 14 frames per second).

Abstract Virtual endoscopy is among the most active topics in virtual medicine and medical imaging. It focuses on the virtual representation of minimally invasive procedures for training, planning and diagnosis without an actual invasive intervention. In the past few years, virtual endoscopy modes have been transferred from research systems in virtually every commercial medical imaging software, but with varying quality and flexibility. This report covers concepts used in current systems in research and products, and how they might be applied to daily practice in health care. Specifically, I will start with an introduction into virtual endoscopy and the related medical field. This will also include typical scenarios of virtual endoscopy applications as they appear in clinical practice. This part will be followed by a discussion of the technical issues of virtual endoscopy and how they are addressed in currently available systems. Among these issues are navigation through the respective body organ and the orientation aids for the users. In addition, I will highlight the different rendering techniques used and its impact on rendering speed and quality.

Utilizing augmented reality for applications in medicine has been a topic of intense research for several years. A number of challenging tasks need to be addressed when designing a medical AR system. These include the import and management of medical datasets and preoperatively created planning data, the registration of the patient with respect to a global coordinate system, and accurate tracking of the camera used in the AR setup as well as the respective surgical instruments. Most research systems rely on specialized hardware or algorithms for realizing augmented reality in medicine. Such base technologies can be expensive or very time-consuming to implement. In this paper, we propose an alternative approach of building a surgical AR system by harnessing existing, commercially available equipment for image guided surgery (IGS). We describe the prototype of an augmented reality application, which receives all necessary information from a device for intraoperative navigation.

The support of surgical interventions has long been in the focus of application-oriented augmented reality research. Modern methods of surgery, like minimally-invasive procedures, can benefit from the additional information visualization provided by augmented reality. The usability of medical augmented reality depends on a rendering scheme for virtual objects designed to generate easily and quickly understandable augmented views. One important factor for providing such an accessible reality augmentation is the correct handling of the occlusion of virtual objects by real scene elements. The usually large volumetric datasets used in medicine are ill-suited for use as phantom models for static occlusion handling. We present a simple and fast preprocessing pipeline for medical volume datasets which extracts their visual hull volume. The resulting, significantly simplified visual hull iso-surface is used for real-time static occlusion handling in our AR system, which is based on off-the-shelf medical equipment.

Segmentation of the tracheo-bronchial tree of the lungs is notoriously difficult. This is due to the fact that the small size of some of the anatomical structures is subject to partial volume effects. Furthermore, the limited intensity contrast between the participating materials (air, blood, and tissue) increases the segmentation of difficulties. In this paper, we propose a hybrid segmentation method which is based on a pipeline of three segmentation stages to extract the lower airways down to the seventh generation of the bronchi. User interaction is limited to the specification of a seed point inside the easily detectable trachea at the upper end of the lower airways. Similarly, the complementary vascular tree of the lungs can be segmented. Furthermore, we modified our virtual endoscopy system to visualize the vascular and airway system of the lungs along with other features, such as lung tumors.

We present a new visibility driven rasterization scheme that significantly increases the rendering performance of modern graphic subsystems. Instead of rasterizing, texturing, lighting, and depth‐testing each individual pixel, we introduce a two‐level visibility mask within the rasterization stage which facilitates the removal of groups of pixels and triangles from rasterization and subsequent pipeline stages. Local visibility information is stored within the visibility mask that is updated several times during the generation of a frame. The update can easily be accomplished by extending already available (in hardware) occlusion culling mechanisms (i.e. those of HP and SGI), where it is possible to integrate the additional functionality without any additional delay cycles. In addition to these existing hardware based occlusion culling approaches—which cull only geometry contained in bounding volumes determined as occluded—we are able to significantly accelerate the rendering of the geometry determined as visible. However, our approach does not specifically rely on such occlusion culling hardware.The proposed new rasterization scheme is well suited for hardware implementation, can easily be integrated into low‐cost rasterizers, and its scalability can vary upon available chip real estate. Only incremental modifications of modern graphics subsystems are required to achieve a significant improvement in rendering performance.

We present a system for interactive explorations of extra- and intracranial blood vessels. Starting with a stack of images from 3D angiography, we use virtual clips to limit the segmentation of the vessel tree to the parts the neuroradiologists are interested in. Furthermore, methods of interactive virtual endoscopy are applied in order to provide an interior view of the blood vessels.

Virtual colonoscopy is a non-invasive computerized medical procedure for examining the entire colon to detect polyps.We present an interactive virtual colonoscopy method, which uses a physicallybased camera control model and a hardware-assisted visibility algorithm.By employing a potential field and rigid body dynamics, our camera control supplies a convenient and intuitive mechanism for examining the colonic surface while avoiding collisions.Our Zbuffer-assisted visibility algorithm culls invisible regions based on their visibility through a chain of portals, thus providing interactive rendering speed.We demonstrate our method with experimental results on a plastic pipe phantom, the Visible Human, and several patients.