Neural Radiance Field (NeRF) is a promising deep learning approach for three-dimensional (3D) scene reconstruction and view synthesis, with various applications in fields like robotics and medical imaging. However, similar to other deep learning models, understanding NeRF model inaccuracies and their causes is challenging. The 3D nature of NeRFs further adds challenges such as identifying complex geometrical features and analyzing 2D views that suffer from object occlusions. Existing methods for uncertainty quantification (UQ) in NeRFs address the lack of NeRF model understanding by expressing uncertainty in model predictions, exposing limitations in model design or training data. However, these UQ techniques typically rely on quantitative evaluation that does not facilitate human interpretation. We introduce NeRVis, a visual analytics system that supports model users to explore and analyze uncertainty in NeRF scenes. NeRVis combines spatial uncertainty analysis with per-view uncertainty summaries, fostering analysis of the uncertainty in Lambertian NeRF scenes. As a proof-of-concept, we illustrate our approach using two UQ methods. We demonstrate the effectiveness of NeRVis with two different use scenarios, tackling key challenges in the NeRF UQ literature.

Abstract Real‐time rendering of dynamic line sets is relevant in many visualization tasks, including unsteady flow visualization and interactive white matter reconstruction from Magnetic Resonance Imaging. High‐quality global illumination and transparency are important for conveying the spatial structure of dense line sets, yet remain difficult to achieve at interactive rates. We propose an efficient voxel‐based ray‐tracing framework for rendering large dynamic line sets with ambient occlusion and ground‐truth transparency. We introduce a voxelization algorithm that supports efficient on‐the‐fly construction of acceleration structures for both voxel cone tracing and ray tracing. To further reduce per‐frame preprocessing cost, we propose a voxel‐based culling method that restricts acceleration structure construction to camera‐visible voxels. Together, these contributions enable real‐time rendering of large‐scale dynamic line sets with high quality and physically accurate transparency. We demonstrate that our method outperforms the state of the art in quality and performance when rendering (semi‐)opaque dynamic line sets.

Correlation clustering (CC) offers an effective approach to analyze scalar field ensembles by detecting correlated regions and consistent structures, enabling the extraction of meaningful patterns. However, existing CC methods are computationally expensive, making them impractical for both interactive analysis and large-scale scalar fields. We introduce the Local-to-Global Correlation Clustering (LoGCC) framework, which accelerates pivot-based CC by leveraging the spatial structure of scalar fields and the weak transitivity of correlation. LoGCC operates in two stages: a local step that uses the neighborhood graph of the scalar field's spatial domain to build highly correlated local clusters, and a global step that merges them into global clusters. We implement the LoGCC framework for two well-known pivot-based CC methods, Pivot and CN-Pivot, demonstrating its generality. Our evaluation using synthetic and real-world meteorological and medical image segmentation datasets shows that LoGCC achieves speedups-up to 15 × for Pivot and 200 × for CN-Pivot-and improved scalability to larger scalar fields, while maintaining cluster quality. These contributions broaden the applicability of correlation clustering in large-scale and interactive analysis settings.

Abstract Comparing gene organization across genomic sequences reveals insights into evolutionary and functional diversity among different organisms and varieties. Performing this task across many sequences, such as from a pangenome, is challenging because of the scale, the density of information, and the inherent variation. Often, analyses are centered on a genomic region of interest—a locus that might be associated with a trait or contain genes within the same family or biological pathway. Within these regions, researchers examine the conservation of gene order and orientation across organisms and assess sequence similarity, along with other gene content features such as gene size, to find biological variations or potential errors in the data. Automated methods in comparative genomics struggle to identify meaningful patterns due to varying and often unknown features of interest, leaving manual, time‐intensive, and scalability‐challenged visualization as the primary alternative. To address these challenges, we present a multiscale design for studying gene organization within pangenomes, developed in close collaboration with domain experts. Our tool, Multipla, enables users to explore organization at multiple levels of detail in a decluttered manner through layout abstractions, semantic zooming, and layouts with flexible distance definitions and feature selections, combining the advantages of manual and automated methods used in practice. We evaluate the design of Multipla through two pangenomic use cases and conclude with lessons learned from designing multiscale views for pangenomic locus analysis.

Genomics experts rely on visualization to extract and share insights from complex and large-scale datasets. Beyond off-the-shelf tools for data exploration, there is an increasing need for platforms that aid experts in authoring customized visualizations for both exploration and communication of insights. A variety of interactive techniques have been proposed for authoring data visualizations, such as template editing, shelf configuration, natural language input, and code editors. However, it remains unclear how genomics experts create visualizations and which techniques best support their visualization tasks and needs. To address this gap, we conducted two user studies with genomics researchers: (1) semi-structured interviews (n=20) to identify the tasks, user contexts, and current visualization authoring techniques and (2) an exploratory study (n=13) using visual probes to elicit users' intents and desired techniques when creating visualizations. Our contributions include (1) a characterization of how visualization authoring is currently utilized in genomics visualization, identifying limitations and benefits in light of common criteria for authoring tools, and (2) generalizable design implications for genomics visualization authoring tools based on our findings on task- and user-specific usefulness of authoring techniques. All supplemental materials are available at https://osf.io/bdj4v/.

Dynamic three-dimensional (4D) reconstruction from two-dimensional X-ray coronary angiography (CA) remains a significant clinical problem. Existing CA reconstruction methods often require extensive user interaction or large training datasets. Recently, Neural Radiance Field (NeRF) has successfully reconstructed high-fidelity scenes in natural and medical contexts without these requirements. However, challenges such as sparse-views, intra-scan motion, and complex vessel morphology hinder its direct application to CA data. We introduce NeRF-CA, a first step toward a fully automatic 4D CA reconstruction that achieves reconstructions from sparse coronary angiograms. To the best of our knowledge, we are the first to address the challenges of sparse-views and cardiac motion by decoupling the scene into the moving coronary artery and the static background, effectively translating the problem of motion into a strength. NeRF-CA serves as a first stepping stone for solving the 4D CA reconstruction problem, achieving adequate 4D reconstructions from as few as four angiograms, as required by clinical practice, while significantly outperforming state-of-the-art sparse-view X-ray NeRF. We validate our approach quantitatively and qualitatively using representative 4D phantom datasets and ablation studies.

Abstract The contour depth methodology enables non‐parametric summarization of contour ensembles by extracting their representatives, confidence bands, and outliers for visualization (via contour boxplots) and robust downstream procedures. We address two shortcomings of these methods. Firstly, we significantly expedite the computation and recomputation of Inclusion Depth (ID), introducing a linear‐time algorithm for epsilon ID, a variant used for handling ensembles with contours with multiple intersections. We also present the inclusion matrix, which contains the pairwise inclusion relationships between contours, and leverage it to accelerate the recomputation of ID. Secondly, extending beyond the single distribution assumption, we present the Relative Depth (ReD), a generalization of contour depth for ensembles with multiple modes. Building upon the linear‐time eID, we introduce CDclust, a clustering algorithm that untangles ensemble modes of variation by optimizing ReD. Synthetic and real datasets from medical image segmentation and meteorological forecasting showcase the speed advantages, illustrate the use case of progressive depth computation and enable non‐parametric multimodal analysis. To promote research and adoption, we offer the contour‐depth Python package.

Genomics researchers increasingly use multiple reference genomes to comprehensively explore genetic variants underlying differences in detectable characteristics between organisms. Pangenomes allow for an efficient data representation of multiple related genomes and their associated metadata. However, current visual analysis approaches for exploring these complex genotype-phenotype relationships are often based on single reference approaches or lack adequate support for interpreting the variants in the genomic context with heterogeneous (meta)data. This design study introduces PanVA, a visual analytics design for pangenomic variant analysis developed with the active participation of genomics researchers. The design uniquely combines tailored visual representations with interactions such as sorting, grouping, and aggregation, allowing users to navigate and explore different perspectives on complex genotype-phenotype relations. Through evaluation in the context of plants and pathogen research, we show that PanVA helps researchers explore variants in genes and generate hypotheses about their role in phenotypic variation.

Ensembles of contours arise in various applications like simulation, computer-aided design, and semantic segmentation. Uncovering ensemble patterns and analyzing individual members is a challenging task that suffers from clutter. Ensemble statistical summarization can alleviate this issue by permitting analyzing ensembles' distributional components like the mean and median, confidence intervals, and outliers. Contour boxplots, powered by Contour Band Depth (CBD), are a popular non-parametric ensemble summarization method that benefits from CBD's generality, robustness, and theoretical properties. In this work, we introduce Inclusion Depth (ID), a new notion of contour depth with three defining characteristics. First, ID is a generalization of functional Half-Region Depth, which offers several theoretical guarantees. Second, ID relies on a simple principle: the inside/outside relationships between contours. This facilitates implementing ID and understanding its results. Third, the computational complexity of ID scales quadratically in the number of members of the ensemble, improving CBD's cubic complexity. This also in practice speeds up the computation enabling the use of ID for exploring large contour ensembles or in contexts requiring multiple depth evaluations like clustering. In a series of experiments on synthetic data and case studies with meteorological and segmentation data, we evaluate ID's performance and demonstrate its capabilities for the visual analysis of contour ensembles.

Data features and class probabilities are two main perspectives when, e.g., evaluating model results and identifying problematic items. Class probabilities represent the likelihood that each instance belongs to a particular class, which can be produced by probabilistic classifiers or even human labeling with uncertainty. Since both perspectives are multi-dimensional data, dimensionality reduction (DR) techniques are commonly used to extract informative characteristics from them. However, existing methods either focus solely on the data feature perspective or rely on class probability estimates to guide the DR process. In contrast to previous work where separate views are linked to conduct the analysis, we propose a novel approach, class-constrained t-SNE, that combines data features and class probabilities in the same DR result. Specifically, we combine them by balancing two corresponding components in a cost function to optimize the positions of data points and iconic representation of classes - class landmarks. Furthermore, an interactive user-adjustable parameter balances these two components so that users can focus on the weighted perspectives of interest and also empowers a smooth visual transition between varying perspectives to preserve the mental map. We illustrate its application potential in model evaluation and visual-interactive labeling. A comparative analysis is performed to evaluate the DR results.

In recent years, visual analytics (VA) has shown promise in alleviating the challenges of interpreting black-box deep learning (DL) models. While the focus of VA for explainable DL has been mainly on classification problems, DL is gaining popularity in high-dimensional-to-high-dimensional (H-H) problems such as image-to-image translation. In contrast to classification, H-H problems have no explicit instance groups or classes to study. Each output is continuous, high-dimensional, and changes in an unknown non-linear manner with changes in the input. These unknown relations between the input, model and output necessitate the user to analyze them in conjunction, leveraging symmetries between them. Since classification tasks do not exhibit some of these challenges, most existing VA systems and frameworks allow limited control of the components required to analyze models beyond classification. Hence, we identify the need for and present a unified conceptual framework, the Transform-and-Perform framework (T&P), to facilitate the design of VA systems for DL model analysis focusing on H-H problems. T&P provides a checklist to structure and identify workflows and analysis strategies to design new VA systems, and understand existing ones to uncover potential gaps for improvements. The goal is to aid the creation of effective VA systems that support the structuring of model understanding and identifying actionable insights for model improvements. We highlight the growing need for new frameworks like T&P with a real-world image-to-image translation application. We illustrate how T&P effectively supports the understanding and identification of potential gaps in existing VA systems.

Deep learning (DL) models have shown performance benefits across many applications, from classification to image-to-image translation. However, low interpretability often leads to unexpected model behavior once deployed in the real world. Usually, this unexpected behavior is because the training data domain does not reflect the deployment data domain. Identifying a model's breaking points under input conditions and domain shifts, i.e., input transformations, is essential to improve models. Although visual analytics (VA) has shown promise in studying the behavior of model outputs under continually varying inputs, existing methods mainly focus on per-class or instance-level analysis. We aim to generalize beyond classification where classes do not exist and provide a global view of model behavior under co-occurring input transformations. We present a DL model-agnostic VA method (ProactiV) to help model developers proactively study output behavior under input transformations to identify and verify breaking points. ProactiV relies on a proposed input optimization method to determine the changes to a given transformed input to achieve the desired output. The data from this optimization process allows the study of global and local model behavior under input transformations at scale. Additionally, the optimization method provides insights into the input characteristics that result in desired outputs and helps recognize model biases. We highlight how ProactiV effectively supports studying model behavior with example classification and image-to-image translation tasks.

Exploration and analysis of high-dimensional data are important tasks in many fields that produce large and complex data, like the financial sector, systems biology, or cultural heritage. Tailor-made visual analytics software is developed for each specific application, limiting their applicability in other fields. However, as diverse as these fields are, their characteristics and requirements for data analysis are conceptually similar. Many applications share abstract tasks and data types and are often constructed with similar building blocks. Developing such applications, even when based mostly on existing building blocks, requires significant engineering efforts. We developed ManiVault, a flexible and extensible open-source visual analytics framework for analyzing high-dimensional data. The primary objective of ManiVault is to facilitate rapid prototyping of visual analytics workflows for visualization software developers and practitioners alike. ManiVault is built using a plugin-based architecture that offers easy extensibility. While our architecture deliberately keeps plugins self-contained, to guarantee maximum flexibility and re-usability, we have designed and implemented a messaging API for tight integration and linking of modules to support common visual analytics design patterns. We provide several visualization and analytics plugins, and ManiVault's API makes the integration of new plugins easy for developers. ManiVault facilitates the distribution of visualization and analysis pipelines and results for practitioners through saving and reproducing complete application states. As such, ManiVault can be used as a communication tool among researchers to discuss workflows and results. A copy of this paper and all supplemental material is available at osf.io/9k6jw, and source code at github.com/ManiVaultStudio.

Event sequence data is a special type of time-dependent data that captures information about the order in which discrete events occur. The time-dimension is one of the factors that makes event sequence data hard to understand. Other factors that contribute to this complexity are the dimensionality in terms of the amount of events and attributes, the frequency of events in terms of consistency and density, varying durations, and parallel occurrences. When end-users need to compare event sequences, all these characteristics need to be considered and justified. In this state-of-the-art report we review visualization techniques for the event sequence comparison task. We focus specifically on comparison, in contrast to general event sequence visualization, and review how different aspects, such as the data attributes or granularity, affect the comparison. We define a taxonomy based on five dimensions: Scale, Comparison Type, Size, Data, and Visualization & Interaction. Based on these dimensions, we provide an overview of the literature of the past 17 years. This overview is accompanied with an analysis of the strengths and weaknesses of the proposed techniques. Furthermore, we identify current gaps and research challenges in the literature.

Abstract In many domains, multivariate event sequence data is collected focused around an entity (the case). Typically, each event has multiple attributes, for example, in healthcare a patient has events such as hospitalization, medication, and surgery. In addition to the multivariate events, also the case (a specific attribute, e.g., patient) has associated multivariate data (e.g., age, gender, weight). Current work typically only visualizes one attribute per event (label) in the event sequences. As a consequence, events can only be explored from a predefined case‐centric perspective. However, to find complex relations from multiple perspectives (e.g., from different case definitions, such as doctor), users also need an event‐ and attribute‐centric perspective. In addition, support is needed to effortlessly switch between and within perspectives. To support such a rich exploration, we present FlexEvent: an exploration and analysis method that enables investigation beyond a fixed case‐centric perspective. Based on an adaptation of existing visualization techniques, such as scatterplots and juxtaposed small multiples, we enable flexible switching between different perspectives to explore the multivariate event sequence data needed to answer multi‐perspective hypotheses. We evaluated FlexEvent with three domain experts in two use cases with sleep disorder and neonatal ICU data that show our method facilitates experts in exploring and analyzing real‐world multivariate sequence data from different perspectives.

Abstract During training sessions, sailors rely on feedback provided by the coaches to reinforce their skills and improve their performance. Nowadays, the incorporation of sensors on the boats enables coaches to potentially provide more informed feedback to the sailors. A common exercise during practice sessions, consists of two boats of the same class, sailing side by side in a straight line with different boat handling techniques. Coaches try to understand which techniques are that make one boat go faster than the other. The analysis of the obtained data from the boats is challenging given its multi‐dimensional, time‐varying and spatial nature. At present, coaches only rely on aggregated statistics reducing the complexity of the data, hereby losing local and temporal information. We describe a new domain characterization and present a visualization design that allows coaches to analyse the data, structuring their analysis and explore the data from different perspectives. A central element of the tool is the glyph design to intuitively represent and aggregate multiple aspects of the sensor data. We have conducted multiple user studies with naive users, sailors and coaches to evaluate the design and potential of the overall tool. image

High-dimensional imaging is becoming increasingly relevant in many fields from astronomy and cultural heritage to systems biology. Visual exploration of such high-dimensional data is commonly facilitated by dimensionality reduction. However, common dimensionality reduction methods do not include spatial information present in images, such as local texture features, into the construction of low-dimensional embeddings. Consequently, exploration of such data is typically split into a step focusing on the attribute space followed by a step focusing on spatial information, or vice versa. In this paper, we present a method for incorporating spatial neighborhood information into distance-based dimensionality reduction methods, such as t-Distributed Stochastic Neighbor Embedding (t-SNE). We achieve this by modifying the distance measure between high-dimensional attribute vectors associated with each pixel such that it takes the pixel's spatial neighborhood into account. Based on a classification of different methods for comparing image patches, we explore a number of different approaches. We compare these approaches from a theoretical and experimental point of view. Finally, we illustrate the value of the proposed methods by qualitative and quantitative evaluation on synthetic data and two real-world use cases.

Abstract Model comparison is an important process to facilitate model diagnosis, improvement, and selection when multiple models are developed for a classification task. It involves careful comparison concerning model performance and interpretation. Current visual analytics solutions often ignore the feature selection process. They either do not support detailed analysis of multiple multi‐class classifiers or rely on feature analysis alone to interpret model results. Understanding how different models make classification decisions, especially classification disagreements of the same instances, requires a deeper model understanding. We present ModelWise, a visual analytics method to compare multiple multi‐class classifiers in terms of model performance, feature space, and model explanation. ModelWise adapts visualizations with rich interactions to support multiple workflows to achieve model diagnosis, improvement, and selection. It considers feature subspaces generated for use in different models and improves model understanding by model explanation. We demonstrate the usability of ModelWise with two case studies, one with a small exemplar dataset and another developed with a machine learning expert with real‐world perioperative data.

Abstract Diffusion Tensor Imaging (DTI) is a non‐invasive magnetic resonance imaging technique that, combined with fiber tracking algorithms, allows the characterization and visualization of white matter structures in the brain. The resulting fiber tracts are used, for example, in tumor surgery to evaluate the potential brain functional damage due to tumor resection. The DTI processing pipeline from image acquisition to the final visualization is rather complex generating undesirable uncertainties in the final results. Most DTI visualization techniques do not provide any information regarding the presence of uncertainty. When planning surgery, a fixed safety margin around the fiber tracts is often used; however, it cannot capture local variability and distribution of the uncertainty, thereby limiting the informed decision‐making process. Stochastic techniques are a possibility to estimate uncertainty for the DTI pipeline. However, it has high computational and memory requirements that make it infeasible in a clinical setting. The delay in the visualization of the results adds hindrance to the workflow. We propose a progressive approach that relies on a combination of wild‐bootstrapping and fiber tracking to be used within the progressive visual analytics paradigm. We present a local bootstrapping strategy, which reduces the computational and memory costs, and provides fiber‐tracking results in a progressive manner. We have also implemented a progressive aggregation technique that computes the distances in the fiber ensemble during progressive bootstrap computations. We present experiments with different scenarios to highlight the benefits of using our progressive visual analytic pipeline in a clinical workflow along with a use case and analysis obtained by discussions with our collaborators.

Nowadays video plays an important role in the coaching of athletes across many different sports. For sailing training sessions, the videos are recorded from the coach boat and provide ways to review and analyze the training sessions aiming at improving the sailors performance. On one hand, videos are commonly recorded with handheld devices that are rather cumbersome to acquire and are prone to missing many important moments. On the other hand, recordings of entire sessions with an integrated camera in the coach boat are difficult to analyze given their length and the lack of image stability. We present a pipeline to facilitate the visual analysis of these full session videos. We extract manoeuvres as interesting segments, and provide a visualization framework to present the video and processed data. Manoeuvres are extracted by detecting and tracking the boat and sailors. With the visualization tool, the user can locate and visually inspect those manoeuvres for coaching tasks. We evaluated the potential of the framework and from the results we conclude that the manoeuvre detection is reasonably accurate and some coaches see potential in the presented framework.

Abstract Phase‐Contrast Magnetic Resonance Imaging (PC‐MRI) surpasses all other imaging methods in quality and completeness for measuring time‐varying volumetric blood flows and has shown potential to improve both diagnosis and risk assessment of cardiovascular diseases. However, like any measurement of physical phenomena, the data are prone to noise, artefacts and has a limited resolution. Therefore, PC‐MRI data itself do not fulfil physics fluid laws making it difficult to distinguish important flow features. For data analysis, physically plausible and high‐resolution data are required. Computational fluid dynamics provides high‐resolution physically plausible flows. However, the flow is inherently coupled to the underlying anatomy and boundary conditions, which are difficult or sometimes even impossible to adequately model with current techniques. We present a novel methodology using data assimilation techniques for PC‐MRI noise and artefact removal, generating physically plausible flow close to the measured data. It also allows us to increase the spatial and temporal resolution. To avoid sensitivity to the anatomical model, we consider and update the full 3D velocity field. We demonstrate our approach using phantom data with various amounts of induced noise and show that we can improve the data while preserving important flow features, without the need of a highly detailed model of the anatomy.

In recent years the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm has become one of the most used and insightful techniques for exploratory data analysis of high-dimensional data. It reveals clusters of high-dimensional data points at different scales while only requiring minimal tuning of its parameters. However, the computational complexity of the algorithm limits its application to relatively small datasets. To address this problem, several evolutions of t-SNE have been developed in recent years, mainly focusing on the scalability of the similarity computations between data points. However, these contributions are insufficient to achieve interactive rates when visualizing the evolution of the t-SNE embedding for large datasets. In this work, we present a novel approach to the minimization of the t-SNE objective function that heavily relies on graphics hardware and has linear computational complexity. Our technique decreases the computational cost of running t-SNE on datasets by orders of magnitude and retains or improves on the accuracy of past approximated techniques. We propose to approximate the repulsive forces between data points by splatting kernel textures for each data point. This approximation allows us to reformulate the t-SNE minimization problem as a series of tensor operations that can be efficiently executed on the graphics card. An efficient implementation of our technique is integrated and available for use in the widely used Google TensorFlow.js, and an open-source C++ library.

Abstract Hierarchical embeddings, such as HSNE, address critical visual and computational scalability issues of traditional techniques for dimensionality reduction. The improved scalability comes at the cost of the need for increased user interaction for exploration. In this paper, we provide a solution for the interactive visual Focus+Context exploration of such embeddings. We explain how to integrate embedding parts from different levels of detail, corresponding to focus and context groups, in a joint visualization. We devise an according interaction model that relates typical semantic operations on a Focus+Context visualization with the according changes in the level‐of‐detail‐hierarchy of the embedding, including also a mode for comparative Focus+Context exploration and extend HSNE to incorporate the presented interaction model. In order to demonstrate the effectiveness of our approach, we present a use case based on the visual exploration of multi‐dimensional images.

Abstract A bipartite graph is a powerful abstraction for modeling relationships between two collections. Visualizations of bipartite graphs allow users to understand the mutual relationships between the elements in the two collections, e.g., by identifying clusters of similarly connected elements. However, commonly‐used visual representations do not scale for the analysis of large bipartite graphs containing tens of millions of vertices, often resorting to an a‐priori clustering of the sets. To address this issue, we present the Who's‐Active‐On‐What‐Visualization (WAOW‐Vis) that allows for multiscale exploration of a bipartite social‐network without imposing an a‐priori clustering. To this end, we propose to treat a bipartite graph as a high‐dimensional space and we create the WAOW‐Vis adapting the multiscale dimensionality‐reduction technique HSNE. The application of HSNE for bipartite graph requires several modifications that form the contributions of this work. Given the nature of the problem, a set‐based similarity is proposed. For efficient and scalable computations, we use compressed bitmaps to represent sets and we present a novel space partitioning tree to efficiently compute similarities; the Sets Intersection Tree. Finally, we validate WAOW‐Vis on several datasets connecting Twitter‐users and ‐streams in different domains: news, computer science and politics. We show how WAOW‐Vis is particularly effective in identifying hierarchies of communities among social‐media users.

Abstract We present theBladder Runner, a novel tool to enable detailed visual exploration and analysis of the impact of bladder shape variation on the accuracy of dose delivery, during the course of prostate cancer radiotherapy (RT). Our tool enables the investigation of individual patients and cohorts through the entire treatment process, and it can give indications of RT‐induced complications for the patient. In prostate cancer RT treatment, despite the design of an initial plan prior to dose administration, bladder toxicity remains very common. The main reason is that the dose is delivered in multiple fractions over a period of weeks, during which, the anatomical variation of the bladder – due to differences in urinary filling – causes deviations between planned and delivered doses. Clinical researchers want to correlate bladder shape variations to dose deviations and toxicity risk through cohort studies, to understand which specific bladder shape characteristics are more prone to side effects. This is currently done with Dose‐Volume Histograms (DVHs), which provide limited, qualitative insight. The effect of bladder variation on dose delivery and the resulting toxicity cannot be currently examined with the DVHs. To address this need, we designed and implemented the Bladder Runner, which incorporates visualization strategies in a highly interactive environment with multiple linked views. Individual patients can be explored and analyzed through the entire treatment period, while inter‐patient and temporal exploration, analysis and comparison are also supported. We demonstrate the applicability of our presented tool with a usage scenario, employing a dataset of 29 patients followed through the course of the treatment, across 13 time points. We conducted an evaluation with three clinical researchers working on the investigation of RT‐induced bladder toxicity. All participants agreed that Bladder Runner provides better understanding and new opportunities for the exploration and analysis of the involved cohort data.

Single-cell analysis through mass cytometry has become an increasingly important tool for immunologists to study the immune system in health and disease. Mass cytometry creates a high-dimensional description vector for single cells by time-of-flight measurement. Recently, t-Distributed Stochastic Neighborhood Embedding (t-SNE) has emerged as one of the state-of-the-art techniques for the visualization and exploration of single-cell data. Ever increasing amounts of data lead to the adoption of Hierarchical Stochastic Neighborhood Embedding (HSNE), enabling the hierarchical representation of the data. Here, the hierarchy is explored selectively by the analyst, who can request more and more detail in areas of interest. Such hierarchies are usually explored by visualizing disconnected plots of selections in different levels of the hierarchy. This poses problems for navigation, by imposing a high cognitive load on the analyst. In this work, we present an interactive summary-visualization to tackle this problem. CyteGuide guides the analyst through the exploration of hierarchically represented single-cell data, and provides a complete overview of the current state of the analysis. We conducted a two-phase user study with domain experts that use HSNE for data exploration. We first studied their problems with their current workflow using HSNE and the requirements to ease this workflow in a field study. These requirements have been the basis for our visual design. In the second phase, we verified our proposed solution in a user evaluation.

Deep neural networks are now rivaling human accuracy in several pattern recognition problems. Compared to traditional classifiers, where features are handcrafted, neural networks learn increasingly complex features directly from the data. Instead of handcrafting the features, it is now the network architecture that is manually engineered. The network architecture parameters such as the number of layers or the number of filters per layer and their interconnections are essential for good performance. Even though basic design guidelines exist, designing a neural network is an iterative trial-and-error process that takes days or even weeks to perform due to the large datasets used for training. In this paper, we present DeepEyes, a Progressive Visual Analytics system that supports the design of neural networks during training. We present novel visualizations, supporting the identification of layers that learned a stable set of patterns and, therefore, are of interest for a detailed analysis. The system facilitates the identification of problems, such as superfluous filters or layers, and information that is not being captured by the network. We demonstrate the effectiveness of our system through multiple use cases, showing how a trained network can be compressed, reshaped and adapted to different problems.

Abstract A Diffusion Tensor Imaging (DTI) group study consists of a collection of volumetric diffusion tensor datasets (i.e., an ensemble) acquired from a group of subjects. The multivariate nature of the diffusion tensor imposes challenges on the analysis and the visualization. These challenges are commonly tackled by reducing the diffusion tensors to scalar‐valued quantities that can be analyzed with common statistical tools. However, reducing tensors to scalars poses the risk of losing intrinsic information about the tensor. Visualization of tensor ensemble data without loss of information is still a largely unsolved problem. In this work, we propose an overview + detail visualization to facilitate the tensor ensemble exploration. We define an ensemble representative tensor and variations in terms of the three intrinsic tensor properties (i.e., scale, shape, and orientation) separately. The ensemble summary information is visually encoded into the newly designed aggregate tensor glyph which, in a spatial layout, functions as the overview. The aggregate tensor glyph guides the analyst to interesting areas that would need further detailed inspection. The detail views reveal the original information that is lost during aggregation. It helps the analyst to further understand the sources of variation and formulate hypotheses. To illustrate the applicability of our prototype, we compare with most relevant previous work through a user study and we present a case study on the analysis of a brain diffusion tensor dataset ensemble from healthy volunteers.

Progressive Visual Analytics aims at improving the interactivity in existing analytics techniques by means of visualization as well as interaction with intermediate results. One key method for data analysis is dimensionality reduction, for example, to produce 2D embeddings that can be visualized and analyzed efficiently. t-Distributed Stochastic Neighbor Embedding (tSNE) is a well-suited technique for the visualization of high-dimensional data. tSNE can create meaningful intermediate results but suffers from a slow initialization that constrains its application in Progressive Visual Analytics. We introduce a controllable tSNE approximation (A-tSNE), which trades off speed and accuracy, to enable interactive data exploration. We offer real-time visualization techniques, including a density-based solution and a Magic Lens to inspect the degree of approximation. With this feedback, the user can decide on local refinements and steer the approximation level during the analysis. We demonstrate our technique with several datasets, in a real-world research scenario and for the real-time analysis of high-dimensional streams to illustrate its effectiveness for interactive data analysis.

Due to the intricate relationship between the pelvic organs and vital structures, such as vessels and nerves, pelvic anatomy is often considered to be complex to comprehend. In oncological pelvic surgery, a trade-off has to be made between complete tumor resection and preserving function by preventing damage to the nerves. Damage to the autonomic nerves causes undesirable post-operative side-effects such as fecal and urinal incontinence, as well as sexual dysfunction in up to 80 percent of the cases. Since these autonomic nerves are not visible in pre-operative MRI scans or during surgery, avoiding nerve damage during such a surgical procedure becomes challenging. In this work, we present visualization methods to represent context, target, and risk structures for surgical planning. We employ distance-based and occlusion management techniques in an atlas-based surgical planning tool for oncological pelvic surgery. Patient-specific pre-operative MRI scans are registered to an atlas model that includes nerve information. Through several interactive linked views, the spatial relationships and distances between the organs, tumor and risk zones are visualized to improve understanding, while avoiding occlusion. In this way, the surgeon can examine surgically relevant structures and plan the procedure before going into the operating theater, thus raising awareness of the autonomic nerve zone regions and potentially reducing post-operative complications. Furthermore, we present the results of a domain expert evaluation with surgical oncologists that demonstrates the advantages of our approach.

Abstract To understand how the immune system works, one needs to have a clear picture of its cellular compositon and the cells' corresponding properties and functionality. Mass cytometry is a novel technique to determine the properties of single‐cells with unprecedented detail. This amount of detail allows for much finer differentiation but also comes at the cost of more complex analysis. In this work, we present Cytosplore, implementing an interactive workflow to analyze mass cytometry data in an integrated system, providing multiple linked views, showing different levels of detail and enabling the rapid definition of known and unknown cell types. Cytosplore handles millions of cells, each represented as a high‐dimensional data point, facilitates hypothesis generation and confirmation, and provides a significant speed up of the current workflow. We show the effectiveness of Cytosplore in a case study evaluation.

Abstract In recent years, dimensionality‐reduction techniques have been developed and are widely used for hypothesis generation in Exploratory Data Analysis. However, these techniques are confronted with overcoming the trade‐off between computation time and the quality of the provided dimensionality reduction. In this work, we address this limitation, by introducing Hierarchical Stochastic Neighbor Embedding (Hierarchical‐SNE). Using a hierarchical representation of the data, we incorporate the well‐known mantra of Overview‐First, Details‐On‐Demand in non‐linear dimensionality reduction. First, the analysis shows an embedding, that reveals only the dominant structures in the data (Overview). Then, by selecting structures that are visible in the overview, the user can filter the data and drill down in the hierarchy. While the user descends into the hierarchy, detailed visualizations of the high‐dimensional structures will lead to new insights. In this paper, we explain how Hierarchical‐SNE scales to the analysis of big datasets. In addition, we show its application potential in the visualization of Deep‐Learning architectures and the analysis of hyperspectral images.

Abstract In radiotherapy, tumors are irradiated with a high dose, while surrounding healthy tissues are spared. To quantify the probability that a tumor is effectively treated with a given dose, statistical models were built and employed in clinical research. These are called tumor control probability (TCP) models. Recently, TCP models started incorporating additional information from imaging modalities. In this way, patient‐specific properties of tumor tissues are included, improving the radiobiological accuracy of models. Yet, the employed imaging modalities are subject to uncertainties with significant impact on the modeling outcome, while the models are sensitive to a number of parameter assumptions. Currently, uncertainty and parameter sensitivity are not incorporated in the analysis, due to time and resource constraints. To this end, we propose a visual tool that enables clinical researchers working on TCP modeling, to explore the information provided by their models, to discover new knowledge and to confirm or generate hypotheses within their data. Our approach incorporates the following four main components: (1) It supports the exploration of uncertainty and its effect on TCP models; (2) It facilitates parameter sensitivity analysis to common assumptions; (3) It enables the identification of inter‐patient response variability; (4) It allows starting the analysis from the desired treatment outcome, to identify treatment strategies that achieve it. We conducted an evaluation with nine clinical researchers. All participants agreed that the proposed visual tool provides better understanding and new opportunities for the exploration and analysis of TCP modeling.

We present the first visualization tool that combines pathlines from blood flow and wall thickness information. Our method uses illustrative techniques to provide occlusion-free visualization of the flow. We thus offer medical researchers an effective visual analysis tool for aneurysm treatment risk assessment. Such aneurysms bear a high risk of rupture and significant treatment-related risks. Therefore, to get a fully informed decision it is essential to both investigate the vessel morphology and the hemodynamic data. Ongoing research emphasizes the importance of analyzing the wall thickness in risk assessment. Our combination of blood flow visualization and wall thickness representation is a significant improvement for the exploration and analysis of aneurysms. As all presented information is spatially intertwined, occlusion problems occur. We solve these occlusion problems by dynamic cutaway surfaces. We combine this approach with a glyph-based blood flow representation and a visual mapping of wall thickness onto the vessel surface. We developed a GPU-based implementation of our visualizations which facilitates wall thickness analysis through real-time rendering and flexible interactive data exploration mechanisms. We designed our techniques in collaboration with domain experts, and we provide details about the evaluation of the technique and tool.

Parallel Coordinate Plots (PCPs) is one of the most powerful techniques for the visualization of multivariate data. However, for large datasets, the representation suffers from clutter due to overplotting. In this case, discerning the underlying data information and selecting specific interesting patterns can become difficult. We propose a new and simple technique to improve the display of PCPs by emphasizing the underlying data structure. Our Orientation-enhanced Parallel Coordinate Plots (OPCPs) improve pattern and outlier discernibility by visually enhancing parts of each PCP polyline with respect to its slope. This enhancement also allows us to introduce a novel and efficient selection method, the Orientation-enhanced Brushing (O-Brushing). Our solution is particularly useful when multiple patterns are present or when the view on certain patterns is obstructed by noise. We present the results of our approach with several synthetic and real-world datasets. Finally, we conducted a user evaluation, which verifies the advantages of the OPCPs in terms of discernibility of information in complex data. It also confirms that O-Brushing eases the selection of data patterns in PCPs and reduces the amount of necessary user interactions compared to state-of-the-art brushing techniques.

Diffusion Tensor Imaging (DTI) is a magnetic resonance imaging modality that enables the in-vivo reconstruction and visualization of fibrous structures. To inspect the local and individual diffusion tensors, glyph-based visualizations are commonly used since they are able to effectively convey full aspects of the diffusion tensor. For several applications it is necessary to compare tensor fields, e.g., to study the effects of acquisition parameters, or to investigate the influence of pathologies on white matter structures. This comparison is commonly done by extracting scalar information out of the tensor fields and then comparing these scalar fields, which leads to a loss of information. If the glyph representation is kept, simple juxtaposition or superposition can be used. However, neither facilitates the identification and interpretation of the differences between the tensor fields. Inspired by the checkerboard style visualization and the superquadric tensor glyph, we design a new glyph to locally visualize differences between two diffusion tensors by combining juxtaposition and explicit encoding. Because tensor scale, anisotropy type, and orientation are related to anatomical information relevant for DTI applications, we focus on visualizing tensor differences in these three aspects. As demonstrated in a user study, our new glyph design allows users to efficiently and effectively identify the tensor differences. We also apply our new glyphs to investigate the differences between DTI datasets of the human brain in two different contexts using different b-values, and to compare datasets from a healthy and HIV-infected subject.

Abstract Tumors are heterogeneous tissues consisting of multiple regions with distinct characteristics. Characterization of these intra‐tumor regions can improve patient diagnosis and enable a better targeted treatment. Ideally, tissue characterization could be performed non‐invasively, using medical imaging data, to derive per voxel a number of features, indicative of tissue properties. However, the high dimensionality and complexity of this imaging‐derived feature space is prohibiting for easy exploration and analysis ‐ especially when clinical researchers require to associate observations from the feature space to other reference data, e.g., features derived from histopathological data. Currently, the exploratory approach used in clinical research consists of juxtaposing these data, visually comparing them and mentally reconstructing their relationships. This is a time consuming and tedious process, from which it is difficult to obtain the required insight. We propose a visual tool for: (1) easy exploration and visual analysis of the feature space of imaging‐derived tissue characteristics and (2) knowledge discovery and hypothesis generation and confirmation, with respect to reference data used in clinical research. We employ, as central view, a 2D embedding of the imaging‐derived features. Multiple linked interactive views provide functionality for the exploration and analysis of the local structure of the feature space, enabling linking to patient anatomy and clinical reference data. We performed an initial evaluation with ten clinical researchers. All participants agreed that, unlike current practice, the proposed visual tool enables them to identify, explore and analyze heterogeneous intra‐tumor regions and particularly, to generate and confirm hypotheses, with respect to clinical reference data.

Abstract Modern MRI measurements deliver volumetric and time‐varying blood‐flow data of unprecedented quality. Visual analysis of these data potentially leads to a better diagnosis and risk assessment of various cardiovascular diseases. Recent advances have improved the speed and quality of the imaging data considerably. Nevertheless, the data remains compromised by noise and a lack of spatiotemporal resolution. Besides imaging data, also numerical simulations are employed. These are based on mathematical models of specific features of physical reality. However, these models require realistic parameters and boundary conditions based on measurements. We propose to use data assimilation to bring measured data and physically‐based simulation together, and to harness the mutual benefits. The accuracy and noise robustness of the coupled approach is validated using an analytic flow field. Furthermore, we present a comparative visualization that conveys the differences between using conventional interpolation and our coupled approach.

Diffusion Tensor Imaging (DTI) and fiber tracking provide unique insight into the 3D structure of fibrous tissues in the brain. However, the output of fiber tracking contains a significant amount of uncertainty accumulated in the various steps of the processing pipeline. Existing DTI visualization methods do not present these uncertainties to the end-user. This creates a false impression of precision and accuracy that can have serious consequences in applications that rely heavily on risk assessment and decision-making, such as neurosurgery. On the other hand, adding uncertainty to an already complex visualization can easily lead to information overload and visual clutter. In this work, we propose Illustrative Confidence Intervals to reduce the complexity of the visualization and present only those aspects of uncertainty that are of interest to the user. We look specifically at the uncertainty in fiber shape due to noise and modeling errors. To demonstrate the flexibility of our framework, we compute this uncertainty in two different ways, based on (1) fiber distance and (2) the probability of a fiber connection between two brain regions. We provide the user with interactive tools to define multiple confidence intervals, specify visual styles and explore the uncertainty with a Focus+Context approach. Finally, we have conducted a user evaluation with three neurosurgeons to evaluate the added value of our visualization.

Abstract Advancements in the acquisition and modeling of flow fields result in unsteady volumetric flow fields of unprecedented quality. An important example is found in the analysis of unsteady blood‐flow data. Preclinical research strives for a better understanding of correlations between the hemodynamics and the progression of cardiovascular diseases. Modern‐day computer models and MRI acquisition provide time‐resolved volumetric blood‐flow velocity fields. Unfortunately, these fields often remain unexplored, as high‐dimensional data are difficult to conceive. We present a spatiotemporal, i.e., four‐dimensional, hierarchical clustering, yielding a sparse representation of the velocity data. The clustering results underpin an illustrative visualization approach, facilitating visual analysis. The hierarchy allows an intuitive level‐of‐detail selection, largely retaining important flow patterns. The clustering employs dissimilarity measures to construct the hierarchy. We have adapted two existing measures for steady vector fields for use in the spacetime domain. Because of the inherent computational complexity of the multidimensional clustering, we introduce a coarse hierarchical clustering approach, which closely approximates the full hierarchy generation, and considerably improves the performance. The resulting clusters are visualized by representative patharrows, in combination with an illustrative anatomical context. We present various seeding approaches and visualization styles, providing sparse overviews of the unsteady behavior of volumetric flow fields.

Diffusion tensor imaging (DTI) is a magnetic resonance technique used to explore anatomical fibrous structures, like brain white matter. Fiber-tracking methods use the diffusion tensor (DT) field to reconstruct the corresponding fibrous structure. A group of fiber-tracking methods trace geodesics on a Riemannian manifold whose metric is defined as a function of the DT. These methods are more robust to noise than more commonly used methods where just the main eigenvector of the DT is considered. Until now, geodesic-based methods were not able to resolve all geodesics, since they solved the Eikonal equation, and therefore were not able to deal with multivalued solutions. Our algorithm computes multivalued solutions using an Euler–Lagrange form of the geodesic equations. The multivalued solutions become relevant in regions with sharp anisotropy and complex geometries, or when the first arrival time does not describe the geodesic close to the anatomical fibrous structure. In this paper, we compare our algorithm with the commonly used Hamilton–Jacobi (HJ) equation approach. We describe and analyze the characteristics of both methods. In the analysis we show that in cases where, e.g., U-shaped bundles appear, our algorithm can capture the underlying fiber structure, while other approaches will fail. A feasibility study with results for synthetic and real data is shown.

Cerebral aneurysms are a pathological vessel dilatation that bear a high risk of rupture. For the understanding and evaluation of the risk of rupture, the analysis of hemodynamic information plays an important role. Besides quantitative hemodynamic information, also qualitative flow characteristics, e.g., the inflow jet and impingement zone are correlated with the risk of rupture. However, the assessment of these two characteristics is currently based on an interactive visual investigation of the flow field, obtained by computational fluid dynamics (CFD) or blood flow measurements. We present an automatic and robust detection as well as an expressive visualization of these characteristics. The detection can be used to support a comparison, e.g., of simulation results reflecting different treatment options. Our approach utilizes local streamline properties to formalize the inflow jet and impingement zone. We extract a characteristic seeding curve on the ostium, on which an inflow jet boundary contour is constructed. Based on this boundary contour we identify the impingement zone. Furthermore, we present several visualization techniques to depict both characteristics expressively. Thereby, we consider accuracy and robustness of the extracted characteristics, minimal visual clutter and occlusions. An evaluation with six domain experts confirms that our approach detects both hemodynamic characteristics reasonably.

Abstract-Any sufficiently smooth, positive, real-valued function ψ : S(2) → K+ on a sphere S(2) can be expanded by a Laplace expansion into a sum of spherical harmonics. Given the Laplace expansion coefficients, we provide a CPU and GPU-based algorithm that renders the radial graph of ψ in a fast and efficient way by ray-casting the glyph of ψ in the fragment shader of a GPU. The proposed rendering algorithm has proven highly useful in the visualization of high angular resolution diffusion imaging (HARDI) data. Our implementation of the rendering algorithm can display simultaneously thousands of glyphs depicting the local diffusivity of water. The rendering is fast enough to allow for interactive manipulation of large HARDI data sets.

Better understanding of hemodynamics conceivably leads to improved diagnosis and prognosis of cardiovascular diseases. Therefore, an elaborate analysis of the blood-flow in heart and thoracic arteries is essential. Contemporary MRI techniques enable acquisition of quantitative time-resolved flow information, resulting in 4D velocity fields that capture the blood-flow behavior. Visual exploration of these fields provides comprehensive insight into the unsteady blood-flow behavior, and precedes a quantitative analysis of additional blood-flow parameters. The complete inspection requires accurate segmentation of anatomical structures, encompassing a time-consuming and hard-to-automate process, especially for malformed morphologies. We present a way to avoid the laborious segmentation process in case of qualitative inspection, by introducing an interactive virtual probe. This probe is positioned semi-automatically within the blood-flow field, and serves as a navigational object for visual exploration. The difficult task of determining position and orientation along the view-direction is automated by a fitting approach, aligning the probe with the orientations of the velocity field. The aligned probe provides an interactive seeding basis for various flow visualization approaches. We demonstrate illustration-inspired particles, integral lines and integral surfaces, conveying distinct characteristics of the unsteady blood-flow. Lastly, we present the results of an evaluation with domain experts, valuing the practical use of our probe and flow visualization techniques.

High-angular resolution diffusion imaging (HARDI) is a diffusion weighted MRI technique that overcomes some of the decisive limitations of its predecessor, diffusion tensor imaging (DTI), in the areas of composite nerve fiber structure. Despite its advantages, HARDI raises several issues: complex modeling of the data, nonintuitive and computationally demanding visualization, inability to interactively explore and transform the data, etc. To overcome these drawbacks, we present a novel, multifield visualization framework that adopts the benefits of both DTI and HARDI. By applying a classification scheme based on HARDI anisotropy measures, the most suitable model per imaging voxel is automatically chosen. This classification allows simplification of the data in areas with single fiber bundle coherence. To accomplish fast and interactive visualization for both HARDI and DTI modalities, we exploit the capabilities of modern GPUs for glyph rendering and adopt DTI fiber tracking in suitable regions. The resulting framework, allows user-friendly data exploration of fused HARDI and DTI data. Many incorporated features such as sharpening, normalization, maxima enhancement and different types of color coding of the HARDI glyphs, simplify the data and enhance its features. We provide a qualitative user evaluation that shows the potentials of our visualization tools in several HARDI applications.

Abstract Diffusion Tensor Imaging (DTI) has made feasible the visualization of the fibrous structure of the brain white matter. In the last decades, several fiber‐tracking methods have been developed to reconstruct the fiber tracts from DTI data. Usually these fiber tracts are shown individually based on some selection criteria like region of interest. However, if the white matter as a whole is being visualized clutter is generated by directly rendering the individual fiber tracts. Often users are actually interested in fiber bundles, anatomically meaningful entities that abstract from the fibers they contain. Several clustering techniques have been developed that try to group the fiber tracts in fiber bundles. However, even if clustering succeeds, the complex nature of white matter still makes it difficult to investigate. In this paper, we propose the use of illustration techniques to ease the exploration of white matter clusters. We create a technique to visualize an individual cluster as a whole. The amount of fibers visualized for the cluster is reduced to just a few hint lines, and silhouette and contours are used to improve the definition of the cluster borders. Multiple clusters can be easily visualized by a combination of the single cluster visualizations. Focus+context concepts are used to extend the multiple‐cluster renderings. Exploded views ease the exploration of the focus cluster while keeping the context clusters in an abstract form. Real‐time results are achieved by the GPU implementation of the presented techniques.

Insight into the dynamics of blood-flow considerably improves the understanding of the complex cardiovascular system and its pathologies. Advances in MRI technology enable acquisition of 4D blood-flow data, providing quantitative blood-flow velocities over time. The currently typical slice-by-slice analysis requires a full mental reconstruction of the unsteady blood-flow field, which is a tedious and highly challenging task, even for skilled physicians. We endeavor to alleviate this task by means of comprehensive visualization and interaction techniques. In this paper we present a framework for pre-clinical cardiovascular research, providing tools to both interactively explore the 4D blood-flow data and depict the essential blood-flow characteristics. The framework encompasses a variety of visualization styles, comprising illustrative techniques as well as improved methods from the established field of flow visualization. Each of the incorporated styles, including exploded planar reformats, flow-direction highlights, and arrow-trails, locally captures the blood-flow dynamics and may be initiated by an interactively probed vessel cross-section. Additionally, we present the results of an evaluation with domain experts, measuring the value of each of the visualization styles and related rendering parameters.

Illustrative techniques are generally applied to produce stylized renderings. Various illustrative styles have been applied to volumetric data sets, producing clearer images and effectively conveying visual information. We adopt particle systems to produce user-configurable stylized renderings from the volume data, imitating traditional pen-and-ink drawings. In the following, we present an interactive GPU-based illustrative volume rendering framework, called VolFliesGPU. In this framework, isosurfaces are sampled by evenly distributed particle sets, delineating surface shape by illustrative styles. The appearance of these styles is based on locally-measured surface properties. For instance, hatches convey surface shape by orientation and shape characteristics are enhanced by color, mapped using a curvature-based transfer function. Hidden-surfaces are generally removed to avoid visual clutter, after that a combination of styles is applied per isosurface. Multiple surfaces and styles can be explored interactively, exploiting parallelism in both graphics hardware and particle systems. We achieve real-time interaction and prompt parametrization of the illustrative styles, using an intuitive GPGPU paradigm that delivers the computational power to drive our particle system and visualization algorithms.

High angular resolution diffusion imaging (HARDI) is an emerging magnetic resonance imaging (MRI) technique that overcomes some decisive limitations of its predecessor diffusion tensor imaging (DTI). HARDI can resolve locally more than one direction in the diffusion pattern of water molecules and thereby opens up the opportunity to display and track crossing fibers. Showing the local structure of the reconstructed, angular probability profiles in a fast, detailed, and interactive way can improve the quality of the research in this area and help to move it into clinical application. In this paper we present a novel approach for HARDI glyph visualization or, more generally, for the visualization of any function that resides on a sphere and that can be expressed by a Laplace series. Our GPU-accelerated glyph rendering improves the performance of the traditional way of HARDI glyph visualization as well as the visual quality of the reconstructed data, thus offering interactive HARDI data exploration of the local structure of the white brain matter in-vivo. In this paper we exploit the capabilities of modern GPUs to overcome the large, processor-intensive and memory-consuming data visualization.

Abstract The heart consists of densely packed muscle fibres. The orientation of these fibres can be acquired by using Diffusion Tensor Imaging (DTI) ex vivo. A good way to visualize the fibre structure in a cross section of the heart is by showing short line segments originating from the cross section and aligned with the local direction of the fibres. If the line segments are placed dense enough, one can see how the fibre orientations change. However, generation of the line segments takes time and thus the user has to wait for new geometry to be generated when the plane defining the cross section is changed. We present a new direct rendering method for the visualization of the 3D vector field in a 2D user‐definable cross section of a heart. On the intersection of the plane with the vector field, the full 3D vectors are rendered as 3D line segments with a local ray casting approach. No preprocessing of the data is needed and no geometry is generated. This technique allows a fast inspection of the data to identify interesting areas where further analysis is necessary (e.g. quantification or generation of streamlines). We also show how the technique is generalized to other glyph shapes than line segments by implementing ellipsoids.

Pulmonary embolism (PE) is a common life-threatening disorder for which an early diagnosis is desirable. We propose a new system for the automatic detection of PE in contrast-enhanced CT images. The system consists of candidate detection, feature computation and classification. Candidate detection focuses on the inclusion of PE--even complete occlusions--and the exclusion of false detections, such as tissue and parenchymal diseases. Feature computation does not only focus on the intensity, shape and size of an embolus, but also on locations and the shape of the pulmonary vascular tree. Several classifiers have been tested and the results show that the performance is optimized by using a bagged tree classifier with two features based on the shape of a blood vessel and the distance to the vessel boundary. The system was trained on 38 CT data sets. Evaluation on 19 other data sets showed that the system generalizes well. The sensitivity of our system on the evaluation data is 63% at 4.9 false positives per data set, which allowed the radiologist to improve the number of detected PE by 22%.

Fiber tracking of Diffusion Tensor Imaging (DTI) data offers a unique insight into the three-dimensional organisation of white matter structures in the living brain. However, fiber tracking algorithms require a number of user-defined input parameters that strongly affect the output results. Usually the fiber tracking parameters are set once and are then re-used for several patient datasets. However, the stability of the chosen parameters is not evaluated and a small change in the parameter values can give very different results. The user remains completely unaware of such effects. Furthermore, it is difficult to reproduce output results between different users. We propose a visualization tool that allows the user to visually explore how small variations in parameter values affect the output of fiber tracking. With this knowledge the user cannot only assess the stability of commonly used parameter values but also evaluate in a more reliable way the output results between different patients. Existing tools do not provide such information. A small user evaluation of our tool has been done to show the potential of the technique.

Visually assessing the effect of the coronary artery anatomy on the perfusion of the heart muscle in patients with coronary artery disease remains a challenging task. We explore the feasibility of visualizing this effect on perfusion using a numerical approach. We perform a computational simulation of the way blood is perfused throughout the myocardium purely based on information from a three-dimensional anatomical tomographic scan. The results are subsequently visualized using both three-dimensional visualizations and bull's eye plots, partially inspired by approaches currently common in medical practice. Our approach results in a comprehensive visualization of the coronary anatomy that compares well to visualizations commonly used for other scanning technologies. We demonstrate techniques giving detailed insight in blood supply, coronary territories and feeding coronary arteries of a selected region. We demonstrate the advantages of our approach through visualizations that show information which commonly cannot be directly observed in scanning data, such as a separate visualization of the supply from each coronary artery. We thus show that the results of a computational simulation can be effectively visualized and facilitate visually correlating these results to for example perfusion data.

We present novel, comprehensive visualization techniques for the diagnosis of patients with Coronary Artery Disease using segmented cardiac MRI data. We extent an accepted medical visualization technique called the bull's eye plot by removing discontinuities, preserving the volumetric nature of the left ventricular wall and adding anatomical context. The resulting volumetric bull's eye plot can be used for the assessment of transmurality. We link these visualizations to a 3D view that presents viability information in a detailed anatomical context. We combine multiple MRI scans (whole heart anatomical data, late enhancement data) and multiple segmentations (polygonal heart model, late enhancement contours, coronary artery tree). By selectively combining different rendering techniques we obtain comprehensive yet intuitive visualizations of the various data sources.

This paper discusses two applications of probing dense volumetric data for MR orthopedics and dynamic contrast enhanced MRI mammography. In order not to reduce the context information and to extract the essential part of the data, we apply Profile Flags. A Profile Flag is a 3D glyph for probing and annotating the volumetric data. The first application area deals with visualization of T2 profiles for interactive inspection of knee cartilage and detection of lesions. In the second application, we present the usability the Profile Flags for measuring of time-signal profiles for a set of time-dependent MR volumes. Several extensions of the basic Profile Flag concept are described in detail and discussed. These extensions include selection of a set of profiles based on spatial as well as curve differences, automatic positioning of the Profile Flags, and adaptation for probing of time-varying volumetric data. Additionally, we include the evaluation of the used methods by our medical partners.

Transfer function design plays a crucial role in direct volume rendering. Furthermore, it has a major influence on the efficiency of the visualization process. We have developed a framework that facilitates the semi-automatic design of transfer functions. Similarly to other approaches we generate clusters in the transfer function domain. We created a real-time interaction with a hierarchy of clusters. This interaction effectively substitutes cumbersome settings of clustering thresholds. Our framework is also able to easily combine different clustering criteria. We have developed two similarity measures for clustering of material boundaries. One is based on the similarity of the boundaries in the transfer function domain and the other on their spatial relation. We use the LH space as the transfer function domain. This space facilitates the clustering of material boundaries. We demonstrate our approach on several examples.

A crucial step in volume rendering is the design of transfer functions that will highlight those aspects of the volume data that are of interest to the user. For many applications, boundaries carry most of the relevant information. Reliable detection of boundaries is often hampered by limitations of the imaging process, such as blurring and noise. We present a method to identify the materials that form the boundaries. These materials are then used in a new domain that facilitates interactive and semiautomatic design of appropriate transfer functions. We also show how the obtained boundary information can be used in region-growing-based segmentation.

This paper describes a tool for the visualization of T/sub 2/ maps of knee cartilage. Given the anatomical scan, and the T/sub 2/ map of the cartilage, we combine the information on the shape and the quality of the cartilage in a single image. The Profile Flag is an intuitive 3D glyph for probing and annotating of the underlying data. It comprises a bulletin board pin-like shape with a small flag on top of it. While moving the glyph along the reconstructed surface of an object, the curve data measured along the pin's needle and in its neighborhood are shown on the flag. The application area of the Profile Flag is manifold, enabling the visualization of profile data of dense but in-homogeneous objects. Furthermore, it extracts the essential part of the data without removing or even reducing the context information. By sticking Profile Flags into the investigated structure, one or more significant locations can be annotated by showing the local characteristics of the data at that locations. In this paper we are demonstrating the properties of the tool by visualizing T/sub 2/ maps of knee cartilage.

Fiber tracking is a standard approach for the visualization of the results of diffusion tensor imaging (DTI). If fibers are reconstructed and visualized individually through the complete white matter, the display gets easily cluttered making it difficult to get insight in the data. Various clustering techniques have been proposed to automatically obtain bundles that should represent anatomical structures, but it is unclear which clustering methods and parameter settings give the best results. We propose a framework to validate clustering methods for white-matter fibers. Clusters are compared with a manual classification which is used as a ground truth. For the quantitative evaluation of the methods, we developed a new measure to assess the difference between the ground truth and the clusterings. The measure was validated and calibrated by presenting different clusterings to physicians and asking them for their judgement. We found that the values of our new measure for different clusterings match well with the opinions of physicians. Using this framework, we have evaluated different clustering algorithms, including shared nearest neighbor clustering, which has not been used before for this purpose. We found that the use of hierarchical clustering using single-link and a fiber similarity measure based on the mean distance between fibers gave the best results.

Conventional edge-detection methods suffer from the dislocation of curved surfaces due to the PSF. We propose a new method that uses the isophote curvature to circumvent this. It is accurate for objects with locally constant curvature, even for small objects (like blood vessels) and in the presence of noise.

This work describes a method to visualize the thickness of curved thin objects. Given the MRI volume data of articular cartilage, medical doctors investigate pathological changes of the thickness. Since the tissue is very thin, it is impossible to reliably map the thickness information by direct volume rendering. Our idea is based on unfolding of such structures preserving their thickness. This allows to perform anisotropic geometrical operations (e.g., scaling the thickness). However, flattening of a curved structure implies a distortion of its surface. The distortion problem is alleviated through a focus-and-context minimization approach. Distortion is smallest close to a focal point which can be interactively selected by the user.

We report on using computed tomography (CT) as a model acquisition tool for complex objects in computer graphics. Unlike other modeling and scanning techniques the complexity of the object is irrelevant in CT, which naturally enables to model objects with, for example, concavities, holes, twists or fine surface details. Once the data is scanned, one can apply post-processing techniques for data enhancement, modification or presentation. For demonstration purposes we chose to scan a Christmas tree which exhibits high complexity which is difficult or even impossible to handle with other techniques. However, care has to be taken to achieve good scanning results with CT. Further, we illustrate post-processing by means of data segmentation and photorealistic as well as non-photorealistic surface and volume rendering techniques.

The majority of virtual endoscopy techniques tries to simulate a real endoscopy. A real endoscopy does not always give the optimal information due to the physical limitations it is subject to. In this paper, we deal with the unfolding of the surface of the colon as a possible visualization technique for diagnosis and polyp detection. A new two-step technique is presented which deals with the problems of double appearance of polyps and nonuniform sampling that other colon unfolding techniques suffer from. In the first step, a distance map from a central path induces nonlinear rays for unambiguous parameterization of the surface. The second step compensates for locally varying distortions of the unfolded surface. A technique similar to magnification fields in information visualization is hereby applied. The technique produces a single view of a complete, virtually dissected colon.

Abstract Virtual endoscopy is a promising medical application for volume‐rendering techniques where perspective projection is mandatory. Most of the acceleration techniques for direct volume rendering use parallel projection. This paper presents an algorithm to approximate perspective volume rendering using parallel projected slabs. The introduced error due to the approximation is investigated. An analytical study of the maximum and average error is made. This method is applied to VolumePro 500. Based on the error analysis, the basic algorithm is improved. This improvement increases the frame rate, keeping the global maximum error bounded. The usability of the algorithm is shown through the virtual endoscopic investigation of various types of medical data sets. Copyright © 2002 John Wiley & Sons, Ltd.

Virtual endoscopy presents the cross-sectional acquired 3D-data of a computer tomograph as an endoluminal view. The common approach for the visualization of a virtual endoscopy is surface rendering, yielding images close to a real endoscopy. If external structures are of interest, volume rendering techniques have to be used. These methods do not display the exact shape of the inner lumen very well. For certain applications, e.g. operation planning of a transbronchial biopsy, both the shape of the inner lumen as well as outer structures like blood vessels and the tumor have to be delineated. A method is described, that allows a quick and easy hybrid visualization using overlays of different visualization methods like different surfaces or volume renderings with different transfer functions in real time on a low-end PC. To achieve real time frame rates, image based rendering techniques have been used.