We propose two novel visualization methods optimized for supine breast images that “flatten” breast tissue, facilitating examination of larger tissue areas within each coronal slice. Breast cancer is the most frequently diagnosed cancer in women, and early lesion detection is crucial for reducing mortality. Supine breast magnetic resonance imaging (MRI) enables better lesion localization for image-guided interventions; however, traditional axial visualization is suboptimal because the tissue spreads over the chest wall, resulting in numerous fragmented slices that radiologists must scroll through during standard interpretation. Using a human-centered design approach, we incorporated user and expert feedback throughout the co-design and evaluation stages of our flattening methods. Our first proposed method, a surface-cutting approach, generates offset surfaces and flattens them independently using As-Rigid-As-Possible (ARAP) surface mesh parameterization. The second method uses a landmark-based warp to flatten the entire breast volume at once. Expert evaluations revealed that the surface-cutting method provides intuitive overviews and clear vascular detail, with low metric (2–2.5%) and area (3.7–4.4%) distortions. However, independent slice flattening can introduce depth distortions across layers. The landmark warp offers consistent slice alignment and supports direct annotations and measurements, with radiologists favoring it for its anatomical accuracy. Both methods significantly reduced the number of slices needed to review, highlighting their potential for time savings and clinical impact — an essential factor for adopting supine MRI.

Translating the success of deep learning-based computer-assisted classification into clinical adaptation hinges on the ability to explain a prediction's causality. Post-hoc interpretability approaches, especially counterfactual techniques, have shown both technical and psychological potential. Nevertheless, currently dominant approaches utilize heuristic, unvalidated methodology. Thereby, they potentially operate the underlying networks outside their validated domain, adding doubt in the predictor's abilities instead of generating knowledge and trust. In this work, we investigate this out-of-distribution problem for medical image pathology classifiers and propose marginalization techniques and evaluation procedures to overcome it. Furthermore, we propose a complete domain-aware pipeline for radiology environments. Its validity is demonstrated on a synthetic and two publicly available image datasets. Specifically, we evaluate using the CBIS-DDSM/DDSM mammography collection and the Chest X-ray14 radiographs. Our solution shows, both quantitatively and qualitatively, a significant reduction of localization ambiguity and clearer conveying results.

Recent advances in neuro-imaging enable scientists to create brain network data that can lead to novel insights into neurocircuitry, and a better understanding of the brain’s organization. These networks inherently involve a spatial component, depicting which brain regions are structurally, functionally or genetically related. Their visualization in 3D suffers from occlusion and clutter, especially with increasing number of nodes and connections, while 2D representations such as connectograms, connectivity matrices, and node-link diagrams neglect the spatio-anatomical context. Approaches to arrange 2D-graphs manually are tedious, species-dependent, and require the knowledge of domain experts. In this paper, we present a spatial-data-driven approach for layouting 3D brain networks in 2D node-link diagrams, while maintaining their spatial organization. The produced graphs do not need manual positioning of nodes, are consistent (even for sub-graphs), and provide a perspective-dependent arrangement for orientation. Furthermore, we provide a visual design for highlighting anatomical context, including the shape of the brain, and the size of brain regions. We present in several case-studies the applicability of our approach for different neuroscience-relevant species, including the mouse, human, and Drosophila larvae. In a user study conducted with several domain experts, we demonstrate its relevance and validity, as well as its potential for neuroscientific publications, presentations, and education.

Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. We fuse model predictions and high-level features from deep learning models with hybrid patient models to build stronger predictors on patient level. To this end, we propose a multi-branch deep learning model which efficiently fuses features across different tasks and mammograms to obtain a comprehensive patient-level prediction. We train and evaluate our full pipeline on public mammography data, i.e., DDSM and its curated version CBIS-DDSM, and report an AUC score of 0.962 for predicting the presence of any lesion and 0.791 for predicting the presence of malignant lesions on patient level. Overall, our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling. Moreover, by providing not only global patient-level predictions but also task-specific model results that are related to radiological features, our pipeline aims to closely support the reading workflow of radiologists.

• VCBM is the Eurographics Workshop on Visual Computing for Biology and Medicine. • VCBM addresses the state of the art in visual computing research with a strong focus on applications in biology and medicine. • The topics of VCBM include visualization, visual analytics, computer graphics, image processing, computer vision, human computer interfaces. • This section contains significantly extended and revised papers from VCBM 2019 and completely new articles within the same scope.

Abstract Attrition or dropout is the most severe missingness problem in longitudinal cohort study data where some participants do not show up for follow‐up examinations. Dropouts result in biased data and cause the reduction of 1ata set size. Moreover, they limit the power of statistical analysis and the validity of study findings. Visualization can play a strong role in analysing and displaying the missingness patterns. In this work, we present VIVID, a framework for the visual analysis of missing values in cohort study data. VIVID is inspired by discussions with epidemiologists and adds visual components to their current statistics‐based approaches. VIVID provides functions for exploration, imputation and validity check of imputations. The main focus of this paper is multiple imputation to fix the missing data.

Abstract Radiation therapy (RT) is one of the major curative approaches for cancer. It is a complex and risky treatment approach, which requires precise planning, prior to the administration of the treatment. Visual Computing (VC) is a fundamental component of RT planning, providing solutions in all parts of the process—from imaging to delivery. Despite the significant technological advancements of RT over the last decades, there are still many challenges to address. This survey provides an overview of the compound planning process of RT, and of the ways that VC has supported RT in all its facets. The RT planning process is described to enable a basic understanding in the involved data, users and workflow steps. A systematic categorization and an extensive analysis of existing literature in the joint VC/RT research is presented, covering the entire planning process. The survey concludes with a discussion on lessons learnt, current status, open challenges, and future directions in VC/RT research.

Biological network figures are ubiquitous in the biology and medical literature. On the one hand, a good network figure can quickly provide information about the nature and degree of interactions between items and enable inferences about the reason for those interactions. On the other hand, good network figures are difficult to create. In this paper, we outline 10 simple rules for creating biological network figures for communication, from choosing layouts, to applying color or other channels to show attributes, to the use of layering and separation. These rules are accompanied by illustrative examples. We also provide a concise set of references and additional resources for each rule.

Segmentation in 3-D scans is playing an increasingly important role in current clinical practice supporting diagnosis, tissue quantification, or treatment planning. The current 3-D approaches based on convolutional neural networks usually suffer from at least three main issues caused predominantly by implementation constraints-first, they require resizing the volume to the lower-resolutional reference dimensions, and second, the capacity of such approaches is very limited due to memory restrictions, and third, all slices of volumes have to be available at any given training or testing time. We address these problems by a U-Net-like architecture consisting of bidirectional convolutional long short-term memory and convolutional, pooling, upsampling, and concatenation layers enclosed into time-distributed wrappers. Our network can either process the full volumes in a sequential manner or segment slabs of slices on demand. We demonstrate performance of our architecture on vertebrae and liver segmentation tasks in 3-D computed tomography scans.

Linking and brushing is an essential technique for interactive data exploration and analysis that leverages coordinated multiple views to identify, select, and combine data points of interest. We propose to augment this technique by directly exploring data space using textual queries. Textual and visual queries are freely combined and modified during the data exploration process. Visual queries are used to refine the results of textual queries and vice versa. This mixed brushing integrates procedural, textual, and visual based data exploration to provide a unified approach to brushing. We also propose an interface --- the Text Query Browser View, that allows users to specify and edit data queries as well as to browse the data query history. Further, we argue why an interactive, on-demand, data aggregation and derivation is necessary, and we provide a flexible mechanism that supports it. We have implemented the proposed approach within an existing visualization tool using a client-server architecture. The approach was illustrated and evaluated using two example data sets.

Abstract Multi‐modal data of the complex human anatomy contain a wealth of information. To visualize and explore such data, techniques for emphasizing important structures and controlling visibility are essential. Such fused overview visualizations guide physicians to suspicious regions to be analysed in detail, e.g. with slice‐based viewing. We give an overview of state of the art in multi‐modal medical data visualization techniques. Multi‐modal medical data consist of multiple scans of the same subject using various acquisition methods, often combining multiple complimentary types of information. Three‐dimensional visualization techniques for multi‐modal medical data can be used in diagnosis, treatment planning, doctor–patient communication as well as interdisciplinary communication. Over the years, multiple techniques have been developed in order to cope with the various associated challenges and present the relevant information from multiple sources in an insightful way. We present an overview of these techniques and analyse the specific challenges that arise in multi‐modal data visualization and how recent works aimed to solve these, often using smart visibility techniques. We provide a taxonomy of these multi‐modal visualization applications based on the modalities used and the visualization techniques employed. Additionally, we identify unsolved problems as potential future research directions.

The success of deep convolutional neural networks (NNs) on image classification and recognition tasks has led to new applications in very diversified contexts, including the field of medical imaging. In this paper, we investigate and propose NN architectures for automated multiclass segmentation of anatomical organs in chest radiographs (CXRs), namely for lungs, clavicles, and heart. We address several open challenges including model overfitting, reducing number of parameters, and handling of severely imbalanced data in CXR by fusing recent concepts in convolutional networks and adapting them to the segmentation problem task in CXR. We demonstrate that our architecture combining delayed subsampling, exponential linear units, highly restrictive regularization, and a large number of high-resolution low-level abstract features outperforms state-of-the-art methods on all considered organs, as well as the human observer on lungs and heart. The models use a multiclass configuration with three target classes and are trained and tested on the publicly available Japanese Society of Radiological Technology database, consisting of 247 X-ray images the ground-truth masks for which are available in the segmentation in CXR database. Our best performing model, trained with the loss function based on the Dice coefficient, reached mean Jaccard overlap scores of 95% for lungs, 86.8% for clavicles, and 88.2% for heart. This architecture outperformed the human observer results for lungs and heart.

Abstract Neurobiologists investigate the brain of the common fruit fly Drosophila melanogaster to discover neural circuits and link them to complex behaviour. Formulating new hypotheses about connectivity requires potential connectivity information between individual neurons, indicated by overlaps of arborizations of two or more neurons. As the number of higher order overlaps (i.e. overlaps of three or more arborizations) increases exponentially with the number of neurons under investigation, visualization is impeded by clutter and quantification becomes a burden. Existing solutions are restricted to visual or quantitative analysis of pairwise overlaps, as they rely on precomputed overlap data. We present a novel tool that complements existing methods for potential connectivity exploration by providing for the first time the possibility to compute and visualize higher order arborization overlaps on the fly and to interactively explore this information in both its spatial anatomical context and on a quantitative level. Qualitative evaluation by neuroscientists and non‐experts demonstrated the utility and usability of the tool.

We propose an automated pipeline for vessel centerline extraction in 3-D computed tomography angiography (CTA) scans with arbitrary fields of view. The principal steps of the pipeline are body part detection, candidate seed selection, segment tracking, which includes centerline extraction, and vessel tree growing. The final tree-growing step can be instantiated in either a semi- or fully automated fashion. The fully automated initialization is carried out using a vessel position regression algorithm. Both semi-and fully automated methods were evaluated on 30 CTA scans comprising neck, abdominal, and leg arteries in multiple fields of view. High detection rates and centerline accuracy values for 38 distinct vessels demonstrate the effectiveness of our approach.

Deformable image registration (DIR) has the potential to improve modern radiotherapy in many aspects, including volume definition, treatment planning and image-guided adaptive radiotherapy. Studies have shown its possible clinical benefits. However, measuring DIR accuracy is difficult without known ground truth, but necessary before integration in the radiotherapy workflow. Visual assessment is an important step towards clinical acceptance. We propose a visualization framework which supports the exploration and the assessment of DIR accuracy. It offers different interaction and visualization features for exploration of candidate regions to simplify the process of visual assessment. The visualization is based on voxel-wise comparison of local image patches for which dissimilarity measures are computed and visualized to indicate locally the registration results. We performed an evaluation with three radiation oncologists to demonstrate the viability of our approach. In the evaluation, lung regions were rated by the participants with regards to their visual accuracy and compared to the registration error measured with expert defined landmarks. Regions rated as "accepted" had an average registration error of 1.8 mm, with the highest single landmark error being 3.3 mm. Additionally, survey results show that the proposed visualizations support a fast and intuitive investigation of DIR accuracy, and are suitable for finding even small errors.

Neurobiology investigates how anatomical and physiological relationships in the nervous system mediate behavior. Molecular genetic techniques, applied to species such as the common fruit fly Drosophila melanogaster, have proven to be an important tool in this research. Large databases of transgenic specimens are being built and need to be analyzed to establish models of neural information processing. In this paper we present an approach for the exploration and analysis of neural circuits based on such a database. We have designed and implemented BrainGazer, a system which integrates visualization techniques for volume data acquired through confocal microscopy as well as annotated anatomical structures with an intuitive approach for accessing the available information. We focus on the ability to visually query the data based on semantic as well as spatial relationships. Additionally, we present visualization techniques for the concurrent depiction of neurobiological volume data and geometric objects which aim to reduce visual clutter. The described system is the result of an ongoing interdisciplinary collaboration between neurobiologists and visualization researchers.

Surgical approaches tailored to an individual patient's anatomy and pathology have become standard in neurosurgery. Precise preoperative planning of these procedures, however, is necessary to achieve an optimal therapeutic effect. Therefore, multiple radiological imaging modalities are used prior to surgery to delineate the patient's anatomy, neurological function, and metabolic processes. Developing a three-dimensional perception of the surgical approach, however, is traditionally still done by mentally fusing multiple modalities. Concurrent 3D visualization of these datasets can, therefore, improve the planning process significantly. In this paper we introduce an application for planning of individual neurosurgical approaches with high-quality interactive multimodal volume rendering. The application consists of three main modules which allow to (1) plan the optimal skin incision and opening of the skull tailored to the underlying pathology; (2) visualize superficial brain anatomy, function and metabolism; and (3) plan the patient-specific approach for surgery of deep-seated lesions. The visualization is based on direct multi-volume raycasting on graphics hardware, where multiple volumes from different modalities can be displayed concurrently at interactive frame rates. Graphics memory limitations are avoided by performing raycasting on bricked volumes. For preprocessing tasks such as registration or segmentation, the visualization modules are integrated into a larger framework, thus supporting the entire workflow of preoperative planning.

Virtual endoscopy has proven to be a very powerful tool in endoscopic surgery. However, most virtual endoscopy systems are restricted to rendering isosurfaces or require segmentation in order to visualize additional objects behind occluding tissue. This paper presents a system for real-time perspective direct volume and isosurface rendering, which allows to simultaneously visualize both the interesting tissue and everything that is behind. Large volume data can be viewed seamlessly from inside or outside the volume without any pre-computation or segmentation. Our system uses a novel ray-casting pipeline for GPUs that has been optimized for the requirements of virtual endoscopy and also allows easy incorporation of auxiliary geometry, e.g., for displaying parts of the endoscopic device, pointers, or grid lines for orientation purposes. We present three main applications of this system and the underlying ray-casting algorithm. Although our ray-casting approach is of general applicability, we have specifically applied it to virtual colonoscopy, virtual angioscopy, and virtual pituitary surgery.

The major problem when building shape models by minimising the description length (DL) is the computationally demanding optimisation in a highdimensional space. To speed up the convergence when dealing with discretised contours, Ericsson and Astrom [6] have shown how to approximate and how to exploit the gradient of the DL. We derive the gradient of the DL for differentiable training sets. Additionally, we propose a class of polynomial reparameterisations that allows us to avoid numerical approximation of the functions and integrals involved in computation of the model and of its DL’s gradient, making the whole process exact and more efficient.

Endoscopy has recently been introduced to endonasal transsphenoidal pituitary surgery as a minimally invasive procedure for the removal of various kinds of pituitary tumors. To reduce morbidity and mortality with this new technique, the surgeon must be well-trained and well-prepared. Virtual endoscopy can be beneficial as a tool for training, preoperative planning, and intraoperative support. This paper introduces STEPS, a virtual endoscopy system designed to aid surgeons in getting acquainted with the endoscopic view of the anatomy, the handling of instruments, the transsphenoidal approach, and challenges associated with the procedure. STEPS also assists experienced surgeons in planning a real endoscopic intervention by getting familiar with the individual patient anatomy, identifying landmarks, planning the approach, and deciding upon the ideal target position of the actual surgical activity. The application provides interactive visualization, navigation, and perception aids and the possibility of simulating the procedure, including haptic feedback and simulation of surgical instruments.

This paper describes a new method for generating surfaces of branching tubular structures with given center-lines and radii. As the centerlines are not straight lines, the cross-sections are not parallel and well-known algorithms for surface tiling from parallel cross-sections cannot be used. Nonparallel cross-sections can be tiled by means of the maximal-disc interpolation method; special methods for branching-structures modeling by means of convolution surfaces produce excellent results, but these methods are more complex than our approach. The proposed method tiles nonparallel circular cross-sections and constructs a topologically-correct surface mesh. The method is not artifact-free, but it is fast and simple. The surface mesh serves as a data representation of a vessel tree suitable for real-time virtual reality operation planning and operation support within a medical application. Proposed method extracts a "classical" polygonal representation, which can be used in common surface-oriented graphic accelerators

Endonasal transsphenoidal pituitary surgery is a minimally invasive endoscopic procedure, applied to remove various kinds of pituitary tumors. To reduce the risk associated with this treatment, the surgeon must be skilled and well-prepared. Virtual endoscopy can be beneficial as a tool for training, preoperative planning and intraoperative support. This work introduces STEPS, a virtual endoscopy system designed to aid surgeons in getting acquainted with the endoscopic view, the handling of instruments, the transsphenoidal approach and challenges associated with the procedure. STEPS also assists experienced surgeons in planning a real endoscopic intervention by getting familiar with the individual patient anatomy, identifying landmarks, planning the approach and deciding upon the ideal target position of the actual surgical activity. Besides interactive visualization using two different first-hit ray casting techniques, the application provides navigation and perception aids and the possibility to simulate the procedure, including haptic feedback and simulation of surgical instruments.

The new concept of parametrized bounding volumes for parametric objects is proposed to replace the common compact bounding volumes like axis aligned bounding boxes and parallelepipeds. Linear Interval Estimations (LIEs) are developed as a realization of the discussed ideas. Two reliable methods for the computation of LIEs are introduced based on a new understanding of the use of affine arithmetics and a special application of Taylor Models. The particular structure of LIEs allows an effective intersection test of LIEs with rays, boxes and other LIEs. The test gives besides of a possible location of the intersection in object space information about affected parts in the parameter spaces of the enclosed objects. A subdivision algorithm for the intersection of two parametric surface patches with remarkable experimental results is presented as a possible application.