Humans inherently connect certain colors with particular concepts in semantically meaningful ways that facilitate visual communication. These colors are known as semantically resonant colors. For instance, we associate "sky" and "ocean" with shades of blue, and "cherry" with red. In this paper, we investigate how language models, including Word2Vec, RoBERTa, GPT-4o mini and the vision language model CLIP generate and represent nuanced semantically resonant colors for diverse concepts. To achieve this, we utilized a large dataset of color names and concepts, tailored models for the structure of each language model, and developed an interactive web interface, Concept2Color, as a use case. Additionally, we conducted experiments and a detailed analysis to assess the ability of these models to generate meaningful colors. Through these experiments, we examined how factors such as model design, training data and context affect the color output. Our findings reveal the capabilities and limitations of language models in processing and generating semantically resonant colors for concepts, thus contributing insights into how they depict semantic color-concept connections. These insights have implications for data visualization, design, and human-computer interaction, where leveraging effective semantic color generation can enhance communication and user experience.

Misleading visualizations pose a significant challenge to accurate data interpretation. While recent research has explored the use of Large Language Models (LLMs) for detecting such misinformation, practical tools that also support explanation and correction remain limited. We present MisVisFix, an interactive dashboard that leverages both Claude and GPT models to support the full workflow of detecting, explaining, and correcting misleading visualizations. MisVisFix correctly identifies 96% of visualization issues and addresses all 74 known visualization misinformation types, classifying them as major, minor, or potential concerns. It provides detailed explanations, actionable suggestions, and automatically generates corrected charts. An interactive chat interface allows users to ask about specific chart elements or request modifications. The dashboard adapts to newly emerging misinformation strategies through targeted user interactions. User studies with visualization experts and developers of fact-checking tools show that MisVisFix accurately identifies issues and offers useful suggestions for improvement. By transforming LLM-based detection into an accessible, interactive platform, MisVisFix advances visualization literacy and supports more trustworthy data communication.

This paper evaluates the visualization literacy of modern Large Language Models (LLMs) and introduces a novel prompting technique called Charts-of-Thought. We tested three state-of-the-art LLMs (Claude-3.7-sonnet, GPT-4.5-preview, and Gemini-2.0-pro) on the Visualization Literacy Assessment Test (VLAT) using standard prompts and our structured approach. The Charts-of-Thought method guides LLMs through a systematic data extraction, verification, and analysis process before answering visualization questions. Our results show Claude-3.7-sonnet achieved a score of 50.17 using this method, far exceeding the human baseline of 28.82. This approach improved performance across all models, with score increases of 21.8% for GPT-4.5, 9.4% for Gemini-2.0, and 13.5% for Claude-3.7 compared to standard prompting. The performance gains were consistent across original and modified VLAT charts, with Claude correctly answering 100% of questions for several chart types that previously challenged LLMs. Our study reveals that modern multimodal LLMs can surpass human performance on visualization literacy tasks when given the proper analytical framework. These findings establish a new benchmark for LLM visualization literacy and demonstrate the importance of structured prompting strategies for complex visual interpretation tasks. Beyond improving LLM visualization literacy, Charts-of-Thought could also enhance the accessibility of visualizations, potentially benefiting individuals with visual impairments or lower visualization literacy.

Multivariate data contain an abundance of information and many techniques have been proposed to allow humans to navigate this information in an ordered fashion. For this work, we focus on methods that seek to convey multivariate data as a collection of bivariate scatterplots or parallel coordinates plots. Presenting multivariate data in this way requires a regime that determines in what order the bivariate scatterplots are presented or in what order the parallel coordinate axes are arranged. We refer to this order as a visualization sequence. Common techniques utilize standard statistical metrics like correlation, similarity or consistency. We expand on the family of statistical metrics by incorporating the rigidity of causal relationships. To capture these relationships, we first derive a causal graph from the data and then allow users to select from several semantic traversal schemes to derive the respective chart sequence. We tested the sequences with a crowd-sourced user study and a user interview to confirm that the causality-informed visualization sequences help viewers to better grasp the causal relationships that exist in the data, as opposed to sequences derived from correlations or randomization alone.

Organizations increasingly rely on data mining to discover predictive patterns that inform decision-making. However, the vast number of patterns produced by mining algorithms, especially in high-dimensional datasets, can overwhelm analysts. To address this challenge, we present the evolutionary design of a visual analytics interface that enables users to explore and interpret mined predictive patterns effectively. Starting from a projection-based prototype rooted in research tools, we iteratively refined the interface through feedback from data analysts and real-world usage. Each design iteration – projection maps, bubble plots, and finally a card-based layout – was shaped by insights into user comprehension, trust, and pattern comparison needs. The final interface supports overview-to-detail exploration, pattern organization, and contextual analysis, as demonstrated in a case study on student attrition in Computer Science. We reflect on the lessons learned across iterations and offer guidance for designing interpretable pattern exploration tools for high-dimensional data. • A comparative analysis of three visual interface designs for pattern exploration in high-dimensional data. • Design insights on how visualization techniques influence pattern interpretability and user trust. • A production-ready interface demonstrated through a real-world case study. • Design guidance for building effective pattern exploration tools that bridge machine learning and human sensemaking.

The issue of fairness in decision-making is a critical one, especially given the variety of stakeholder demands for differing and mutually incompatible versions of fairness. Adopting a strategic interaction of perspectives provides an alternative to enforcing a singular standard of fairness. We present a web-based software application, FairPlay, that enables multiple stakeholders to debias datasets collaboratively. With FairPlay, users can negotiate and arrive at a mutually acceptable outcome without a universally agreed-upon theory of fairness. In the absence of such a tool, reaching a consensus would be highly challenging due to the lack of a systematic negotiation process and the inability to modify and observe changes. We have conducted user studies that demonstrate the success of FairPlay, as users could reach a consensus within about five rounds of gameplay, illustrating the application's potential for enhancing fairness in AI systems.

We present Explainable XR, an end-to-end framework for analyzing user behavior in diverse eXtended Reality (XR) environments by leveraging Large Language Models (LLMs) for data interpretation assistance. Existing XR user analytics frameworks face challenges in handling cross-virtuality - AR, VR, MR - transitions, multi-user collaborative application scenarios, and the complexity of multimodal data. Explainable XR addresses these challenges by providing a virtuality-agnostic solution for the collection, analysis, and visualization of immersive sessions. We propose three main components in our framework: (1) A novel user data recording schema, called User Action Descriptor (UAD), that can capture the users' multimodal actions, along with their intents and the contexts; (2) a platform-agnostic XR session recorder, and (3) a visual analytics interface that offers LLM-assisted insights tailored to the analysts' perspectives, facilitating the exploration and analysis of the recorded XR session data. We demonstrate the versatility of Explainable XR by demonstrating five use-case scenarios, in both individual and collaborative XR applications across virtualities. Our technical evaluation and user studies show that Explainable XR provides a highly usable analytics solution for understanding user actions and delivering multifaceted, actionable insights into user behaviors in immersive environments.

We present a comprehensive pipeline, integrated with a visual analytics system called GapMiner, capable of exploring and exploiting untapped opportunities within the empty regions of high-dimensional datasets. Our approach utilizes a novel Empty-Space Search Algorithm (ESA) to identify the center points of these uncharted voids, which represent reservoirs for potentially valuable new configurations. Initially, this process is guided by user interactions through GapMiner, which visualizes Empty-Space Configurations (ESCs) within the context of the dataset and allows domain experts to explore and refine ESCs for subsequent validation in domain experiments or simulations. These activities iteratively enhance the dataset and contribute to training a connected deep neural network (DNN). As training progresses, the DNN gradually assumes the role of identifying and validating high-potential ESCs, reducing the need for direct user involvement. Once the DNN achieves sufficient accuracy, it autonomously guides the exploration of optimal configurations by predicting performance and refining configurations through a combination of gradient ascent and improved empty-space searches. Domain experts were actively involved throughout the system's development. Our findings demonstrate that this methodology consistently generates superior novel configurations compared to conventional randomization-based approaches. We illustrate its effectiveness in multiple case studies with diverse objectives.

Causal networks are widely used in many fields to model the complex relationships between variables. A recent approach has sought to construct causal networks by leveraging the wisdom of crowds through the collective participation of humans. While this can yield detailed causal networks that model the underlying phenomena quite well, it requires a large number of individuals with domain understanding. We adopt a different approach: leveraging the causal knowledge that large language models, such as OpenAI's GPT-4, have learned by ingesting massive amounts of literature. Within a dedicated visual analytics interface, called CausalChat, users explore single variables or variable pairs recursively to identify causal relations, latent variables, confounders, and mediators, constructing detailed causal networks through conversation. Each probing interaction is translated into a tailored GPT-4 prompt and the response is conveyed through visual representations which are linked to the generated text for explanations. We demonstrate the functionality of CausalChat across diverse data contexts and conduct user studies involving both domain experts and laypersons.

Causal belief is a cognitive practice that humans apply everyday to reason about cause and effect relations between factors, phenomena, or events. Like optical illusions, humans are prone to drawing causal relations between events that are only coincidental (i.e., causal illusions). Researchers in domains such as cognitive psychology and healthcare often use logistically expensive experiments to understand causal beliefs and illusions. In this paper, we propose Belief Miner, a crowdsourcing method for evaluating people's causal beliefs and illusions. Our method uses the (dis)similarities between the causal relations collected from the crowds and experts to surface the causal beliefs and illusions. Through an iterative design process, we developed a web-based interface for collecting causal relations from a target population. We then conducted a crowdsourced experiment with 101 workers on Amazon Mechanical Turk and Prolific using this interface and analyzed the collected data with Belief Miner. We discovered a variety of causal beliefs and potential illusions, and we report the design implications for future research.

In volume visualization transfer functions are widely used for mapping voxel properties to color and opacity. Typically, volume density data are scalars which require simple 1D transfer functions to achieve this mapping. If the volume densities are vectors of three channels, one can straightforwardly map each channel to either red, green or blue, which requires a trivial extension of the 1D transfer function editor. We devise a new method that applies to volume data with more than three channels. These types of data often arise in scientific scanning applications, where the data are separated into spectral bands or chemical elements. Our method expands on prior work in which a multivariate information display, RadViz, was fused with a radial color map, in order to visualize multi-band 2D images. In this work, we extend this joint interface to blended volume rendering. The information display allows users to recognize the presence and value distribution of the multivariate voxels and the joint volume rendering display visualizes their spatial distribution. We design a set of operators and lenses that allow users to interactively control the mapping of the multivariate voxels to opacity and color. This enables users to isolate or emphasize volumetric structures with desired multivariate properties. Furthermore, it turns out that our method also enables more insightful displays even for RGB data. We demonstrate our method with three datasets obtained from spectral electron microscopy, high energy X-ray scanning, and atmospheric science.

Numerous physical objects in our daily lives are grouped or ranked according to a stereotyped presentation style. For example, in a library, books are typically grouped and ranked based on classification numbers. However, for better comparison, we often need to re-group or re-rank the books using additional attributes such as ratings, publishers, comments, publication years, keywords, prices, etc., or a combination of these factors. In this article, we propose a novel mobile DR/MR-based application framework named DRCmpVis to achieve in-context multi-attribute comparisons of physical objects with text labels or textual information. The physical objects are scanned in the real world using mobile cameras. All scanned objects are then segmented and labeled by a convolutional neural network and replaced (diminished) by their virtual avatars in a DR environment. We formulate three visual comparison strategies, including filtering, re-grouping, and re-ranking, which can be intuitively, flexibly, and seamlessly performed on their avatars. This approach avoids breaking the original layouts of the physical objects. The computation resources in virtual space can be fully utilized to support efficient object searching and multi-attribute visual comparisons. We demonstrate the usability, expressiveness, and efficiency of DRCmpVis through a user study, NASA TLX assessment, quantitative evaluation, and case studies involving different scenarios.

Multivariate datasets with many variables are increasingly common in many application areas. Most methods approach multivariate data from a singular perspective. Subspace analysis techniques, on the other hand. provide the user a set of subspaces which can be used to view the data from multiple perspectives. However, many subspace analysis methods produce a huge amount of subspaces, a number of which are usually redundant. The enormity of the number of subspaces can be overwhelming to analysts, making it difficult for them to find informative patterns in the data. In this article, we propose a new paradigm that constructs semantically consistent subspaces. These subspaces can then be expanded into more general subspaces by ways of conventional techniques. Our framework uses the labels/meta-data of a dataset to learn the semantic meanings and associations of the attributes. We employ a neural network to learn a semantic word embedding of the attributes and then divide this attribute space into semantically consistent subspaces. The user is provided with a visual analytics interface that guides the analysis process. We show via various examples that these semantic subspaces can help organize the data and guide the user in finding interesting patterns in the dataset.

With the rise of AI, algorithms have become better at learning underlying patterns from the training data including ingrained social biases based on gender, race, etc. Deployment of such algorithms to domains such as hiring, healthcare, law enforcement, etc. has raised serious concerns about fairness, accountability, trust and interpretability in machine learning algorithms. To alleviate this problem, we propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases from tabular datasets. It uses a graphical causal model to represent causal relationships among different features in the dataset and as a medium to inject domain knowledge. A user can detect the presence of bias against a group, say females, or a subgroup, say black females, by identifying unfair causal relationships in the causal network and using an array of fairness metrics. Thereafter, the user can mitigate bias by refining the causal model and acting on the unfair causal edges. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset based on the current causal model while ensuring a minimal change from the original dataset. Users can visually assess the impact of their interactions on different fairness metrics, utility metrics, data distortion, and the underlying data distribution. Once satisfied, they can download the debiased dataset and use it for any downstream application for fairer predictions. We evaluate D-BIAS by conducting experiments on 3 datasets and also a formal user study. We found that D-BIAS helps reduce bias significantly compared to the baseline debiasing approach across different fairness metrics while incurring little data distortion and a small loss in utility. Moreover, our human-in-the-loop based approach significantly outperforms an automated approach on trust, interpretability and accountability.

Presents the editorial for this issue of the publication.

Presents the editorial for this issue of the publication

Parallel coordinate plots (PCPs) have been widely used for high-dimensional (HD) data storytelling because they allow for presenting a large number of dimensions without distortions. The axes ordering in PCP presents a particular story from the data based on the user perception of PCP polylines. Existing works focus on directly optimizing for PCP axes ordering based on some common analysis tasks like clustering, neighborhood, and correlation. However, direct optimization for PCP axes based on these common properties is restrictive because it does not account for multiple properties occurring between the axes, and for local properties that occur in small regions in the data. Also, many of these techniques do not support the human-in-the-loop (HIL) paradigm, which is crucial (i) for explainability and (ii) in cases where no single reordering scheme fits the users' goals. To alleviate these problems, we present PC-Expo, a real-time visual analytics framework for all-in-one PCP line pattern detection and axes reordering. We studied the connection of line patterns in PCPs with different data analysis tasks and datasets. PC-Expo expands prior work on PCP axes reordering by developing real-time, local detection schemes for the 12 most common analysis tasks (properties). Users can choose the story they want to present with PCPs by optimizing directly over their choice of properties. These properties can be ranked, or combined using individual weights, creating a custom optimization scheme for axes reordering. Users can control the granularity at which they want to work with their detection scheme in the data, allowing exploration of local regions. PC-Expo also supports HIL axes reordering via local-property visualization, which shows the regions of granular activity for every axis pair. Local-property visualization is helpful for PCP axes reordering based on multiple properties, when no single reordering scheme fits the user goals. A comprehensive evaluation was done with real users and diverse datasets confirm the efficacy of PC-Expo in data storytelling with PCPs.

The success of DL can be attributed to hours of parameter and architecture tuning by human experts. Neural Architecture Search (NAS) techniques aim to solve this problem by automating the search procedure for DNN architectures making it possible for non-experts to work with DNNs. Specifically, One-shot NAS techniques have recently gained popularity as they are known to reduce the search time for NAS techniques. One-Shot NAS works by training a large template network through parameter sharing which includes all the candidate NNs. This is followed by applying a procedure to rank its components through evaluating the possible candidate architectures chosen randomly. However, as these search models become increasingly powerful and diverse, they become harder to understand. Consequently, even though the search results work well, it is hard to identify search biases and control the search progression, hence a need for explainability and human-in-the-loop (HIL) One-Shot NAS. To alleviate these problems, we present NAS-Navigator, a visual analytics (VA) system aiming to solve three problems with One-Shot NAS; explainability, HIL design, and performance improvements compared to existing state-of-the-art (SOTA) techniques. NAS-Navigator gives full control of NAS back in the hands of the users while still keeping the perks of automated search, thus assisting non-expert users. Analysts can use their domain knowledge aided by cues from the interface to guide the search. Evaluation results confirm the performance of our improved One-Shot NAS algorithm is comparable to other SOTA techniques. While adding Visual Analytics (VA) using NAS-Navigator shows further improvements in search time and performance. We designed our interface in collaboration with several deep learning researchers and evaluated NAS-Navigator through a control experiment and expert interviews.

Current work on using visual analytics to determine causal relations among variables has mostly been based on the concept of counterfactuals. As such the derived static causal networks do not take into account the effect of time as an indicator. However, knowing the time delay of a causal relation can be crucial as it instructs how and when actions should be taken. Yet, similar to static causality, deriving causal relations from observational time-series data, as opposed to designed experiments, is not a straightforward process. It can greatly benefit from human insight to break ties and resolve errors. We hence propose a set of visual analytics methods that allow humans to participate in the discovery of causal relations associated with windows of time delay. Specifically, we leverage a well-established method, logic-based causality, to enable analysts to test the significance of potential causes and measure their influences toward a certain effect. Furthermore, since an effect can be a cause of other effects, we allow users to aggregate different temporal cause-effect relations found with our method into a visual flow diagram to enable the discovery of temporal causal networks. To demonstrate the effectiveness of our methods we constructed a prototype system named DOMINO and showcase it via a number of case studies using real-world datasets. Finally, we also used DOMINO to conduct several evaluations with human analysts from different science domains in order to gain feedback on the utility of our system in practical scenarios.

An exemplar is an entity that represents a desirable instance in a multi-attribute configuration space. It offers certain strengths in some of its attributes without unduly compromising the strengths in other attributes. Exemplars are frequently sought after in real life applications, such as systems engineering, investment banking, drug advisory, product marketing and many others. We study a specific method for the visualization of multi-attribute configuration spaces, the Data Context Map (DCM), for its capacity in enabling users to identify proper exemplars. The DCM produces a 2D embedding where users can view the data objects in the context of the data attributes. We ask whether certain graphical enhancements can aid users to gain a better understanding of the attribute-wise tradeoffs and so select better exemplar sets. We conducted several user studies for three different graphical designs, namely iso-contour, value-shaded topographic rendering and terrain topographic rendering, and compare these with a baseline DCM display. As a benchmark we use an exemplar set generated via Pareto optimization which has similar goals but unlike humans can operate in the native high-dimensional data space. Our study finds that the two topographic maps are statistically superior to both the iso-contour and the DCM baseline display.

Understanding the cumulative effect of multiple fairness-enhancing interventions at different stages of the machine learning (ML) pipeline is a critical and underexplored facet of the fairness literature. Such knowledge can be valuable to data scientists/ML practitioners in designing fair ML pipelines. This paper takes the first step in exploring this area by undertaking an extensive empirical study comprising 60 combinations of interventions, 9 fairness metrics, 2 utility metrics (Accuracy and F1 Score) across 4 benchmark datasets. We quantitatively analyze the experimental data to measure the impact of multiple interventions on fairness, utility and population groups. We found that applying multiple interventions results in better fairness and lower utility than individual interventions on aggregate. However, adding more interventions do no always result in better fairness or worse utility. The likelihood of achieving high performance (F1 Score) along with high fairness increases with larger number of interventions. On the downside, we found that fairness-enhancing interventions can negatively impact different population groups, especially the privileged group. This study highlights the need for new fairness metrics that account for the impact on different population groups apart from just the disparity between groups. Lastly, we offer a list of combinations of interventions that perform best for different fairness and utility metrics to aid the design of fair ML pipelines.

Abstract Infographics are an aesthetic visual representation of information following specific design principles of human perception. Designing infographics can be a tedious process for non‐experts and time‐consuming, even for professional designers. With the help of designers, we propose a semi‐automated infographic framework for general structured and flow‐based infographic design generation. For novice designers, our framework automatically creates and ranks infographic designs for a user‐provided text with no requirement for design input. However, expert designers can still provide custom design inputs to customize the infographics. We will also contribute an individual visual group (VG) designs dataset (in SVG), along with a 1k complete infographic image dataset with segmented VGs in this work. Evaluation results confirm that by using our framework, designers from all expertise levels can generate generic infographic designs faster than existing methods while maintaining the same quality as hand‐designed infographics templates.

Welcome to the November 2022 issue of the IEEE Transactions on Visualization and Computer Graphics (TVCG). This issue contains selected papers accepted at the IEEE International Symposium on Mixed and Augmented Reality (ISMAR). The conference took place in Singapore from October 17-22,2022 in hybrid mode.

The widespread adoption of algorithmic decision-making systems has brought about the necessity to interpret the reasoning behind these decisions. The majority of these systems are complex black box models, and auxiliary models are often used to approximate and then explain their behavior. However, recent research suggests that such explanations are not overly accessible to lay users with no specific expertise in machine learning and this can lead to an incorrect interpretation of the underlying model. In this article, we show that a predictive and interactive model based on causality is inherently interpretable, does not require any auxiliary model, and allows both expert and non-expert users to understand the model comprehensively. To demonstrate our method we developed Outcome Explorer, a causality guided interactive interface, and evaluated it by conducting think-aloud sessions with three expert users and a user study with 18 non-expert users. All three expert users found our tool to be comprehensive in supporting their explanation needs while the non-expert users were able to understand the inner workings of a model easily.

Welcome to the January 2022 issue of theIEEE Transactions on Visualization and Computer Graphics (TVCG).This is traditionally the time of the year when we feature the IEEE VIS special issue and this year is no different. But there is one significant difference — IEEE VIS is now fully unified. There is no more IEEE VAST, IEEE InfoVis and IEEE SciVis; there is just IEEE VIS, tiled into six overarching areas: Applications (24 papers), Analytics & Decisions (19), Theoretical & Empirical (24), Representations & Interaction (19), Data Transformations (14) and Systems & Rendering (11). The conference was scheduled to take place in New Orleans (LA) but was moved to a virtual event due to the global coronavirus pandemic. This virtual conference took place from October 24-29, 2021. Contained in this special issue are the top 110 papers selected by the unified program committee from a total of 441 submissions. In addition, this issue also contains the Best Paper of the 2021 IEEE Large Scale Data Analysis and Visualization (LDAV) Symposium and the Best Paper of the 2021 IEEE Symposium on Visualization for Cyber Security (VizSec).

Presents the introductory editorial for this issue of the publication.

Welcome to the 11th IEEE Transactions on Visualization and Computer Graphics (TVCG)special issue on IEEE Virtual Reality and 3D User Interfaces. This volume contains a total of 29 full papers selected for and presented at the IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2022), held fully virtual in Christchurch, New Zealand from March 12 to 16, 2022.

By skeptics and undecided we refer to nodes in clustered social networks that cannot be assigned easily to any of the clusters. Such nodes are typically found either at the interface between clusters (the undecided) or at their boundaries (the skeptics). Identifying these nodes is relevant in marketing applications like voter targeting, because the persons represented by such nodes are often more likely to be affected in marketing campaigns than nodes deeply within clusters. So far this identification task is not as well studied as other network analysis tasks like clustering, identifying central nodes, and detecting motifs. We approach this task by deriving novel geometric features from the network structure that naturally lend themselves to an interactive visual approach for identifying interface and boundary nodes.

Stuttering is a speech disorder which impacts the personal and professional lives of millions of people worldwide. To save themselves from stigma and discrimination, people who stutter (PWS) may adopt different strategies to conceal their stuttering. One of the common strategies is word substitution where an individual avoids saying a word they might stutter on and use an alternative instead. This process itself can cause stress and add more burden. In this work, we present Fluent, an AI augmented writing tool which assists PWS in writing scripts which they can speak more fluently. Fluent embodies a novel active learning based method of identifying words an individual might struggle pronouncing. Such words are highlighted in the interface. On hovering over any such word, Fluent presents a set of alternative words which have similar meaning but are easier to speak. The user is free to accept or ignore these suggestions. Based on such user interaction (feedback), Fluent continuously evolves its classifier to better suit the personalized needs of each user. We evaluated our tool by measuring its ability to identify difficult words for 10 simulated users. We found that our tool can identify difficult words with a mean accuracy of over 80% in under 20 interactions and it keeps improving with more feedback. Our tool can be beneficial for certain important life situations like giving a talk, presentation, etc. The source code for this tool has been made publicly accessible at github.com/bhavyaghai/Fluent.

Eye movements are closely related to the cognitive processes and often act as a window to the brain and mind. To facilitate a window to the brain using eye movements, we propose EyeFIX, an interactive visual analytics interface. Our interface currently focuses on processing gaze movements to generate the two most prominent events, fixations, and saccades which are used in a large part of the eye movement literature. Our interface has multiple interactive widgets that allow the users to choose the algorithm and tune its hyper parameters that control how fixations and saccades are generated. It has four major visualizations: (1) A Gaze Plot to showcase the gaze movement of each participant, (2) a Fixation Plot that helps to identify regions of the stimulus which are of interest to a particular participant, (3) a Heat Map to study the denser regions of fixation and regions which the participant frequently visited, and (4) a Timeline Visualization to assist in having a closer look at temporal regions of interest. We design a set of brushing and linking operators that allow users to interactively control all the visualizations to dig deeper in both, the spatial and temporal domains. We demonstrate the applicability of our interface with datasets obtained by human subjects viewing naturalistic stimuli while performing various viewing tasks, including visual exploration, visual search, and prolonged visual fixation.

Welcome to the 10thIEEE Transactions on Visualization and Computer Graphics (TVCG)special issue on IEEE Virtual Reality and 3D User Interfaces. This volume contains a total of 25 full papers selected for and presented at the IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2021), held fully virtual from March 27 to April 3, 2021.Founded in 1993, IEEE VR has a long tradition as the premier venue where new research results in the field of Virtual Reality (VR) are presented. With the emergence of VR as a major technology in a diverse set of fields, such as entertainment, education, data analytics, artificial intelligence, medicine, construction, training, and many others, the papers presented at IEEE VR and published in theIEEE TVCGVR special issue mark a major highlight of the year.

Welcome to the November 2021 issue of theIEEE Transactions on Visualization and Computer Graphics (TVCG). This issue contains selected papers accepted at the IEEE International Symposium on Mixed and Augmented Reality (ISMAR). The conference took place in Bari, Italy from October 4–8, 2021 in virtual mode due to the COVID-19 pandemic.

Automatic speech recognition(ASR) systems play a key role in many commercial products including voice assistants. Typically, they require large amounts of high quality speech data for training which gives an undue advantage to large organizations which have tons of private data. We investigated if speech data obtained from publicly available sources can be further enhanced to train better speech recognition models. We begin with noisy/contaminated speech data, apply speech enhancement to produce 'cleaned' version and use both the versions to train the ASR model. We have found that using speech enhancement gives 9.5% better word error rate than training on just the original noisy data and 9% better than training on just the ground truth 'clean' data. It's performance is also comparable to the ideal case scenario when trained on noisy and it's ground truth 'clean' version.

Abstract Recent studies have indicated that visually embellished charts such as infographics have the ability to engage viewers and positively affect memorability. Fueled by these findings, researchers have proposed a variety of infographic design tools. However, these tools do not cover the entire design space. In this work, we identify a subset of infographics that we call infomages. Infomages are casual visuals of data in which a data chart is embedded into a thematic image such that the content of the image reflects the subject and the designer's interpretation of the data. Creating an effective infomage, however, can require a fair amount of design expertise and is thus out of reach for most people. In order to also afford non‐artists with the means to design convincing infomages, we first study the principled design of existing infomages and identify a set of key chart embedding techniques. Informed by these findings we build a design tool that links web‐scale image search with a set of interactive image processing tools to empower novice users with the ability to design a wide variety of infomages. As the embedding process might introduce some amount of visual distortion of the data our tool also aids users to gauge the amount of this distortion, if any. We experimentally demonstrate the usability of our tool and conclude with a discussion of infomages and our design tool.

The wide adoption of Machine Learning (ML) technologies has created a growing demand for people who can train ML models. Some advocated the term "machine teacher'' to refer to the role of people who inject domain knowledge into ML models. This "teaching'' perspective emphasizes supporting the productivity and mental wellbeing of machine teachers through efficient learning algorithms and thoughtful design of human-AI interfaces. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query a machine teacher for labels, so that the labeling workload could be largely reduced. However, in current AL settings, the human-AI interface remains minimal and opaque. A dearth of empirical studies further hinders us from developing teacher-friendly interfaces for AL algorithms. In this work, we begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one's own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the teacher ought to be able to understand the reasoning underlying its mistakes. When the model matures, the teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model's prediction without explanation). Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and additional cognitive workload. Our study also reveals important individual factors that mediate a machine teacher's reception to AI explanations, including task knowledge, AI experience and Need for Cognition. By reflecting on the results, we suggest future directions and design implications for XAL, and more broadly, machine teaching through AI explanations.

Several prominent studies have shown that the imbalanced on-screen exposure of observable phenotypic traits like gender and skin-tone in movies, TV shows, live telecasts, and other visual media can reinforce gender and racial stereotypes in society. Researchers and human rights organizations alike have long been calling to make media producers more aware of such stereotypes. While awareness among media producers is growing, balancing the presence of different phenotypes in a video requires substantial manual effort and can typically only be done in the post-production phase. The task becomes even more challenging in the case of a live telecast where video producers must make instantaneous decisions with no post-production phase to refine or revert a decision. In this paper, we propose Screen-Balancer, an interactive tool that assists media producers in balancing the presence of different phenotypes in a live telecast. The design of Screen-Balancer is informed by a field study conducted in a professional live studio. Screen-Balancer analyzes the facial features of the actors to determine phenotypic traits using facial detection packages; it then facilitates real-time visual feedback for interactive moderation of gender and skin-tone distributions. To demonstrate the effectiveness of our approach, we conducted a user study with 20 participants and asked them to compose live telecasts from a set of video streams simulating different camera angles, and featuring several male and female actors with different skin-tones. The study revealed that the participants were able to reduce the difference of screen times of male and female actors by 43%, and that of light-skinned and dark-skinned actors by 44%, thus showing the promise and potential of using such a tool in commercial production systems.

The formation of social groups is defined by the interactions among the group members. Studying this group formation process can be useful in understanding the status of members, decision-making behaviors, spread of knowledge and diseases, and much more. A defining characteristic of these groups is the pecking order or hierarchy the members form which help groups work towards their goals. One area of social science deals with understanding the formation and maintenance of these hierarchies, and in our work we provide social scientists with a visual analytics tool - PeckVis - to aid this process. While online social groups or social networks have been studied deeply and lead to a variety of analyses and visualization tools, the study of smaller groups in the field of social science lacks the support of suitable tools. Domain experts believe that visualizing their data can save them time as well as reveal findings they may have failed to observe. We worked alongside domain experts to build an interactive visual analytics system to investigate social hierarchies. Our system can discover patterns and relationships between the members of a group as well as compare different groups. The results are presented to the user in the form of an interactive visual analytics dashboard. We demonstrate that domain experts were able to effectively use our tool to analyze animal behavior data.

Organizing multivariate data spaces by their dimensions or attributes can be a rather difficult task. Most of the work in this area focuses on the statistical aspects such as correlation clustering, dimension reduction, and the like. These methods typically produce hierarchies in which the leaf nodes are labeled by the attribute names while the inner nodes are often represented by just a statistical measure and criterion, such as a threshold. This makes them difficult to understand for mainstream users. Taxonomies in science, biology, engineering, etc. on the other hand, are easy to comprehend since they provide meaningful labels at the inner nodes as well. Labeling inner nodes of taxonomies automatically requires the identification of hypernyms. Our proposed framework, called Taxonomizer, takes a visual analytics approach to meet this challenge. It appeals to the wisdom of humans to liaise with state of the art data analytics, neural word embeddings, and lexical databases. It consists of a set of visual tools that starts out with an automatically computed hierarchy where the leaf nodes are the original data attributes, and it then allows users to sculpt high-quality taxonomies for any multivariate dataset.

Presents an editorial which examines the current state of the journal and planned future directions for the publication.

Welcome to the 9thIEEE Transactions on Visualization and Computer Graphics (TVCG) special issue on IEEE Virtual Reality and 3D User Interfaces. This volume contains a total of 29 full papers selected for and presented at the IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2020) held in Atlanta, United States on March 22–26, 2020. Founded in 1993, IEEE VR has a long tradition as the premier venue where new research results in the field of Virtual Reality (VR) are presented. With the emergence of VR as a major technology in a diverse set of fields, such as entertainment, education, data analytics, artificial intelligence, medicine, construction, training, and many others, the papers presented at IEEE VR and published in theIEEE TVCG VRspecial issue mark a major highlight of the year.

Analyzing and visualizing eye movement data can provide useful insights into the connectivities and linkings of points and areas of interest (POIs and AOIs). Those typically time-varying relations can give hints about applied visual scanning strategies by either individual or many eye tracked people. However, the challenging issue with this kind of data is its spatio-temporal nature requiring a good visual encoding in order to first, achieve a scalable overview-based diagram, and second, to derive static or dynamic patterns that might correspond to certain comparable visual scanning strategies. To reliably identify the dynamic strategies we describe a visualization technique that generates a more linear representation of the spatio-temporal scan paths. This is achieved by applying different visual encodings of the spatial dimensions that typically build a limitation for an eye movement data visualization causing visual clutter effects, overdraw, and occlusions while the temporal dimension is depicted as a linear time axis. The presented interactive visualization concept is composed of three linked views depicting spatial, metrics-related, as well as distance-based aspects over time.

Eye movements recorded for many study participants are difficult to interpret, in particular when the task is to identify similar scanning strategies over space, time, and participants. In this paper we describe an approach in which we first compare scanpaths, not only based on Jaccard (JD) and bounding box (BB) similarities, but also on more complex approaches like longest common subsequence (LCS), Frechet distance (FD), dynamic time warping (DTW), and edit distance (ED). The results of these algorithms generate a weighted comparison matrix while each entry encodes the pairwise participant scanpath comparison strength. To better identify participant groups of similar eye movement behavior we reorder this matrix by hierarchical clustering, optimal-leaf ordering, dimensionality reduction, or a spectral approach. The matrix visualization is linked to the original stimulus overplotted with visual attention maps and gaze plots on which typical interactions like temporal, spatial, or participant-based filtering can be applied.

Yarbus' claim to decode the observer's task from eye movements has received mixed reactions. In this paper, we have supported the hypothesis that it is possible to decode the task. We conducted an exploratory analysis on the dataset by projecting features and data points into a scatter plot to visualize the nuance properties for each task. Following this analysis, we eliminated highly correlated features before training an SVM and Ada Boosting classifier to predict the tasks from this filtered eye movements data. We achieve an accuracy of 95.4% on this task classification problem and hence, support the hypothesis that task classification is possible from a user's eye movement data.

In this paper, we describe the design of an interactive visualization tool for the comparison of eye movement data with a special focus on the outliers. In order to make the tool usable and accessible to anyone with a data science background, we provide a web-based solution by using the Dash library based on the Python programming language and the Python library Plotly. Interactive visualization is very well supported by Dash, which makes the visualization tool easy to use. We support multiple ways of comparing user scanpaths like bounding boxes and Jaccard indices to identify similarities. Moreover, we support matrix reordering to clearly separate the outliers in the scanpaths. We further support the data analyst by complementary views such as gaze plots and visual attention maps.

For real-world learning tasks (e.g., classification), graph-based models are commonly used to fuse the information distributed in diverse data sources, which can be heterogeneous, redundant, and incomplete. These models represent the relations in different datasets as pairwise links. However, these links cannot deal with high-order relations which connect multiple objects (e.g., in public health datasets, more than two patient groups admitted by the same hospital in 2014). In this article, we propose a visual analytics approach for the classification on heterogeneous datasets using the hypergraph model. The hypergraph is an extension to traditional graphs in which a hyperedge connects multiple vertices instead of just two. We model various high-order relations in heterogeneous datasets as hyperedges and fuse different datasets with a unified hypergraph structure. We use the hypergraph learning algorithm for predicting missing labels in the datasets. To allow users to inject their domain knowledge into the model-learning process, we augment the traditional learning algorithm in a number of ways. In addition, we also propose a set of visualizations which enable the user to construct the hypergraph structure and the parameters of the learning model interactively during the analysis. We demonstrate the capability of our approach via two real-world cases.

A wide variety of color schemes have been devised for mapping scalar data to color. We address the challenge of color-mapping multivariate data. While a number of methods can map low-dimensional data to color, for example, using bilinear or barycentric interpolation for two or three variables, these methods do not scale to higher data dimensions. Likewise, schemes that take a more artistic approach through color mixing and the like also face limits when it comes to the number of variables they can encode. Our approach does not have these limitations. It is data driven in that it determines a proper and consistent color map from first embedding the data samples into a circular interactive multivariate color mapping display (ICD) and then fusing this display with a convex (CIE HCL) color space. The variables (data attributes) are arranged in terms of their similarity and mapped to the ICD's boundary to control the embedding. Using this layout, the color of a multivariate data sample is then obtained via modified generalized barycentric coordinate interpolation of the map. The system we devised has facilities for contrast and feature enhancement, supports both regular and irregular grids, can deal with multi-field as well as multispectral data, and can produce heat maps, choropleth maps, and diagrams such as scatterplots.

Presents the message from the new Editor-in-Chief.

Welcome to the November 2019 issue of the IEEE Transactions on Visualization and Computer Graphics (TVCG). This issue contains selected papers accepted at the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), held this year in Beijing, China from October 14 to October 18, 2019.

The thirty-three papers included in this special issue were presented at the 2019 8th Virtual Reality Conference that was held in Osaka, Japan, March 23-27, 2019.

Anomalous runtime behavior detection is one of the most important tasks for performance diagnosis in High Performance Computing (HPC). Most of the existing methods find anomalous executions based on the properties of individual functions, such as execution time. However, it is insufficient to identify abnormal behavior without taking into account the context of the executions, such as the invocations of children functions and the communications with other HPC nodes. We improve upon the existing anomaly detection approaches by utilizing the call stack structures of the executions, which record rich temporal and contextual information. With our call stack tree (CSTree) representation of the executions, we formulate the anomaly detection problem as finding anomalous tree structures in a call stack forest. The CSTrees are converted to vector representations using our proposed stack2vec embedding. Structural and temporal visualizations of CSTrees are provided to support users in the identification and verification of the anomalies during an active anomaly detection process. Three case studies of real-world HPC applications demonstrate the capabilities of our approach.

Iterative algorithms have become increasingly popular in computed tomography (CT) image reconstruction, since they better deal with the adverse image artifacts arising from low radiation dose image acquisition. But iterative methods remain computationally expensive. The main cost emerges in the projection and back projection operations, where accurate CT system modeling can greatly improve the quality of the reconstructed image. We present a framework that improves upon one particular aspect-the accurate projection of the image basis functions. It differs from current methods in that it substitutes the high computational complexity associated with accurate voxel projection by a small number of memory operations. Coefficients are computed in advance and stored in look-up tables parameterized by the CT system's projection geometry. The look-up tables only require a few kilobytes of storage and can be efficiently accelerated on the GPU. We demonstrate our framework with both numerical and clinical experiments and compare its performance with the current state-of-the-art scheme-the separable footprint method.

Over past several years, machine learning, or more generally artificial intelligence, has generated overwhelming research interest and attracted unprecedented public attention. As tomographic imaging researchers, we share the excitement from our imaging perspective [item 1) in the Appendix], and organized this special issue dedicated to the theme of "Machine learning for image reconstruction." This special issue is a sister issue of the special issue published in May 2016 of this journal with the theme "Deep learning in medical imaging" [item 2) in the Appendix]. While the previous special issue targeted medical image processing/analysis, this special issue focuses on data-driven tomographic reconstruction. These two special issues are highly complementary, since image reconstruction and image analysis are two of the main pillars for medical imaging. Together we cover the whole workflow of medical imaging: from tomographic raw data/features to reconstructed images and then extracted diagnostic features/readings.

Analyzing high-dimensional data and finding hidden patterns is a difficult problem and has attracted numerous research efforts. Automated methods can be useful to some extent but bringing the data analyst into the loop via interactive visual tools can help the discovery process tremendously. An inherent problem in this effort is that humans lack the mental capacity to truly understand spaces exceeding three spatial dimensions. To keep within this limitation, we describe a framework that decomposes a high-dimensional data space into a continuum of generalized 3D subspaces. Analysts can then explore these 3D subspaces individually via the familiar trackball interface while using additional facilities to smoothly transition to adjacent subspaces for expanded space comprehension. Since the number of such subspaces suffers from combinatorial explosion, we provide a set of data-driven subspace selection and navigation tools which can guide users to interesting subspaces and views. A subspace trail map allows users to manage the explored subspaces, keep their bearings, and return to interesting subspaces and views. Both trackball and trail map are each embedded into a word cloud of attribute labels which aid in navigation. We demonstrate our system via several use cases in a diverse set of application areas-cluster analysis and refinement, information discovery, and supervised training of classifiers. We also report on a user study that evaluates the usability of the various interactions our system provides.

X-ray images obtained from synchrotron beamlines are large-scale, high-resolution and high-dynamic-range grayscale data encoding multiple complex properties of the measured materials. They are typically associated with a variety of metadata which increases their inherent complexity. There is a wealth of information embedded in these data but so far scientists lack modern exploration tools to unlock these hidden treasures. To bridge this gap, we propose MultiSciView , a multivariate scientific x-ray image visualization and exploration system for beamline-generated x-ray scattering data. Our system is composed of three complementary and coordinated interactive visualizations to enable a coordinated exploration across the images and their associated attribute and feature spaces . The first visualization features a multi-level scatterplot visualization dedicated for image exploration in attribute, image, and pixel scales. The second visualization is a histogram-based attribute cross filter by which users can extract desired subset patterns from data. The third one is an attribute projection visualization designed for capturing global attribute correlations. We demonstrate our framework by ways of a case study involving a real-world material scattering dataset. We show that our system can efficiently explore large-scale x-ray images, accurately identify preferred image patterns, anomalous images and erroneous experimental settings, and effectively advance the comprehension of material nanostructure properties.

Color filter array (CFA) interpolation, or three-band demosaicking, is a process of interpolating the missing color samples in each band to reconstruct a full color image. In this paper, we are concerned with the challenging problem of multispectral demosaicking, where each band is significantly undersampled due to the increment in the number of bands. Specifically, we demonstrate a frequency-domain analysis of the subsampled color-difference signal and observe that the conventional assumption of highly correlated spectral bands for estimating undersampled components is not precise. Instead, such a spectral correlation assumption is image dependent and rests on the aliasing interferences among the various color-difference spectra. To address this problem, we propose an adaptive spectral-correlation-based demosaicking (ASCD) algorithm that uses a novel anti-aliasing filter to suppress these interferences, and we then integrate it with an intra-prediction scheme to generate a more accurate prediction for the reconstructed image. Our ASCD is computationally very simple, and exploits the spectral correlation property much more effectively than the existing algorithms. Experimental results conducted on two data sets for multispectral demosaicking and one data set for CFA demosaicking demonstrate that the proposed ASCD outperforms the state-of-the-art algorithms.

Oftentimes multivariate data are not available as sets of equally multivariate tuples, but only as sets of projections into subspaces spanned by subsets of these attributes. For example, one may find data with five attributes stored in six tables of two attributes each, instead of a single table of five attributes. This prohibits the visualization of these data with standard high-dimensional methods, such as parallel coordinates or MDS, and there is hence the need to reconstruct the full multivariate (joint) distribution from these marginal ones. Most of the existing methods designed for this purpose use an iterative procedure to estimate the joint distribution. With insufficient marginal distributions and domain knowledge, they lead to results whose joint errors can be large. Moreover, enforcing smoothness for regularizations in the joint space is not applicable if the attributes are not numerical but categorical. We propose a visual analytics approach that integrates both anecdotal data and human experts to iteratively narrow down a large set of plausible solutions. The solution space is populated using a Monte Carlo procedure which uniformly samples the solution space. A level-of-detail high dimensional visualization system helps the user understand the patterns and the uncertainties. Constraints that narrow the solution space can then be added by the user interactively during the iterative exploration, and eventually a subset of solutions with narrow uncertainty intervals emerges.

Numerous methods have been described that allow the visualization of the data matrix. But all suffer from a common problem - observing the data points in the context of the attributes is either impossible or inaccurate. We describe a method that allows these types of comprehensive layouts. We achieve it by combining two similarity matrices typically used in isolation - the matrix encoding the similarity of the attributes and the matrix encoding the similarity of the data points. This combined matrix yields two of the four submatrices needed for a full multi-dimensional scaling type layout. The remaining two submatrices are obtained by creating a fused similarity matrix - one that measures the similarity of the data points with respect to the attributes, and vice versa. The resulting layout places the data objects in direct context of the attributes and hence we call it the data context map. It allows users to simultaneously appreciate (1) the similarity of data objects, (2) the similarity of attributes in the specific scope of the collection of data objects, and (3) the relationships of data objects with attributes and vice versa. The contextual layout also allows data regions to be segmented and labeled based on the locations of the attributes. This enables, for example, the map's application in selection tasks where users seek to identify one or more data objects that best fit a certain configuration of factors, using the map to visually balance the tradeoffs.

Uncovering the causal relations that exist among variables in multivariate datasets is one of the ultimate goals in data analytics. Causation is related to correlation but correlation does not imply causation. While a number of casual discovery algorithms have been devised that eliminate spurious correlations from a network, there are no guarantees that all of the inferred causations are indeed true. Hence, bringing a domain expert into the casual reasoning loop can be of great benefit in identifying erroneous casual relationships suggested by the discovery algorithm. To address this need we present the Visual Causal Analyst-a novel visual causal reasoning framework that allows users to apply their expertise, verify and edit causal links, and collaborate with the causal discovery algorithm to identify a valid causal network. Its interface consists of both an interactive 2D graph view and a numerical presentation of salient statistical parameters, such as regression coefficients, p-values, and others. Both help users in gaining a good understanding of the landscape of causal structures particularly when the number of variables is large. Our framework is also novel in that it can handle both numerical and categorical variables within one unified model and return plausible results. We demonstrate its use via a set of case studies using multiple practical datasets.

Contextual layouts preserve the context of the data with the associated attributes (variables). However, their linear mapping causes errors in the layout - similar data points and variable nodes may not map to similar regions, and vice versa. In this paper, we first unify the various data layout schemes and choose the Generalized Bary-centric Coordinates (GBC) plot as the standard way to describe them. Second, we propose three algorithms - distance spaced lay-out, iterative error reduction, and force directed adjustment - to reduce the layout error of variables to variables, data to variables and data to data, respectively. We find that the combination of these three algorithms can yield large improvements in the layout error and so achieve a more comprehensive layout. Third, we describe an interface, the GBC Error Explorer, which allows users to explore the error using a variety of visualization schemes combined with some interactions.

Existing high-level, source-to-source compilers can accept input programs in a high-level language (e.g., C) and perform complex automatic parallelization and other mappings using various optimizations. These optimizations often require trade-offs and can benefit from the user's involvement in the process. However, because of the inherent complexity, the barrier to entry for new users of these high-level optimizing compilers can often be high. We propose visualization as an effective gateway for non-expert users to gain insight into the effects of parameter choices and so aid them in the selection of levels best suited to their specific optimization goals.

We propose the concept of teaching (and learning) unfamiliar visualizations by analogy, that is, demonstrating an unfamiliar visualization method by linking it to another more familiar one, where the in-betweens are designed to bridge the gap of these two visualizations and explain the difference in a gradual manner. As opposed to a textual description, our morphing explains an unfamiliar visualization through purely visual means. We demonstrate our idea by ways of four visualization pair examples: data table and parallel coordinates, scatterplot matrix and hyperbox, linear chart and spiral chart, and hierarchical pie chart and treemap. The analogy is commutative i.e. any member of the pair can be the unfamiliar visualization. A series of studies showed that this new paradigm can be an effective teaching tool. The participants could understand the unfamiliar visualization methods in all of the four pairs either fully or at least significantly better after they observed or interacted with the transitions from the familiar counterpart. The four examples suggest how helpful visualization pairings be identified and they will hopefully inspire other visualization morphings and associated transition strategies to be identified.

Correlation analysis can reveal the complex relationships that often exist among the variables in multivariate data. However, as the number of variables grows, it can be difficult to gain a good understanding of the correlation landscape and important intricate relationships might be missed. We previously introduced a technique that arranged the variables into a 2D layout, encoding their pairwise correlations. We then used this layout as a network for the interactive ordering of axes in parallel coordinate displays. Our current work expresses the layout as a correlation map and employs it for visual correlation analysis. In contrast to matrix displays where correlations are indicated at intersections of rows and columns, our map conveys correlations by spatial proximity which is more direct and more focused on the variables in play. We make the following new contributions, some unique to our map: (1) we devise mechanisms that handle both categorical and numerical variables within a unified framework, (2) we achieve scalability for large numbers of variables via a multi-scale semantic zooming approach, (3) we provide interactive techniques for exploring the impact of value bracketing on correlations, and (4) we visualize data relations within the sub-spaces spanned by correlated variables by projecting the data into a corresponding tessellation of the map.

Although the euclidean distance does well in measuring data distances within high-dimensional clusters, it does poorly when it comes to gauging intercluster distances. This significantly impacts the quality of global, low-dimensional space embedding procedures such as the popular multidimensional scaling (MDS) where one can often observe nonintuitive layouts. We were inspired by the perceptual processes evoked in the method of parallel coordinates which enables users to visually aggregate the data by the patterns the polylines exhibit across the dimension axes. We call the path of such a polyline its structure and suggest a metric that captures this structure directly in high-dimensional space. This allows us to better gauge the distances of spatially distant data constellations and so achieve data aggregations in MDS plots that are more cognizant of existing high-dimensional structure similarities. Our biscale framework distinguishes far-distances from near-distances. The coarser scale uses the structural similarity metric to separate data aggregates obtained by prior classification or clustering, while the finer scale employs the appropriate euclidean distance.

Clustering is an important preparation step in big data processing. It may even be used to detect redundant data points as well as outliers. Elimination of redundant data and duplicates can serve as a viable means for data reduction and it can also aid in sampling. Visual feedback is very valuable here to give users confidence in this process. Furthermore, big data preprocessing is seldom interactive, which stands at conflict with users who seek answers immediately. The best one can do is incremental preprocessing in which partial and hopefully quite accurate results become available relatively quickly and are then refined over time. We propose a correlation clustering framework which uses MDS for layout and GPU-acceleration to accomplish these goals. Our domain application is the correlation clustering of atmospheric mass spectrum data with 8 million data points of 450 dimensions each.

We investigate the strength of patterns as secret signatures in Android's pattern based authentication mechanism. Parallelism of GPU is exploited to exhaustively search for the secret pattern. Typically, searching for a pattern, composed of a number of nodes and edges, requires an exhaustive search for the pattern. In this work, we show that the use of GPU can speed up the graph search, hence the pattern password, through parallelization. Preliminary results on cracking the Android pattern based passwords shows that the technique can be used as the basis to implement a tool that can check the strength of a pattern based password and thereby recommend strong patterns to the user.

No abstract available.

Gaining a true appreciation of high-dimensional space remains difficult since all of the existing high-dimensional space exploration techniques serialize the space travel in some way. This is not so foreign to us since we, when traveling, also experience the world in a serial fashion. But we typically have access to a map to help with positioning, orientation, navigation, and trip planning. Here, we propose a multivariate data exploration tool that compares high-dimensional space navigation with a sightseeing trip. It decomposes this activity into five major tasks: 1) Identify the sights: use a map to identify the sights of interest and their location; 2) Plan the trip: connect the sights of interest along a specifyable path; 3) Go on the trip: travel along the route; 4) Hop off the bus: experience the location, look around, zoom into detail; and 5) Orient and localize: regain bearings in the map. We describe intuitive and interactive tools for all of these tasks, both global navigation within the map and local exploration of the data distributions. For the latter, we describe a polygonal touchpad interface which enables users to smoothly tilt the projection plane in high-dimensional space to produce multivariate scatterplots that best convey the data relationships under investigation. Motion parallax and illustrative motion trails aid in the perception of these transient patterns. We describe the use of our system within two applications: 1) the exploratory discovery of data configurations that best fit a personal preference in the presence of tradeoffs and 2) interactive cluster analysis via cluster sculpting in N-D.

High-dimensional data visualization has been attracting much attention. To fully test related software and algorithms, researchers require a diverse pool of data with known and desired features. Test data do not always provide this, or only partially. Here we propose the paradigm WYDIWYGS (What You Draw Is What You Get). Its embodiment, SketchPadND, is a tool that allows users to generate high-dimensional data in the same interface they also use for visualization. This provides for an immersive and direct data generation activity, and furthermore it also enables users to interactively edit and clean existing high-dimensional data from possible artifacts. SketchPadND offers two visualization paradigms, one based on parallel coordinates and the other based on a relatively new framework using an N-D polygon to navigate in high-dimensional space. The first interface allows users to draw arbitrary profiles of probability density functions along each dimension axis and sketch shapes for data density and connections between adjacent dimensions. The second interface embraces the idea of sculpting. Users can carve data at arbitrary orientations and refine them wherever necessary. This guarantees that the data generated is truly high-dimensional. We demonstrate our tool's usefulness in real data visualization scenarios.

The Five Ws is a popular concept for information gathering in journalistic reporting. It captures all aspects of a story or incidence: who, when, what, where, and why. We propose a framework composed of a suite of cooperating visual information displays to represent the Five Ws and demonstrate its use within a healthcare informatics application. Here, the who is the patient, the where is the patient's body, and the when, what, why is a reasoning chain which can be interactively sorted and brushed. The patient is represented as a radial sunburst visualization integrated with a stylized body map. This display captures all health conditions of the past and present to serve as a quick overview to the interrogating physician. The reasoning chain is represented as a multistage flow chart, composed of date, symptom, data, diagnosis, treatment, and outcome. Our system seeks to improve the usability of information captured in the electronic medical record (EMR) and we show via multiple examples that our framework can significantly lower the time and effort needed to access the medical patient information required to arrive at a diagnostic conclusion.

The navigation of high-dimensional data spaces remains challenging, making multivariate data exploration difficult. To be effective and appealing for mainstream application, navigation should use paradigms and metaphors that users are already familiar with. One such intuitive navigation paradigm is interactive route planning on a connected network. We have employed such an interface and have paired it with a prominent high-dimensional visualization paradigm showing the N-D data in undistorted raw form: parallel coordinates. In our network interface, the dimensions form nodes that are connected by a network of edges representing the strength of association between dimensions. A user then interactively specifies nodes/edges to visit, and the system computes an optimal route, which can be further edited and manipulated. In our interface, this route is captured by a parallel coordinate data display in which the dimension ordering is configured by the specified route. Our framework serves both as a data exploration environment and as an interactive presentation platform to demonstrate, explain, and justify any identified relationships to others. We demonstrate our interface within a business scenario and other applications.

Due to the inherent characteristics of the visualization process, most of the problems in this field have strong ties with human cognition and perception. This makes the human brain and sensory system the only truly appropriate evaluation platform for evaluating and fine-tuning a new visualization method or paradigm. However, getting humans to volunteer for these purposes has always been a significant obstacle, and thus this phase of the development process has traditionally formed a bottleneck, slowing down progress in visualization research. We propose to take advantage of the newly emerging field of Human Computation (HC) to overcome these challenges. HC promotes the idea that rather than considering humans as users of the computational system, they can be made part of a hybrid computational loop consisting of traditional computation resources and the human brain and sensory system. This approach is particularly successful in cases where part of the computational problem is considered intractable using known computer algorithms but is trivial to common sense human knowledge. In this paper, we focus on HC from the perspective of solving visualization problems and also outline a framework by which humans can be easily seduced to volunteer their HC resources. We introduce a purpose-driven game titled "Disguise" which serves as a prototypical example for how the evaluation of visualization algorithms can be mapped into a fun and addicting activity, allowing this task to be accomplished in an extensive yet cost effective way. Finally, we sketch out a framework that transcends from the pure evaluation of existing visualization methods to the design of a new one.

Color mapping and semitransparent layering play an important role in many visualization scenarios, such as information visualization and volume rendering. The combination of color and transparency is still dominated by standard alpha-compositing using the Porter-Duff over operator which can result in false colors with deceiving impact on the visualization. Other more advanced methods have also been proposed, but the problem is still far from being solved. Here we present an alternative to these existing methods specifically devised to avoid false colors and preserve visual depth ordering. Our approach is data driven and follows the recently formulated knowledge-assisted visualization (KAV) paradigm. Preference data, that have been gathered in web-based user surveys, are used to train a support-vector machine model for automatically predicting an optimized hue-preserving blending. We have applied the resulting model to both volume rendering and a specific information visualization technique, illustrative parallel coordinate plots. Comparative renderings show a significant improvement over previous approaches in the sense that false colors are completely removed and important properties such as depth ordering and blending vividness are better preserved. Due to the generality of the defined data-driven blending operator, it can be easily integrated also into other visualization frameworks.

We present the conformal magnifier, a novel interactive focus+context visualization technique that magnifies a region of interest (ROI) using conformal mapping. Our framework supports the arbitrary shape design of magnifiers for the user to enlarge the ROI while globally deforming the context region without any cropping. By using the mathematically well-defined conformal mapping theory and algorithm, the ROI is magnified with local shape preservation (angle distortion minimization), while the transition area between the focus and context regions is deformed smoothly and continuously. After the selection of a specified magnifier shape, our system can automatically magnify the ROI in real time with full resolution even for large volumetric data sets. These properties are important for many visualization applications, especially for the computer aided detection and diagnosis (CAD). Our framework is suitable for diverse applications, including the map visualization, and volumetric visualization. Experimental results demonstrate the effectiveness, robustness, and efficiency of our framework.

The unguided visual exploration of volumetric data can be both a challenging and a time-consuming undertaking. Identifying a set of favorable vantage points at which to start exploratory expeditions can greatly reduce this effort and can also ensure that no important structures are being missed. Recent research efforts have focused on entropy-based viewpoint selection criteria that depend on scalar values describing the structures of interest. In contrast, we propose a viewpoint suggestion pipeline that is based on feature-clustering in high-dimensional space. We use gradient/normal variation as a metric to identify interesting local events and then cluster these via k-means to detect important salient composite features. Next, we compute the maximum possible exposure of these composite feature for different viewpoints and calculate a 2D entropy map parameterized in longitude and latitude to point out promising view orientations. Superimposed onto an interactive track-ball interface, users can then directly use this entropy map to quickly navigate to potentially interesting viewpoints where visibility-based transfer functions can be employed to generate volume renderings that minimize occlusions. To give full exploration freedom to the user, the entropy map is updated on the fly whenever a view has been selected, pointing to new and promising but so far unseen view directions. Alternatively, our system can also use a set-cover optimization algorithm to provide a minimal set of views needed to observe all features. The views so generated could then be saved into a list for further inspection or into a gallery for a summary presentation.

Practical volume visualization pipelines are never without compromises and errors. A delicate and often-studied component is the interpolation of off-grid samples, where aliasing can lead to misleading artifacts and blurring, potentially hiding fine details of critical importance. The verifiable visualization framework we describe aims to account for these errors directly in the volume generation stage, and we specifically target volumetric data obtained via computed tomography (CT) reconstruction. In this case the raw data are the X-ray projections obtained from the scanner and the volume data generation process is the CT algorithm. Our framework informs the CT reconstruction process of the specific filter intended for interpolation in the subsequent visualization process, and this in turn ensures an accurate interpolation there at a set tolerance. Here, we focus on fast trilinear interpolation in conjunction with an octree-type mixed resolution volume representation without T-junctions. Efficient rendering is achieved by a space-efficient and locality-optimized representation, which can straightforwardly exploit fast fixed-function pipelines on GPUs.

The Lattice Boltzmann method (LBM) for visual simulation of fluid flow generally employs cubic Cartesian (CC) lattices such as the D3Q13 and D3Q19 lattices for the particle transport. However, the CC lattices lead to suboptimal representation of the simulation space. We introduce the face-centered cubic (FCC) lattice, fD3Q13, for LBM simulations. Compared to the CC lattices, the fD3Q13 lattice creates a more isotropic sampling of the simulation domain and its single lattice speed (i.e., link length) simplifies the computations and data storage. Furthermore, the fD3Q13 lattice can be decomposed into two independent interleaved lattices, one of which can be discarded, which doubles the simulation speed. The resulting LBM simulation can be efficiently mapped to the GPU, further increasing the computational performance. We show the numerical advantages of the FCC lattice on channeled flow in 2D and the flow-past-a-sphere benchmark in 3D. In both cases, the comparison is against the corresponding CC lattices using the analytical solutions for the systems as well as velocity field visualizations. We also demonstrate the performance advantages of the fD3Q13 lattice for interactive simulation and rendering of hot smoke in an urban environment using thermal LBM.

Abstract Illustrative parallel coordinates (IPC) is a suite of artistic rendering techniques for augmenting and improving parallel coordinate (PC) visualizations. IPC techniques can be used to convey a large amount of information about a multidimensional dataset in a small area of the screen through the following approaches: (a) edge‐bundling through splines; (b) visualization of “branched ” clusters to reveal the distribution of the data; (c) opacity‐based hints to show cluster density; (d) opacity and shading effects to illustrate local line density on the parallel axes; and (e) silhouettes, shadows and halos to help the eye distinguish between overlapping clusters. Thus, the primary goal of this work is to convey as much information as possible in a manner that is aesthetically pleasing and easy to understand for non‐experts.

Professional designers and artists are quite cognizant of the rules that guide the design of effective color palettes, from both aesthetic and attention-guiding points of view. In the field of visualization, however, the use of systematic rules embracing these aspects has received less attention. The situation is further complicated by the fact that visualization often uses semi-transparencies to reveal occluded objects, in which case the resulting color mixing effects add additional constraints to the choice of the color palette. Color design forms a crucial part in visual aesthetics. Thus, the consideration of these issues can be of great value in the emerging field of illustrative visualization. We describe a knowledge-based system that captures established color design rules into a comprehensive interactive framework, aimed to aid users in the selection of colors for scene objects and incorporating individual preferences, importance functions, and overall scene composition. Our framework also offers new knowledge and solutions for the mixing, ordering and choice of colors in the rendering of semi-transparent layers and surfaces. All design rules are evaluated via user studies, for which we extend the method of conjoint analysis to task-based testing scenarios. Our framework's use of principles rooted in color design with application for the illustration of features in pre-classified data distinguishes it from existing systems which target the exploration of continuous-range density data via perceptual color maps.

Volumetric Subdivision (VS) is a powerful paradigm that enables volumetric sculpting and realistic volume deformations that give rise to the concept of "virtual clay". In VS, volumes are commonly represented as a space-filling set of deformed polyhedra, which can be further decomposed into a mesh of tetrahedra for rendering. Images can then be generated via tetrahedral projection or raycasting. A current shortcoming in VS-based operations is the need for a very high level of subdivision to represent fine detail in the mesh and to obtain a high-fidelity visualization. However, we have discovered that the subdivision process itself can be closely simulated with radial basis functions (RBFs), making it possible to replace the finer subdivision levels by a coarser aggregation of RBF kernels. This reduction to a simplified assembly of RBFs subsequently enables interactive rendering of volumetric subdivision shapes within a GPU-based volume splatting framework.

Visualization algorithms can have a large number of parameters, making the space of possible rendering results rather high-dimensional. Only a systematic analysis of the perceived quality can truly reveal the optimal setting for each such parameter. However, an exhaustive search in which all possible parameter permutations are presented to each user within a study group would be infeasible to conduct. Additional complications may result from possible parameter co-dependencies. Here, we will introduce an efficient user study design and analysis strategy that is geared to cope with this problem. The user feedback is fast and easy to obtain and does not require exhaustive parameter testing. To enable such a framework we have modified a preference measuring methodology, conjoint analysis, that originated in psychology and is now also widely used in market research. We demonstrate our framework by a study that measures the perceived quality in volume rendering within the context of large parameter spaces.

We describe a novel volumetric global illumination framework based on the Face-Centered Cubic (FCC) lattice. An FCC lattice has important advantages over a Cartesian lattice. It has higher packing density in the frequency domain, which translates to better sampling efficiency. Furthermore, it has the maximal possible kissing number (equivalent to the number of nearest neighbors of each site), which provides optimal 3D angular discretization among all lattices. We employ a new two-pass (illumination and rendering) global illumination scheme on an FCC lattice. This scheme exploits the angular discretization to greatly simplify the computation in multiple scattering and to minimize illumination information storage. The GPU has been utilized to further accelerate the rendering stage. We demonstrate our new framework with participating media and volume rendering with multiple scattering, where both are significantly faster than traditional techniques with comparable quality.

We provide a physically-based framework for simulating the natural phenomena related to heat interaction between objects and the surrounding air. We introduce a heat transfer model between the heat source objects and the ambient flow environment, which includes conduction, convection, and radiation. The heat distribution of the objects is represented by a novel temperature texture. We simulate the thermal flow dynamics that models the air flow interacting with the heat by a hybrid thermal lattice Boltzmann model (HTLBM). The computational approach couples a multiple-relaxation-time LBM (MRTLBM) with a finite difference discretization of a standard advection-diffusion equation for temperature. In heat shimmering and mirage, the changes in the index of refraction of the surrounding air are attributed to temperature variation. A nonlinear ray tracing method is used for rendering. Interactive performance is achieved by accelerating the computation of both the MRTLBM and the heat transfer, as well as the rendering on contemporary graphics hardware (GPU).

Radial Basis Functions (RBFs) have become a popular rendering primitive, both in surface and in volume rendering. This paper focuses on volume visualization, giving rise to 3D kernels. RBFs are especially convenient for the representation of scattered and irregularly distributed point samples, where the RBF kernel is used as a blending function for the space in between samples. Common representations employ radially symmetric RBFs, and various techniques have been introduced to render these, also with efficient implementations on programmable graphics hardware (GPUs). In this paper, we extend the existing work to more generalized, ellipsoidal RBF kernels, for the rendering of scattered volume data. We devise a post-shaded kernel-centric rendering approach, specifically designed to run efficiently on GPUs, and we demonstrate our renderer using datasets from subdivision volumes and computational science.

Based on the finding that refraction imposes significantly higher demands onto gradient filters than illumination and shading, we evaluate the family of spline filters as a good alternative to the cubic filters, which so far have served as the gold standard of efficient yet high-quality interpolation filters in present visualization applications. Using a regular background texture to visualize the refractive properties of the volumetric object, we also describe an efficient scheme to achieve the effects of supersampling without incurring any extra raycasting overhead. Our results indicate that splines can be superior to the Catmull-Rom filter, with potentially less computational overhead, also offering a convenient means to adjust the extent of lowpassing and smoothing.

The size and resolution of volume datasets in science and medicine are increasing at a rate much greater than the resolution of the screens used to view them. This limits the amount of data that can be viewed simultaneously, potentially leading to a loss of overall context of the data when the user views or zooms into a particular area of interest. We propose a focus+context framework that uses various standard and advanced magnification lens rendering techniques to magnify the features of interest, while compressing the remaining volume regions without clipping them away completely. Some of these lenses can be interactively configured by the user to specify the desired magnification patterns, while others are feature-adaptive. All our lenses are accelerated on the GPU. They allow the user to interactively manage the available screen area, dedicating more area to the more resolution-important features.

THIS issue contains extended versions of three of the bestrated application papers taken from the IEEE Visualization 2004 Conference. Application papers present the contribution of visualization techniques toward the understanding of application-specific data. In particular, application papers must explain the effectiveness of the visualization methods for the particular application domain. For the IEEE Visualization 2004 Conference, 24 out of 71 submitted applications papers were chosen for inclusion in the conference program. As paper cochairs of the IEEE Visualization applications track, we started from the opinion of the reviewers. From the 24 accepted papers, we have chosen three very high quality submissions and asked the authors to provide extended and revised versions of their original work for this issue. Each of the extended application papers was thoroughly reviewed by several experts and underwent revisions before it was recommended for acceptance. We are grateful to the reviewers for their detailed and timely reviews and to David Ebert, Editor-in-Chief of TVCG, for his strong support and assistance. The paper “Reconstruction and Visualization of Planetary Nebulae” by Marcus Magnor, Gordon Kindlmann, Charles Hansen, and Neb Duric exploits strong symmetry characteristics of planetary nebulae to recover spatial structures from 2D images. With GPU-based volume rendering and nonlinear optimization, the nebula’s emission density is recovered. Fascinating 3D visualizations of planetary nebulae are created, which further educational purposes and can support astrophysicists in better understanding the formation process of these phenomena. The paper “Advanced Virtual Endoscopic Pituitary Surgery” by Andre Neubauer, Stefan Wolfsberger, MarieTherese Forster, Lukas Mroz, Rainer Wegenkittl, and Katja Buhler investigates a minimally invasive endoscopic procedure for tumor removal. STEPS, a virtual endoscopy system, is presented to train surgeons on the transsphenoidal approach and assist experienced surgeons in planning a real endoscopic intervention. Interactive visualization is achieved through first-hit ray casting. The application provides various navigation and perception aids in the simulation of the surgical procedure. The paper “Visualization of Geologic Stress Perturbations Using Mohr Diagrams” by Patricia Crossno, David H. Rogers, Rebecca M. Brannon, David Coblentz, and Joanne T. Fredrich presents an inspection tool for finite element analyses of 3D real-valued second-order tensors. The Mohr diagram is a paper and pencil method from the material mechanics community. The authors originate the adaptation of this diagram type to visualize perturbed in-situ stress fields of geologic features. The Mohr diagram is used as an interactive glyph for probing as well as filtering through brushing and linking of principal stresses. Scientific visualization research is largely application driven. Many techniques are strongly motivated by the application scenario for which they were developed. Visualization research cannot be a goal by itself. It just acts as an interface tool to the domain expert to facilitate his understanding of his data and problems. Applications are therefore crucial in providing fruitful incitation for the further research of the visualization experts. The papers presented here are prime examples of how visualization helps to understand intricate application phenomena. We hope that the readers will find the papers inspiring and stimulating samples of work from our fascinating field.

Resampling is a frequent task in visualization and medical imaging. It occurs whenever images or volumes are magnified, rotated, translated, or warped. Resampling is also an integral procedure in the registration of multimodal datasets, such as CT, PET, and MRI, in the correction of motion artifacts in MRI, and in the alignment of temporal volume sequences in fMRI. It is well known that the quality of the resampling result depends heavily on the quality of the interpolation filter used. However, high-quality filters are rarely employed in practice due to their large spatial extents. We explore a new resampling technique that operates in the frequency-domain where high-quality filtering is feasible. Further, unlike previous methods of this kind, our technique is not limited to integer-ratio scaling factors, but can resample image and volume datasets at any rate. This would usually require the application of slow discrete Fourier transforms (DFT) to return the data to the spatial domain. We studied two methods that successfully avoid these delays: the chirp-z transform and the FFTW package. We also outline techniques to avoid the ringing artifacts that may occur with frequency-domain filtering. Thus, our method can achieve high-quality interpolation at speeds that are usually associated with spatial filters of far lower quality.

We present a system for simulating and visualizing the propagation of dispersive contaminants with an application to urban security. In particular, we simulate airborne contaminant propagation in open environments characterised by sky-scrapers and deep urban canyons. Our approach is based on the multiple relaxation time lattice Boltzmann model (MRTLBM), which can efficiently handle complex boundary conditions such as buildings. In addition, we model thermal effects on the flow field using the hybrid thermal MRTLBM. Our approach can also accommodate readings from various sensors distributed in the environment and adapt the simulation accordingly. We accelerate the computation and efficiently render many buildings with small textures on the GPU. We render streamlines and the contaminant smoke with self-shadowing composited with the textured buildings.

A common deficiency of discretized datasets is that detail beyond the resolution of the dataset has been irrecoverably lost. This lack of detail becomes immediately apparent once one attempts to zoom into the dataset and only recovers blur. We describe a method that generates the missing detail from any available and plausible high-resolution data, using texture synthesis. Since the detail generation process is guided by the underlying image or volume data and is designed to fill in plausible detail in accordance with the coarse structure and properties of the zoomed-in neighborhood, we refer to our method as constrained texture synthesis. Regular zooms become "semantic zooms", where each level of detail stems from a data source attuned to that resolution. We demonstrate our approach by a medical application - the visualization of a human liver - but its principles readily apply to any scenario, as long as data at all resolutions are available. We first present a 2D viewing application, called the "virtual microscope", and then extend our technique to 3D volumetric viewing.

We present a physically-based, yet fast and simple method to simulate gaseous phenomena. In our approach, the incompressible Navier-Stokes (NS) equations governing fluid motion have been modeled in a novel way to achieve a realistic animation. We introduce the Lattice Boltzmann Model (LBM), which simulates the microscopic movement of fluid particles by linear and local rules on a grid of cells so that the macroscopic averaged properties obey the desired NS equations. The LBM is defined on a 2D or 3D discrete lattice, which is used to solve fluid animation based on different boundary conditions. The LBM simulation generates, in real-time, an accurate velocity field and can incorporate an optional temperature field to account for the buoyancy force of hot gas. Because of the linear and regular operations in each local cell of the LBM grid, we implement the computation in commodity texture hardware, further improving the simulation speed. Finally, textured splats are used to add small scale turbulent details, achieving high-quality real-time rendering. Our method can also simulate the physically correct action of stationary or mobile obstacles on gaseous phenomena in real-time, while still maintaining highly plausible visual details.

We explore real time volume rendering of multichannel data for volumes with color and multimodal information. We demonstrate volume rendering of the visible human male color dataset and photorealistic rendering of voxelized terrains, and achieve high quality visualizations. We render multimodal volumes utilizing hardware programmability for accumulation level mixing, and use CT and MRI information as examples. We also use multiboard parallel/distributed rendering schemes for large datasets and investigate scalability issues. We employ the VolumePro 1000 for real time multichannel volume rendering. Our approach, however, is not hardware-specific and can use commodity texture hardware instead.

We describe a software-based method for interactive transfer function modification. Our approach exploits the fact that, in general, a user will rarely want to modify the viewpoint and the transfer functions at the same time. In that spirit, we optimize the latter by first fixing the viewpoint and then storing a compressed list of samples along each ray. Then, each time the transfer function is modified, the algorithm traverses the compressed sample lists, decompressing the runs and compositing the newly colored samples along each ray until full opacity is reached. Since the expensive sample interpolation and shading is no longer necessary, we can obtain near-interactive framerates for a variety of datasets, while our RLE-based compression of linearly varying sample runs helps keep the storage complexity down, with little decompression overhead. Decompression cost is reduced by storing decompression results in an on-the-fly constructed 2D table.

We propose methods to accelerate texture-based volume rendering by skipping invisible voxels. We partition the volume into sub-volumes, each containing voxels with similar properties. Sub-volumes composed of only voxels mapped to empty by the transfer function are skipped. To render the adaptively partitioned sub-volumes in visibility order, we reorganize them into an orthogonal BSP tree. We also present an algorithm that computes incrementally the intersection of the volume with the slicing planes, which avoids the overhead of the intersection and texture coordinates computation introduced by the partitioning. Rendering with empty space skipping is 2 to 5 times faster than without it. To skip occluded voxels, we introduce the concept of orthogonal opacity map, that simplifies the transformation between the volume coordinates and the opacity map coordinates, which is intensively used for occlusion detection. The map is updated efficiently by the GPU. The sub-volumes are then culled and clipped against the opacity map. We also present a method that adaptively adjusts the optimal number of the opacity map updates. With occlusion clipping, about 60% of non-empty voxels can be skipped and an additional 80% speedup on average is gained for iso-surface-like rendering.

This paper introduces a method for converting an image or volume sampled on a regular grid into a space-efficient irregular point hierarchy. The conversion process retains the original frequency characteristics of the dataset by matching the spatial distribution of sample points with the required frequency. To achieve good blending, the spherical points commonly used in volume rendering are generalized to ellipsoidal point primitives. A family of multiresolution, oriented Gabor wavelets provide the frequency-space analysis of the dataset. The outcome of this frequency analysis is the reduced set of points, in which the sampling rate is decreased in originally oversampled areas. During rendering, the traversal of the hierarchy can be controlled by any suitable error metric or quality criteria. The local level of refinement is also sensitive to the transfer function. Areas with density ranges mapped to high transfer function variability are rendered at higher point resolution than others. Our decomposition is flexible and can be used for iso-surface rendering, alpha compositing and X-ray rendering of volumes. We demonstrate our hierarchy with an interactive splatting volume renderer, in which the traversal of the point hierarchy for rendering is modulated by a user-specified frame rate.

This paper describes a software-based method for interactive transfer function modification. Our approach exploits the fact that, in general, a user will rarely want to modify the viewpoint and the transfer functions at the same time. In that spirit, we optimize the latter by first fixing the viewpoint and then storing a list of pre-shaded, but uncolored, samples along each ray. Then, each time the RGBA transfer function is modified, the algorithm traverses the sample lists, colors the samples, and composites them along each ray until full opacity is reached. Since neither the expensive sample interpolation nor the shading are no longer necessary, we can obtain near-interactive framerates for a variety of datasets.

We propose the use of textured splats as the basic display primitives for an open surface fire model. The high-detail textures help to achieve a smooth boundary of the fire and gain the small-scale turbulence appearance. We utilize the Lattice Boltzmann Model (LBM) to simulate physically-based equations describing the fire evolution and its interaction with the environment (e.g., obstacles, wind and temperature). The property of fuel and non-burning objects are defined on the lattice of the computation domain. A temperature field is also incorporated to model the generation of smoke from the fire due to incomplete combustion. The linear and local characteristics of the LBM enable us to accelerate the computation with graphics hardware to reach real-time simulation speed, while the texture splat primitives enable interactive rendering frame rates.

We introduce a general technique for "colorizing" greyscale images by transferring color between a source, color image and a destination, greyscale image. Although the general problem of adding chromatic values to a greyscale image has no exact, objective solution, the current approach attempts to provide a method to help minimize the amount of human labor required for this task. Rather than choosing RGB colors from a palette to color individual components, we transfer the entire color "mood" of the source to the target image by matching luminance and texture information between the images. We choose to transfer only chromatic information and retain the original luminance values of the target image. Further, the procedure is enhanced by allowing the user to match areas of the two images with rectangular swatches. We show that this simple technique can be successfully applied to a variety of images and video, provided that texture and luminance are sufficiently distinct. The images generated demonstrate the potential and utility of our technique in a diverse set of application domains.

Brain imaging methods used in experimental brain research such as Positron Emission Tomography (PET) and Functional Magnetic Resonance (fMRI) require the analysis of large amounts of data. Exploratory statistical methods can be used to generate new hypotheses and to provide a reliable measure of a given effect. Typically, researchers report their findings by listing those regions which show significant statistical activity in a group of subjects under some experimental condition or task. A number of methods create statistical parametric maps (SPMs) of the brain on a voxel-basis. In our approach statistics are computed not on individual voxels but on predefined anatomical regions-of-interest (ROIs). A correlation coefficient is used to quantify similarity in response for various regions during an experimental setting. Since the functional inter-relationships can become rather complex and spatially widespread, they are best understood in the context of the underlying 3-D brain anatomy. However despite the power of the 3-D model, the relative location of ROIs in 3-D can be obscured due the inherent problem of presenting 3-D spatial information on a 2-D screen. In order to address this problem, we have explored a number of visualization techniques to aid the brain researcher in exploring the spatial relationships of brain activity. In this paper we present a novel 3-D interface that allows the interactive exploration of correlation datasets.

Splatting is widely applied in many areas, including volume, point-based and image-based rendering. Improvements to splatting, such as eliminating popping and color bleeding, occasion-based acceleration, post-rendering classification and shading, have all been recently accomplished. These improvements share a common need for efficient frame-buffer accesses. We present an optimized software splatting package, using a newly designed primitive, called FastSplat, to scan-convert footprints. Our approach does not use texture mapping hardware, but supports the whole pipeline in memory. In such an integrated pipeline, we are then able to study the optimization strategies and address image quality issues. While this research is meant for a study of the inherent trade-off of splatting, our renderer, purely in software, achieves 3- to 5-fold speedups over a top-end texture hardware implementation (for opaque data sets). We further propose a method of efficient occlusion culling using a summed area table of opacity. 3D solid texturing and bump mapping capabilities are demonstrated to show the flexibility of such an integrated rendering pipeline. A detailed numerical error analysis, in addition to the performance and storage issues, is also presented. Our approach requires low storage and uses simple operations. Thus, it is easily implementable in hardware.

Algebraic reconstruction methods, such as the algebraic reconstruction technique (ART) and the related simultaneous ART (SART). reconstruct a two-dimensional (2-D) or three-dimensional (3-D) object from its X-ray projections. The algebraic methods have, in certain scenarios, many advantages over the more popular Filtered Backprojection approaches and have also recently been shown to perform well for 3-D cone-beam reconstruction. However, so far the slow speed of these iterative methods have prohibited their routine use in clinical applications. In this paper, we address this shortcoming and investigate the utility of widely available 2-D texture mapping graphics hardware for the purpose of accelerating the 3-D algebraic reconstruction. We find that this hardware allows 3-D cone-beam reconstructions to be obtained at almost interactive speeds, with speed-ups of over 50 with respect to implementations that only use general-purpose CPUs. However, we also find that the reconstruction quality is rather sensitive to the resolution of the framebuffer, and to address this critical issue we propose a scheme that extends the precision of a given framebuffer by 4 bits, using the color channels. With this extension, a 12-bit framebuffer delivers useful reconstructions for 0.5% tissue contrast, while an 8-bit framebuffer requires 4%. Since graphics hardware generates an entire image for each volume projection, it is most appropriately used with an algebraic reconstruction method that performs volume correction at that granularity as well, such as SART or SIRT. We chose SART for its faster convergence properties.

Splatting is a volume rendering algorithm that combines efficient volume projection with a sparse data representation. Only voxels that have values inside the iso-range need to be considered, and these voxels can be projected via efficient rasterization schemes. In splatting, each projected voxel is represented as a radially symmetric interpolation kernel, equivalent to a fuzzy ball. Projecting such a basis function leaves a fuzzy impression, called a footprint or splat, on the screen. Splatting traditionally classifies and shades the voxels prior to projection, and thus each voxel footprint is weighted by the assigned voxel color and opacity. Projecting these fuzzy color balls provides a uniform screen image for homogeneous object regions, but leads to a blurry appearance of object edges. The latter is clearly undesirable, especially when the view is zoomed on the object. In this work, we manipulate the rendering pipeline of splatting by performing the classification and shading process after the voxels have been projected onto the screen. In this way volume contributions outside the iso-range never affect the image. Since shading requires gradients, we not only splat the density volume, using regular splats, but we also project the gradient volume, using gradient splats. However alternative to gradient splats, we can also compute the gradients on the projection plane using central differencing. This latter scheme cuts the number of footprint rasterization by a factor of four since only the voxel densities have to be projected.

This paper examines the use of the algebraic reconstruction technique (ART) and related techniques to reconstruct 3-D objects from a relatively sparse set of cone-beam projections. Although ART has been widely used for cone-beam reconstruction of high-contrast objects, e.g., in computed angiography, the work presented here explores the more challenging low-contrast case which represents a little-investigated scenario for ART. Preliminary experiments indicate that for cone angles greater than 20 degrees, traditional ART produces reconstructions with strong aliasing artifacts. These artifacts are in addition to the usual off-midplane inaccuracies of cone-beam tomography with planar orbits. We find that the source of these artifacts is the nonuniform reconstruction grid sampling and correction by the cone-beam rays during the ART projection-backprojection procedure. A new method to compute the weights of the reconstruction matrix is devised, which replaces the usual constant-size interpolation filter by one whose size and amplitude is dependent on the source-voxel distance. This enables the generation of reconstructions free of cone-beam aliasing artifacts, at only little extra cost. An alternative analysis reveals that simultaneous ART (SART) also produces reconstructions without aliasing artifacts, however, at greater computational cost. Finally, we thoroughly investigate the influence of various ART parameters, such as volume initialization, relaxation coefficient lambda, correction scheme, number of iterations, and noise in the projection data on reconstruction quality. We find that ART typically requires only three iterations to render satisfactory reconstruction results.

The prime motivation of this work is to devise techniques that make the algebraic reconstruction technique (ART) and related methods more efficient for routine clinical use, while not compromising their accuracy. Since most of the computational effort of ART is spent for projection/backprojection operations, we first seek to optimize the projection algorithm. Existing projection algorithms are surveyed and it is found that these algorithms either lack accuracy or speed, or are not suitable for cone-beam reconstruction. We hence devise a new and more accurate extension to the splatting algorithm, a well-known voxel-driven projection method. We also describe a new three-dimensional (3-D) ray-driven projector that is considerably faster than the voxel-driven projector and, at the same time, more accurate and perfectly suited for the demands of cone beam. We then devise caching schemes for both ART and simultaneous ART (SART), which minimize the number of redundant computations for projection and backprojection and, at the same time, are very memory conscious. We find that with caching, the cost for an ART projection/backprojection operation can be reduced to the equivalent cost of 1.12 projections. We also find that SART, due to its image-based volume correction scheme, is considerably harder to accelerate with caching. Implementations of the algorithms yield run-time ratios TSART/TART between 1.5 and 1.15, depending on the amount of caching used.

Splatting is a popular volume rendering algorithm that pairs good image quality with an efficient volume projection scheme. The current axis-aligned sheet-buffer approach, however, bears certain inaccuracies. The effect of these is less noticeable in still images, but clearly revealed in animated viewing, where disturbing popping of object brightness occurs at certain view angle transitions. In previous work, we presented a new variant of sheet-buffered splatting in which the compositing sheets are oriented parallel to the image plane. This scheme not only eliminates the condition for popping, but also produces images of higher quality. In this paper, we summarize this new paradigm and extend it in a number of ways. We devise a new solution to render rectilinear grids of equivalent cost to the traditional approach that treats the anisotropic volume as being warped into a cubic grid. This enables us to use the usual radially symmetric kernels, which can be projected without inaccuracies. Next, current splatting approaches necessitate the projection of all voxels in the iso-interval(s), although only a subset of these voxels may eventually be visible in the final image. To eliminate these wasteful computations we propose a novel front-to-back approach that employs an occlusion map to determine if a splat contributes to the image before it is projected, thus skipping occluded splats. Additional measures are presented for further speedups. In addition, we present an efficient list-based volume traversal scheme that facilitates the quick modification of transfer functions and iso-values.

Splatting is a fast volume rendering algorithm which achieves its speed by projecting voxels in the form of pre-integrated interpolation kernels, or splats. Presently, two main variants of the splatting algorithm exist: (i) the original method, in which all splats are composited back-to-front, and (ii) the sheet-buffer method, in which the splats are added in cache-sheets, aligned with the volume face most parallel to the image plane, which are subsequently composited back-to-front. The former method is prone to cause bleeding artifacts from hidden objects, while the latter method reduces bleeding, but causes very visible color popping artifacts when the orientation of the compositing sheets changes suddenly as the image screen becomes more parallel to another volume face. We present a new variant of the splatting algorithm in which the compositing sheets are always parallel to the image plane, eliminating the condition for popping, while maintaining the insensitivity to color bleeding. This enables pleasing animated viewing of volumetric objects without temporal color and lighting discontinuities. The method uses a hierarchy of partial splats and employs an efficient list-based volume traversal scheme for fast splat access. It also offers more accuracy for perspective splatting as the decomposition of the individual splats facilitates a better approximation to the diverging nature of the rays that traverse the splatting kernels.

The paper describes three new results for volume rendering algorithms utilizing splatting. First, an antialiasing extension to the basic splatting algorithm is introduced that mitigates the spatial aliasing for high resolution volumes. Aliasing can be severe for high resolution volumes or volumes where a high depth of field leads to converging samples along the perspective axis. Next, an analysis of the common approximation errors in the splatting process for perspective viewing is presented. In this context, we give different implementations, distinguished by efficiency and accuracy, for adding the splat contributions to the image plane. We then present new results in controlling the splatting errors and also show their behavior in the framework of our new antialiasing technique. Finally, current work in progress on extensions to splatting for temporal antialiasing is demonstrated. We present a simple but highly effective scheme for adding motion blur to fast moving volumes.

The task of reconstructing the derivative of a discrete function is essential for its shading and rendering as well as being widely used in image processing and analysis. We survey the possible methods for normal estimation in volume rendering and divide them into two classes based on the delivered numerical accuracy. The three members of the first class determine the normal in two steps by employing both interpolation and derivative filters. Among these is a new method which has never been realized. The members of the first class are all equally accurate. The second class has only one member and employs a continuous derivative filter obtained through the analytic derivation of an interpolation filter. We use the new method to analytically compare the accuracy of the first class with that of the second. As a result of our analysis we show that even inexpensive schemes can in fact be more accurate than high order methods. We describe the theoretical computational cost of applying the schemes in a volume rendering application and provide guidelines for helping one choose a scheme for estimating derivatives. In particular we find that the new method can be very inexpensive and can compete with the normal estimations which pre-shade and pre-classify the volume (M. Levoy, 1988).

Splatting is a popular direct volume rendering algorithm. However, the algorithm does not correctly render cases where the volume sampling rate is higher than the image sampling rate (e.g. more than one voxel maps into a pixel). This situation arises with orthographic projections of high-resolution volumes, as well as with perspective projections of volumes of any resolution. The result is potentially severe spatial and temporal aliasing artifacts. Some volume ray-casting algorithms avoid these artifacts by employing reconstruction kernels which vary in width as the rays diverge. Unlike ray-casting algorithms, existing splatting algorithms do not have an equivalent mechanism for avoiding these artifacts. The authors propose such a mechanism, which delivers high-quality splatted images and has the potential for a very efficient hardware implementation.

The order in which the projections are applied in the algebraic reconstruction technique (ART) has a great effect on speed of convergence, accuracy, and the amount of noise-like artifacts in the reconstructed image. In this paper, a new projection ordering scheme for ART is presented: the weighted-distance scheme (WDS). It heuristically optimizes the angular distance of a newly selected projection with respect to an extended sequence of previously applied projections. This sequence of influential projections may incorporate the complete set of all previously applied projections or any limited time interval subset thereof. The selection algorithm results in uniform sampling of the projection access space, minimizing correlation in the projection sequence. This produces more accurate images with less noise-like artifacts than previously suggested projection ordering schemes.

We describe a new method for analyzing, classifying, and evaluating filters that can be applied to interpolation filters as well as to arbitrary derivative filters of any order. Our analysis is based on the Taylor series expansion of the convolution sum. Our analysis shows the need and derives the method for the normalization of derivative filter weights. Under certain minimal restrictions of the underlying function, we are able to compute tight absolute error bounds of the reconstruction process. We demonstrate the utilization of our methods to the analysis of the class of cubic BC-spline filters. As our technique is not restricted to interpolation filters, we are able to show that the Catmull-Rom spline filter and its derivative are the most accurate reconstruction and derivative filters, respectively, among the class of BC-spline filters. We also present a new derivative filter which features better spatial accuracy than any derivative BC-spline filter, and is optimal within our framework. We conclude by demonstrating the use of these optimal filters for accurate interpolation and gradient estimation in volume rendering.

Volume ray casting is based on sampling the data along sight rays. In this technique, reconstruction is achieved by a convolution, which collects the contribution of multiple voxels to one sample point. Splatting, on the other hand, is based on projecting data points on to the screen, and reconstruction is implemented by an "inverted convolution", where the contribution of one data element is distributed to many sample points (i.e. pixels). Splatting produces images of a quality comparable to ray casting but at greater speeds. This is achieved by pre-computing the projection footprint that the interpolation kernel leaves on the image plane. However, while fast incremental schemes can be utilized for orthographic projection, the perspective projection complicates the mapping of the footprints and is therefore rather slow. In this paper, we merge the technique of splatting with the principles of ray casting to yield a ray-driven splatting approach. We imagine splats as being suspended in object space, a splat at every voxel. Rays are then spawned to traverse the space and intersect the splats. An efficient and accurate way of intersecting and addressing the splats is described. Not only is ray-driven splatting inherently insensitive to the complexity of the perspective viewing transform, it also offers acceleration methods such as early ray termination and bounding volumes, which are methods that traditional voxel-driven splatting cannot benefit from. This results in competitive or superior performance for parallel projection, and superior performance for perspective projection.