Cardiovascular disease (CVD) is one of the leading causes of death and illness across the world. Especially, early prediction of heart failure (HF) is complicated due to the heterogeneity of its clinical presentations and symptoms. These challenges underscore the need for a multidisciplinary approach for comprehensive evaluation of cardiac state. To this end, we specifically select electrocardiogram (ECG) and 3D cardiac anatomy for their complementary coverage of cardiac electrical activities and fine-grained structural modeling. Building upon this, we present a novel pre-training framework, named Anatomy-Electrocardiogram Model (AEM), to explore their complex interactions. AEM adopts a multi-task self-supervised scheme that combines a masked reconstruction objective with a cardiac measurement (CM) regression branch to embed cardiac functional priors and structural details. Unlike image-domain models that typically localize the whole heart within the image, our 3D anatomy is background-free and continuous in 3D space. Hence, the model can naturally concentrate on finer structures at the patch level. The further integration with ECG captures functional dynamics through electrical conduction, encapsulating holistic cardiac representations. Extensive experiments are conducted on the multi-modal datasets collected from the UK Biobank, which contain paired biventricular point cloud anatomy and 12-lead ECG data. Our proposed AEM achieves an area under the receiver operating characteristic curve of 0.8192 for incident HF prediction and a concordance index of 0.6976 for survival prediction under linear evaluation, outperforming the state-of-the-art multi-modal methods. Additionally, we study the interpretability of the disease prediction by observing that our model effectively recognizes clinically plausible patterns and exhibits a high association with clinical features.

This paper presents Visualization Badges, graphical labels shown alongside visualizations to communicate provenance and design considerations to enhance understandability and transparency. Badges may, for example, highlight a major finding, disclose that an axis has been truncated, or warn of possible visual artifacts. Inspired by nutrition and energy labels on product packaging, visualization badges aim (i) to allow visualization authors to justify and disclose analysis and design decisions and (ii) to make readers aware of important information when viewing and interpreting visualizations. Collectively, visualization badges aim to foster trust in visualizations and prevent readers from drawing incorrect conclusions. Based on a series of co-design workshops, we define and evaluate the concept of visualization badges and formulate a conceptual framework for analysis, application, and further research. Our framework includes a catalog of 132 visualization badges, categorization schemes, design options for their visual representations, applied visualization examples, and guidelines for their use. We hope that visualization badges will help communicate data and collectively improve communication, visualization literacy, and the quality of visualization techniques. Our badges, workshops, and guidelines can be found online https://vis-badges.github.io.

In many applications, machine-learned (ML) models are required to hold some invariance qualities, such as rotation, size, and intensity invariance. Among these, testing for background invariance presents a significant challenge due to the vast and complex data space it encompasses. To evaluate invariance qualities, we first use a visualization-based testing framework which allows human analysts to assess and make informed decisions about the invariance properties of ML models. We show that such informative testing framework is preferred as ML models with the same global statistics (e.g., accuracy scores) can behave differently and have different visualized testing patterns. However, such human analysts might not lead to consistent decisions without a systematic sampling approach to select representative testing suites. In this work, we present a technical solution for selecting background scenes according to their semantic proximity to a target image that contains a foreground object being tested. We construct an ontology for storing knowledge about relationships among different objects using association analysis. This ontology enables an efficient and meaningful search for background scenes of different semantic distances to a target image, enabling the selection of a test suite that is both diverse and reasonable. Compared with other testing techniques, e.g., random sampling, nearest neighbors, or other sampled test suites by visual-language models (VLMs), our method achieved a superior balance between diversity and consistency of human annotations, thereby enhancing the reliability and comprehensiveness of background invariance testing.

Grounded theory (GT) is a research methodology that entails a systematic workflow for theory generation grounded on emergent data. In this article, we juxtapose GT workflows with typical workflows in visualization and visual analytics (VIS), unveiling the characteristics shared by these workflows. We explore the research landscape of VIS to study where GT is applied to generate VIS theories, explicitly as well as implicitly. We discuss "why" GT can potentially play a significant role in VIS. We outline a "how" methodology for conducting GT research in VIS, which addresses the need for theoretical advancement in VIS while benefiting from other methods and techniques in VIS. We illustrate this "how" methodology with a use case of adopting GT approaches in studying visualization guidelines.

Glyph-based visualization is one of the main techniques for visualizing complex multivariate data. With small glyphs, data variables are typically encoded with relatively low visual and perceptual precision. Glyph designers have to contemplate the trade-offs in allocating visual channels when there is a large number of data variables. While there are many successful glyph designs in the literature, there is not yet a systematic method for assisting visualization designers to evaluate different design options that feature different types of trade-offs. In this paper, we present an evaluation scheme based on the multi-criteria decision analysis (MCDA) methodology. The scheme provides designers with a structured way to consider their glyph designs from a range of perspectives, while rendering a semi-quantitative template for evaluating different design options. In addition, this work provides guideposts for future empirical research to obtain more quantitative measurements that can be used in MCDA-aided glyph design processes.

Image similarity has been extensively studied in computer vision. In recent years, machine-learned models have shown their ability to encode more semantics than traditional multivariate metrics. However, in labelling semantic similarity, assigning a numerical score to a pair of images is impractical, making the improvement and comparisons on the task difficult. In this work, we present a more intuitive approach to build and compare image similarity models based on labelled data in the form of A:R vs B:R, i.e., determining if an image A is closer to a reference image R than another image B. We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data by using an ensemble model. Our testing results show that the ensemble model constructed performs ~5% better than the best individual context-sensitive models. They also performed better than the models that were directly fine-tuned using mixed imagery data as well as existing deep embeddings, e.g., CLIP [30] and DINO [3]. This work demonstrates that context-based labelling and model training can be effective when an appropriate ensemble approach is used to alleviate the limitation due to sparse sampling.

Quality-sensitive applications of machine learning (ML) require quality assurance (QA) by humans before the predictions of an ML model can be deployed. QA for ML (QA4ML) interfaces require users to view a large amount of data and perform many interactions to correct errors made by the ML model. An optimized user interface (UI) can significantly reduce interaction costs. While UI optimization can be informed by user studies evaluating design options, this approach is not scalable, because there are typically numerous small variations that can affect the efficiency of a QA4ML interface. Hence, we propose using simulation to evaluate and aid the optimization of QA4ML interfaces. In particular, we focus on simulating the combined effects of human intelligence in initiating appropriate interaction commands and machine intelligence in providing algorithmic assistance for accelerating QA4ML processes. As QA4ML is usually labor-intensive, we use the simulated task completion time as the metric for UI optimization under different interface and algorithm setups. We demonstrate the usage of this UI design method in several QA4ML applications.

Icicles and sunbursts are two commonly-used visual representations of trees. While icicle trees can map data values faithfully to rectangles of different sizes, often some rectangles are too narrow to be noticed easily. When an icicle tree is transformed into a sunburst tree, the width of each rectangle becomes the length of an annular sector that is usually longer than the original width. While sunburst trees alleviate the problem of narrow rectangles in icicle trees, it no longer maintains the consistency of size encoding. At different tree depths, nodes of the same data values are displayed in annular sections of different sizes in a sunburst tree, though they are represented by rectangles of the same size in an icicle tree. Furthermore, two nodes from different subtrees could sometimes appear as a single node in both icicle trees and sunburst trees. In this paper, we propose a new visual representation, referred to as radial icicle tree (RIT), which transforms the rectangular bounding box of an icicle tree into a circle, circular sector, or annular sector while introducing gaps between nodes and maintaining area constancy for nodes of the same size. We applied the new visual design to several datasets. Both the analytical design process and user-centered evaluation have confirmed that this new design has improved the design of icicles and sunburst trees without introducing any relative demerit.

This paper introduces design patterns for dashboards to inform dashboard design processes. Despite a growing number of public examples, case studies, and general guidelines there is surprisingly little design guidance for dashboards. Such guidance is necessary to inspire designs and discuss tradeoffs in, e.g., screenspace, interaction, or information shown. Based on a systematic review of 144 dashboards, we report on eight groups of design patterns that provide common solutions in dashboard design. We discuss combinations of these patterns in "dashboard genres" such as narrative, analytical, or embedded dashboard. We ran a 2-week dashboard design workshop with 23 participants of varying expertise working on their own data and dashboards. We discuss the application of patterns for the dashboard design processes, as well as general design tradeoffs and common challenges. Our work complements previous surveys and aims to support dashboard designers and researchers in co-creation, structured design decisions, as well as future user evaluations about dashboard design guidelines. Detailed pattern descriptions and workshop material can be found online: https://dashboarddesignpatterns.github.io.

Computational modeling is a commonly used technology in many scientific disciplines and has played a noticeable role in combating the COVID-19 pandemic. Modeling scientists conduct sensitivity analysis frequently to observe and monitor the behavior of a model during its development and deployment. The traditional algorithmic ranking of sensitivity of different parameters usually does not provide modeling scientists with sufficient information to understand the interactions between different parameters and model outputs, while modeling scientists need to observe a large number of model runs in order to gain actionable information for parameter optimization. To address the above challenge, we developed and compared two visual analytics approaches, namely: algorithm-centric and visualization-assisted, and visualization-centric and algorithm-assisted. We evaluated the two approaches based on a structured analysis of different tasks in visual sensitivity analysis as well as the feedback of domain experts. While the work was carried out in the context of epidemiological modeling, the two approaches developed in this work are directly applicable to a variety of modeling processes featuring time series outputs, and can be extended to work with models with other types of outputs.

Music mood classification has been a challenging problem in comparison with other music classification problems (e.g., genre, composer, or period). One solution for addressing this challenge is to use an ensemble of machine learning models. In this paper, we show that visualization techniques can effectively convey the popular prediction as well as uncertainty at different music sections along the temporal axis while enabling the analysis of individual ML models in conjunction with their application to different musical data. In addition to the traditional visual designs, such as stacked line graph, ThemeRiver, and pixel-based visualization, we introduce a new variant of ThemeRiver, called "dual-flux ThemeRiver", which allows viewers to observe and measure the most popular prediction more easily than stacked line graph and ThemeRiver. Together with pixel-based visualization, dual-flux ThemeRiver plots can also assist in model-development workflows, in addition to annotating music using ensemble model predictions.

We present the design and development of a data visualization service (RAMPVIS) in response to the urgent need to support epidemiological modeling workflows during the COVID-19 pandemic. Facing a set of demanding requirements and several practical challenges, our small team of volunteers had to rely on existing knowledge and components of services computing, while thinking on our feet in configuring services composition and adopting suitable approaches to services engineering. Through developing the RAMPVIS service, we have gained useful experience of ensuring conformation to services computing standards, enabling rapid development and early deployment, and facilitating effective and efficient maintenance and operation with limited resources. This experience can be valuable to the ongoing effort for combating the COVID-19 pandemic, and provides a blueprint for visualization service development when future needs for visual analytics arise during emergency response.

In the process of developing an infrastructure for providing visualization and visual analytics (VIS) tools to epidemiologists and modeling scientists, we encountered a technical challenge for applying a number of visual designs to numerous datasets rapidly and reliably with limited development resources. In this paper, we present a technical solution to address this challenge. Operationally, we separate the tasks of data management, visual designs, and plots and dashboard deployment in order to streamline the development workflow. Technically, we utilize: an ontology to bring datasets, visual designs, and deployable plots and dashboards under the same management framework; multi-criteria search and ranking algorithms for discovering potential datasets that match a visual design; and a purposely-design user interface for propagating each visual design to appropriate datasets (often in tens and hundreds) and quality-assuring the propagation before the deployment. This technical solution has been used in the development of the RAMPVIS infrastructure for supporting a consortium of epidemiologists and modeling scientists through visualization.

Visual analytics enables the coupling of machine learning models and humans in a tightly integrated workflow, addressing various analysis tasks. Each task poses distinct demands to analysts and decision-makers. In this survey, we focus on one canonical technique for rule-based classification, namely decision tree classifiers. We provide an overview of available visualizations for decision trees with a focus on how visualizations differ with respect to 16 tasks. Further, we investigate the types of visual designs employed, and the quality measures presented. We find that (i) interactive visual analytics systems for classifier development offer a variety of visual designs, (ii) utilization tasks are sparsely covered, (iii) beyond classifier development, node-link diagrams are omnipresent, (iv) even systems designed for machine learning experts rarely feature visual representations of quality measures other than accuracy. In conclusion, we see a potential for integrating algorithmic techniques, mathematical quality measures, and tailored interactive visualizations to enable human experts to utilize their knowledge more effectively.

Aspect-based sentiment classification aims to predict the sentiment polarity of an aspect term in a sentence. It has been verified that syntactic dependent trees, especially integrating with graph convolutional networks (GCN) can provide crucial syntactic features for sentiment classification. However, it can not consider the dependency label information between the aspect and context in the sentence, which contains rich semantic information. In this paper, we propose a bidirectional edge-enhanced graph convolutional networks (BE-GCN), which combines the syntactic structure and the dependency label information effectively. Specifically, we design an edge-enhanced module to dynamically update the dependency edge according to dependency relation and contextual information. Then the dependency edge can update the word representation reversely, thereby selectively outputting sentiment features according to the given aspect. Comparison experiments demonstrate the effectiveness of our model using dependency label information and syntactic structure.

Abstract Colormapping is one of the simplest and most widely used data visualization methods within and outside the visualization community. Uniformity, order, discriminative power, and smoothness of continuous colormaps are the most important criteria for evaluating and potentially improving colormaps. We present a local and a global automatic optimization algorithm in Euclidean color spaces for each of these design rules in this work. As a foundation for our optimization algorithms, we used the CCC‐Tool colormap specification (CMS); each algorithm has been implemented in this tool. In addition to synthetic examples that demonstrate each method's effect, we show the outcome of some of the methods applied to a typhoon simulation.

Abstract Visualization is an essential tool for observing and analyzing origin‐destination (OD) data, which encodes flows between geographic locations, e.g., in applications concerning commuting, migration, and transport of goods. However, depicting OD data often encounter issues of cluttering and occlusion. To address these issues, many visual designs feature data abstraction and visual abstraction, such as node aggregation and edge bundling, resulting in information loss. The recent theoretical and empirical developments in visualization have substantiated the merits of such abstraction, while confirming that viewers' knowledge can alleviate the negative impact due to information loss. It is thus desirable to map out different ways of losing and adding information in origin‐destination data visualization (ODDV). We therefore formulate a new design space of ODDV based on the categorization of informative operations on OD data in data abstraction and visual abstraction. We apply this design space to existing ODDV methods, outline strategies for exploring the design space, and suggest ideas for further exploration.

In many applications, while machine learning (ML) can be used to derive algorithmic models to aid decision processes, it is often difficult to learn a precise model when the number of similar data points is limited. One example of such applications is data reconstruction from historical visualizations, many of which encode precious data, but their numerical records are lost. On the one hand, there is not enough similar data for training an ML model. On the other hand, manual reconstruction of the data is both tedious and arduous. Hence, a desirable approach is to train an ML model dynamically using interactive classification, and hopefully, after some training, the model can complete the data reconstruction tasks with less human interference. For this approach to be effective, the number of annotated data objects used for training the ML model should be as small as possible, while the number of data objects to be reconstructed automatically should be as large as possible. In this article, we present a novel technique for the machine to initiate intelligent interactions to reduce the user’s interaction cost in interactive classification tasks. The technique of machine-initiated intelligent interaction (MI3) builds on a generic framework featuring active sampling and default labeling. To demonstrate the MI3 approach, we use the well-known cholera map visualization by John Snow as an example, as it features three instances of MI3 pipelines. The experiment has confirmed the merits of the MI3 approach.

Many computer science disciplines (e.g., combinatorial optimization, natural language processing, and information retrieval) use standard or established test suites for evaluating algorithms. In visualization, similar approaches have been adopted in some areas (e.g., volume visualization), while user testimonies and empirical studies have been the dominant means of evaluation in most other areas, such as designing colormaps. In this paper, we propose to establish a test suite for evaluating the design of colormaps. With such a suite, the users can observe the effects when different continuous colormaps are applied to planar scalar fields that may exhibit various characteristic features, such as jumps, local extrema, ridge or valley lines, different distributions of scalar values, different gradients, different signal frequencies, different levels of noise, and so on. The suite also includes an expansible collection of real-world data sets including the most popular data for colormap testing in the visualization literature. The test suite has been integrated into a web-based application for creating continuous colormaps (https://ccctool.com/), facilitating close inter-operation between design and evaluation processes. This new facility complements traditional evaluation methods such as user testimonies and empirical studies.

Continuous colormaps are integral parts of many visualization techniques, such as heat-maps, surface plots, and flow visualization. Despite that the critiques of rainbow colormaps have been around and well-acknowledged for three decades, rainbow colormaps are still widely used today. One reason behind the resilience of rainbow colormaps is the lack of tools for users to create a continuous colormap that encodes semantics specific to the application concerned. In this paper, we present a web-based software system, CCC-Tool (short for Charting Continuous Colormaps) under the URL https://ccctool.com, for creating, editing, and analyzing such application-specific colormaps. We introduce the notion of "colormap specification (CMS)" that maintains the essential semantics required for defining a color mapping scheme. We provide users with a set of advanced utilities for constructing CMS's with various levels of complexity, examining their quality attributes using different plots, and exporting them to external application software. We present two case studies, demonstrating that the CCC-Tool can help domain scientists as well as visualization experts in designing semantically-rich colormaps.

Visualization has been deemed a useful technique by researchers and practitioners, alike, leaving a trail of arguments behind that reason why visualization works. In addition, examples of misleading usages of visualizations in information communication have occasionally been pointed out. Thus, to contribute to the fundamental understanding of our discipline, we require a comprehensive collection of arguments on "why visualize?" (or "why not?"), untangling the rationale behind positive and negative viewpoints. In this paper, we report a theoretical study to understand the underlying reasons of various arguments; their relationships (e.g., built-on, and conflict); and their respective dependencies on tasks, users, and data. We curated an argumentative network based on a collection of arguments from various fields, including information visualization, cognitive science, psychology, statistics, philosophy, and others. Our work proposes several categorizations for the arguments, and makes their relations explicit. We contribute the first comprehensive and systematic theoretical study of the arguments on visualization. Thereby, we provide a roadmap towards building a foundation for visualization theory and empirical research as well as for practical application in the critique and design of visualizations. In addition, we provide our argumentation network and argument collection online at https://whyvis.dbvis.de, supported by an interactive visualization.

In this paper, we present a visual analytics tool for enabling hypothesis-based evaluation of machine learning (ML) models. We describe a novel ML-testing framework that combines the traditional statistical hypothesis testing (commonly used in empirical research) with logical reasoning about the conclusions of multiple hypotheses. The framework defines a controlled configuration for testing a number of hypotheses as to whether and how some extra information about a "concept" or "feature" may benefit or hinder an ML model. Because reasoning multiple hypotheses is not always straightforward, we provide HypoML as a visual analysis tool, with which, the multi-thread testing results are first transformed to analytical results using statistical and logical inferences, and then to a visual representation for rapid observation of the conclusions and the logical flow between the testing results and hypotheses. We have applied HypoML to a number of hypothesized concepts, demonstrating the intuitive and explainable nature of the visual analysis.

Abstract Preservation of data privacy and protection of sensitive information from potential adversaries constitute a key socio‐technical challenge in the modern era of ubiquitous digital transformation. Addressing this challenge needs analysis of multiple factors: algorithmic choices for balancing privacy and loss of utility, potential attack scenarios that can be undertaken by adversaries, implications for data owners, data subjects, and data sharing policies, and access control mechanisms that need to be built into interactive data interfaces. Visualization has a key role to play as part of the solution space, both as a medium of privacy‐aware information communication and also as a tool for understanding the link between privacy parameters and data sharing policies. The field of privacy‐preserving data visualization has witnessed progress along many of these dimensions. In this state‐of‐the‐art report, our goal is to provide a systematic analysis of the approaches, methods, and techniques used for handling data privacy in visualization. We also reflect on the road‐map ahead by analyzing the gaps and research opportunities for solving some of the pressing socio‐technical challenges involving data privacy with the help of visualization.

Minimization of disclosure risks is a key challenge in publicly available visualizations that can potentially reveal personal information. Such risks are inherently dependent on the amount of information that adversaries can gain by manipulating visual representations and by using their background knowledge. Conventional risk quantification models proposed in the field of privacy-preserving data mining suffer from a lack of transparency in letting data owners control privacy parameters and understand their implications for disclosure risks. To fill this gap, we propose a visual uncertainty model for letting data owners understand the relationships between privacy parameters and vulnerable visualization configurations. Our main contribution is a probabilistic analysis of the disclosure risks associated with vulnerabilities in privacy-preserving parallel coordinates and scatter plots. We quantify the relationship among attack scenarios, adversarial knowledge, and the inherent uncertainty in cluster-based visualizations that can act as defense mechanisms. We present examples and a case study to demonstrate the effectiveness of the model.

Abstract Designing, evaluating, and improving visual analytics (VA) systems is a primary area of activities in our discipline. In this paper, we present an ontological framework for recording and categorizing technical shortcomings to be addressed in a VA workflow, reasoning about the causes of such problems, identifying technical solutions, and anticipating secondary effects of the solutions. The methodology is built on the theoretical premise that designing a VA workflow is an optimization of the cost‐benefit ratio of the processes in the workflow. It makes uses three fundamental measures to group and connect “symptoms”, “causes”, “remedies”, and “side‐effects”, and guide the search for potential solutions to the problems. In terms of requirement analysis and system design, the proposed methodology can enable system designers to explore the decision space in a structured manner. In terms of evaluation, the proposed methodology is time‐efficient and complementary to various forms of empirical studies, such as user surveys, controlled experiments, observational studies, focus group discussions, and so on. In general, it reduces the amount of trial‐and‐error in the lifecycle of VA system development.

Visualization and virtual environments (VEs) have been two interconnected parallel strands in visual computing for decades. Some VEs have been purposely developed for visualization applications, while many visualization applications are exemplary showcases in general-purpose VEs. Because of the development and operation costs of VEs, the majority of visualization applications in practice have yet to benefit from the capacity of VEs. In this paper, we examine this status quo from an information-theoretic perspective. Our objectives are to conduct cost-benefit analysis on typical VE systems (including augmented and mixed reality, theater-based systems, and large powerwalls), to explain why some visualization applications benefit more from VEs than others, and to sketch out pathways for the future development of visualization applications in VEs. We support our theoretical propositions and analysis using theories and discoveries in the literature of cognitive sciences and the practical evidence reported in the literatures of visualization and VEs.

While many VA workflows make use of machine-learned models to support analytical tasks, VA workflows have become increasingly important in understanding and improving Machine Learning (ML) processes. In this paper, we propose an ontology (VIS4ML) for a subarea of VA, namely "VA-assisted ML". The purpose of VIS4ML is to describe and understand existing VA workflows used in ML as well as to detect gaps in ML processes and the potential of introducing advanced VA techniques to such processes. Ontologies have been widely used to map out the scope of a topic in biology, medicine, and many other disciplines. We adopt the scholarly methodologies for constructing VIS4ML, including the specification, conceptualization, formalization, implementation, and validation of ontologies. In particular, we reinterpret the traditional VA pipeline to encompass model-development workflows. We introduce necessary definitions, rules, syntaxes, and visual notations for formulating VIS4ML and make use of semantic web technologies for implementing it in the Web Ontology Language (OWL). VIS4ML captures the high-level knowledge about previous workflows where VA is used to assist in ML. It is consistent with the established VA concepts and will continue to evolve along with the future developments in VA and ML. While this ontology is an effort for building the theoretical foundation of VA, it can be used by practitioners in real-world applications to optimize model-development workflows by systematically examining the potential benefits that can be brought about by either machine or human capabilities. Meanwhile, VIS4ML is intended to be extensible and will continue to be updated to reflect future advancements in using VA for building high-quality data-analytical models or for building such models rapidly.

course Public Access Share on Tutorial on information theory in visualization Authors: Mateu Sbert University of Girona (Spain) and Tianjin University (China) University of Girona (Spain) and Tianjin University (China)View Profile , Han-Wei Shen View Profile , Ivan Viola View Profile , Min Chen View Profile , Anton Bardera View Profile , Miquel Feixas View Profile Authors Info & Claims SA '17: SIGGRAPH Asia 2017 CoursesNovember 2017 Article No.: 17Pages 1–165https://doi.org/10.1145/3134472.3134507Published:27 November 2017Publication History 1citation495DownloadsMetricsTotal Citations1Total Downloads495Last 12 Months76Last 6 weeks3 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Abstract Detecting similarity between texts is a frequently encountered text mining task. Because the measurement of similarity is typically composed of a number of metrics, and some measures are sensitive to subjective interpretation, a generic detector obtained using machine learning often has difficulties balancing the roles of different metrics according to the semantic context exhibited in a specific collection of texts. In order to facilitate human interaction in a visual analytics process for text similarity detection, we first map the problem of pairwise sequence comparison to that of image processing, allowing patterns of similarity to be visualized as a 2D pixelmap. We then devise a visual interface to enable users to construct and experiment with different detectors using primitive metrics, in a way similar to constructing an image processing pipeline. We deployed this new approach for the identification of commonplaces in 18th‐century literary and print culture. Domain experts were then able to make use of the prototype system to derive new scholarly discoveries and generate new hypotheses.

Abstract In this paper, we present an empirical study designed to evaluate the hypothesis that humans’ soft knowledge can enhance the cost‐benefit ratio of a visualization process by reducing the potential distortion. In particular, we focused on the impact of three classes of soft knowledge: (i) knowledge about application contexts, (ii) knowledge about the patterns to be observed (i.e., in relation to visualization task), and (iii) knowledge about statistical measures. We mapped these classes into three control variables, and used real‐world time series data to construct stimuli. The results of the study confirmed the positive contribution of each class of knowledge towards the reduction of the potential distortion, while the knowledge about the patterns prevents distortion more effectively than the other two classes.

Abstract Scatterplots have been in use for about two centuries, primarily for observing the relationship between two variables and commonly for supporting correlation analysis. In this paper, we report an empirical study that examines how humans’ perception of correlation using scatterplots relates to the Pearson's product‐moment correlation coefficient (PPMCC) – a commonly used statistical measure of correlation. In particular, we study human participants’ estimation of correlation under different conditions, e.g., different PPMCC values, different densities of data points, different levels of symmetry of data enclosures, and different patterns of data distribution. As the participants were instructed to estimate the PPMCC of each stimulus scatterplot, the difference between the estimated and actual PPMCC is referred to as an offset. The results of the study show that varying PPMCC values, symmetry of data enclosure, or data distribution does have an impact on the average offsets, while only large variations in density cause an impact that is statistically significant. This study indicates that humans’ perception of correlation using scatterplots does not correlate with computed PPMCC in a consistent manner. The magnitude of offsets may be affected not only by the difference between individuals, but also by geometric features of data enclosures. It suggests that visualizing scatterplots does not provide adequate support to the task of retrieving their corresponding PPMCC indicators, while the underlying model of humans’ perception of correlation using scatterplots ought to feature other variables in addition to PPMCC. The paper also includes a theoretical discussion on the cost‐benefit of using scatterplots.

Mapping a set of categorical values to different colors is an elementary technique in data visualization. Users of visualization software routinely rely on the default colormaps provided by a system, or colormaps suggested by software such as ColorBrewer. In practice, users often have to select a set of colors in a semantically meaningful way (e.g., based on conventions, color metaphors, and logological associations), and consequently would like to ensure their perceptual differentiation is optimized. In this paper, we present an algorithmic approach for maximizing the perceptual distances among a set of given colors. We address two technical problems in optimization, i.e., (i) the phenomena of local maxima that halt the optimization too soon, and (ii) the arbitrary reassignment of colors that leads to the loss of the original semantic association. We paid particular attention to different types of constraints that users may wish to impose during the optimization process. To demonstrate the effectiveness of this work, we tested this technique in two case studies. To reach out to a wider range of users, we also developed a web application called Colourmap Hospital.

In this work, we present a study that traces the technical and cognitive processes in two visual analytics applications to a common theoretic model of soft knowledge that may be added into a visual analytics process for constructing a decision-tree model. Both case studies involved the development of classification models based on the "bag of features" approach. Both compared a visual analytics approach using parallel coordinates with a machine-learning approach using information theory. Both found that the visual analytics approach had some advantages over the machine learning approach, especially when sparse datasets were used as the ground truth. We examine various possible factors that may have contributed to such advantages, and collect empirical evidence for supporting the observation and reasoning of these factors. We propose an information-theoretic model as a common theoretic basis to explain the phenomena exhibited in these two case studies. Together we provide interconnected empirical and theoretical evidence to support the usefulness of visual analytics.

This research measures human performance in inferring the functional types (i.e., home, work, leisure and transport) of locations in geo-location data using different visual representations of the data (textual, static and animated visualizations) along with different amounts of data (1, 3 or 5 day(s)). We first collected real life geo-location data from tweets. We then asked the data owners to tag their location points, resulting in ground truth data. Using this dataset we conducted an empirical study involving 45 participants to analyze how accurately they could infer the functional location of the original data owners under different conditions, i.e., three data representations, three data densities and four location types. The study results indicate that while visual techniques perform better than textual ones, the functional locations of human activities can be inferred with a relatively high accuracy even using only textual representations and a low density of location points. Workplace was more easily inferred than home while transport was the functional location with the highest accuracy. Our results also showed that it was easier to infer functional locations from data exhibiting more stable and consistent mobility patterns, which are thus more vulnerable to privacy disclosures. We discuss the implications of our findings in the context of privacy preservation and provide guidelines to users and companies to help preserve and safeguard people's privacy.

Abstract The design of effective glyphs for visualisation involves a number of different visual encodings. Since spatial position is usually already specified in advance, we must rely on other visual channels to convey additional relationships for multivariate analysis. One such relationship is the apparent order present in the data. This paper presents two crowdsourcing empirical studies that focus on the perceptual evaluation of orderability for visual channels, namely Bertin's retinal variables. The first study investigates the perception of order in a sequence of elements encoded with different visual channels. We found evidence that certain visual channels are perceived as more ordered (for example, value) while others are perceived as less ordered (for example, hue) than the measured order present in the data. As a result, certain visual channels are more/less sensitive to disorder. The second study evaluates how visual orderability affects min and max judgements of elements in the sequence. We found that visual channels that tend to be perceived as ordered, improve the accuracy of identifying these values.

In this paper, we present an abstract model of visualization and inference processes, and describe an information-theoretic measure for optimizing such processes. In order to obtain such an abstraction, we first examined six classes of workflows in data analysis and visualization, and identified four levels of typical visualization components, namely disseminative, observational, analytical and model-developmental visualization. We noticed a common phenomenon at different levels of visualization, that is, the transformation of data spaces (referred to as alphabets) usually corresponds to the reduction of maximal entropy along a workflow. Based on this observation, we establish an information-theoretic measure of cost-benefit ratio that may be used as a cost function for optimizing a data visualization process. To demonstrate the validity of this measure, we examined a number of successful visualization processes in the literature, and showed that the information-theoretic measure can mathematically explain the advantages of such processes over possible alternatives.

Threats in cybersecurity come in a variety of forms, and combating such threats involves handling a huge amount of data from different sources. It is absolutely necessary to use algorithmic models to defend against these threats. However, all models are sensitive to deviation from the original contexts in which the models were developed. Hence, it is not really an overstatement to say that `all models are wrong'. In this paper, we propose a visual analytics loop for supporting the continuous development of models during their deployment. We describe the roles of three types of operators (monitors, analysts and modelers), present the visualization techniques used at different stages of model development, and demonstrate the utility of this approach in conjunction with a prototype software system for corporate insider threat detection. In many ways, our environment facilitates an agile approach to the development and deployment of models in cybersecurity.

Abstract Previous empirical studies for comparing parallel coordinates plots and scatter plots showed some uncertainty about their relative merits. Some of these studies focused on the task of value retrieval, where visualization usually has a limited advantage over reading data directly. In this paper, we report an empirical study that compares user performance, in terms of accuracy and response time, in the context of four different visualization tasks, namely value retrieval, clustering, outlier detection, and change detection. In order to evaluate the relative merits of the two types of plots with a common base line (i.e., reading data directly), we included three forms of stimuli, data tables, scatter plots, and parallel coordinate plots. Our results show that data tables are better suited for the value retrieval task, while parallel coordinates plots generally outperform the two other visual representations in three other tasks. Subjective feedbacks from the users are also consistent with the quantitative analyses. As visualization is commonly used for aiding multiple observational and analytical tasks, our results provided new evidence to support the prevailing enthusiasm for parallel coordinates plots in the field of visualization.

The existing efforts in computer assisted semen analysis have been focused on high speed imaging and automated image analysis of sperm motility. This results in a large amount of data, and it is extremely challenging for both clinical scientists and researchers to interpret, compare and correlate the multidimensional and time-varying measurements captured from video data. In this work, we use glyphs to encode a collection of numerical measurements taken at a regular interval and to summarize spatio-temporal motion characteristics using static visual representations. The design of the glyphs addresses the needs for (a) encoding some 20 variables using separable visual channels, (b) supporting scientific observation of the interrelationships between different measurements and comparison between different sperm cells and their flagella, and (c) facilitating the learning of the encoding scheme by making use of appropriate visual abstractions and metaphors. As a case study, we focus this work on video visualization for computer-aided semen analysis, which has a broad impact on both biological sciences and medical healthcare. We demonstrate that glyph-based visualization can serve as a means of external memorization of video data as well as an overview of a large set of spatiotemporal measurements. It enables domain scientists to make scientific observation in a cost-effective manner by reducing the burden of viewing videos repeatedly, while providing them with a new visual representation for conveying semen statistics.

This paper describes the process of developing a visualisation system with the use of Game Engine. A bottom up approach was used in developing the system, which allows process alignment with the Game Engine structure. The approach allows large abstract data processing, which are channelled through the components of visual interfaces and supported with relational data input structure. The developed system provides automation in much of the pipeline in producing dynamic visualisation selections. As case studies, the UK Research Assessment Exercise (RAE2008) and 2010 World Oil Reserves datasets were used to demonstrate the usability of the Game Engine based visualisation system.

This paper describes a visual analytics method for visualizing the effects of multiple anomaly detection models, exploring the complex model space of a specific type of detection method, namely Query with Conditional Attributes (QCAT), and facilitating the construction of composite models using multiple QCATs. We have developed a prototype system that features a browser-based interface, and database-driven back end. We tested the system using the "Inside Threats Dataset" provided by CMU.

Abstract The majority of display devices used in visualization are 2D displays. Inevitably, it is often necessary to overlay one piece of visual information on top of another, especially in applications such as multi‐field visualization and geo‐spatial information visualization. In this paper, we present a conceptual framework for studying the mechanisms for overlaying multiple pieces of visual information while allowing users to recover occluded information. We adopt the term ‘multiplexing’ from tele‐ and data communication to encompass all such overlapping mechanisms. We establish 10 categories of visual multiplexing mechanisms. We draw support evidence from both perception literature and existing works in visualization to support this conceptual framework. We examine the relationships between multiplexing and information theoretic measures. This new conceptual categorization provides the much‐needed theory of visualization with an integral component.

Abstract In this paper, we present a user‐centered design study on poetry visualization. We develop a rule‐based solution to address the conflicting needs for maintaining the flexibility of visualizing a large set of poetic variables and for reducing the tedium and cognitive load in interacting with the visual mapping control panel. We adopt Munzner's nested design model to maintain high‐level interactions with the end users in a closed loop. In addition, we examine three design options for alleviating the difficulty in visualizing poems latitudinally. We present several example uses of poetry visualization in scholarly research on poetry.

Abstract In previous work, we proposed a technique for preserving the privacy of quasi‐identifiers in sensitive data when visualized using parallel coordinates. This paper builds on that work by introducing a number of metrics that can be used to assess both the level of privacy and the amount of utility that can be gained from the resulting visualizations. We also generalize our approach beyond parallel coordinates to scatter plots and other visualization techniques. Privacy preservation generally entails a trade‐off between privacy and utility: the more the data are protected, the less useful the visualization. Using a visually‐oriented approach, we can provide a higher amount of utility than directly applying data anonymization techniques used in data mining. To demonstrate this, we use the visual uncertainty framework for systematically defining metrics based on cluster artifacts and information theoretic principles. In a case study, we demonstrate the effectiveness of our technique as compared to standard data‐based clustering in the context of privacy‐preserving visualization.

Abstract In this paper, we present a novel visualization technique for assisting the observation and analysis of algorithmic complexity. In comparison with conventional line graphs, this new technique is not sensitive to the units of measurement, allowing multivariate data series of different physical qualities (e.g., time, space and energy) to be juxtaposed together conveniently and consistently. It supports multivariate visualization as well as uncertainty visualization. It enables users to focus on algorithm categorization by complexity classes, while reducing visual impact caused by constants and algorithmic components that are insignificant to complexity analysis. It provides an effective means for observing the algorithmic complexity of programs with a mixture of algorithms and black‐box software through visualization. Through two case studies, we demonstrate the effectiveness of complexity plots in complexity analysis in research, education and application.

Natural image statistics is an important area of research in cognitive sciences and computer vision. Visualization of statistical results can help identify clusters and anomalies as well as analyze deviation, distribution, and correlation. Furthermore, they can provide visual abstractions and symbolism for categorized data. In this paper, we begin our study of visualization of image statistics by considering visual representations of power spectra, which are commonly used to visualize different categories of images. We show that they convey a limited amount of statistical information about image categories and their support for analytical tasks is ineffective. We then introduce several new visual representations, which convey different or more information about image statistics. We apply ANOVA to the image statistics to help select statistically more meaningful measurements in our design process. A task-based user evaluation was carried out to compare the new visual representations with the conventional power spectra plots. Based on the results of the evaluation, we made further improvement of visualizations by introducing composite visual representations of image statistics.

Traditional sketch-based image or video search systems rely on machine learning concepts as their core technology. However, in many applications, machine learning alone is impractical since videos may not be semantically annotated sufficiently, there may be a lack of suitable training data, and the search requirements of the user may frequently change for different tasks. In this work, we develop a visual analytics systems that overcomes the shortcomings of the traditional approach. We make use of a sketch-based interface to enable users to specify search requirement in a flexible manner without depending on semantic annotation. We employ active machine learning to train different analytical models for different types of search requirements. We use visualization to facilitate knowledge discovery at the different stages of visual analytics. This includes visualizing the parameter space of the trained model, visualizing the search space to support interactive browsing, visualizing candidature search results to support rapid interaction for active learning while minimizing watching videos, and visualizing aggregated information of the search results. We demonstrate the system for searching spatiotemporal attributes from sports video to identify key instances of the team and player performance.

This paper is concerned with the creation of 'macros' in workflow visualization as a support tool to increase the efficiency of data curation tasks. We propose computation of candidate macros based on their usage in large collections of workflows in data repositories. We describe an efficient algorithm for extracting macro motifs from workflow graphs. We discovered that the state transition information, used to identify macro candidates, characterizes the structural pattern of the macro and can be harnessed as part of the visual design of the corresponding macro glyph. This facilitates partial automation and consistency in glyph design applicable to a large set of macro glyphs. We tested this approach against a repository of biological data holding some 9,670 workflows and found that the algorithmically generated candidate macros are in keeping with domain expert expectations.

The Expectation Maximization (EM) algorithm is an alternative reconstruction method to the Filtered Back Projection method, providing many advantages including decreased sensitivity to noise. However the algorithm requires a large number of iterations to reach adequate convergence. Due to this, research has been carried out into accelerating the convergence rate of the EM algorithm. In this paper we present an analysis of an EM implementation which uses both OSEM and MGEM, comparing results on a per time basis with both acceleration techniques alone as well as a combination of the two methods. We provide an alternative stopping criterion based on the RMS error of the projections of the current reconstruction and compare the result with an existing variance based approach.

Abstract In recent years, a collection of new techniques which deal with video as input data, emerged in computer graphics and visualization. In this survey, we report the state of the art in video‐based graphics and video visualization. We provide a review of techniques for making photo‐realistic or artistic computer‐generated imagery from videos, as well as methods for creating summary and/or abstract visual representations to reveal important features and events in videos. We provide a new taxonomy to categorize the concepts and techniques in this newly emerged body of knowledge. To support this review, we also give a concise overview of the major advances in automated video analysis, as some techniques in this field (e.g. feature extraction, detection, tracking and so on) have been featured in video‐based modelling and rendering pipelines for graphics and visualization.

Abstract Uncertainty is an intrinsic part of any visual representation in visualization, no matter how precise the input data. Existing research on uncertainty in visualization mainly focuses on depicting data‐space uncertainty in a visual form. Uncertainty is thus often seen as a problem to deal with, in the data, and something to be avoided if possible. In this paper, we highlight the need for analyzing visual uncertainty in order to design more effective visual representations. We study various forms of uncertainty in the visual representation of parallel coordinates and propose a taxonomy for categorizing them. By building a taxonomy, we aim to identify different sources of uncertainty in the screen space and relate them to different effects of uncertainty upon the user. We examine the literature on parallel coordinates and apply our taxonomy to categorize various techniques for reducing uncertainty. In addition, we consider uncertainty from a different perspective by identifying cases where increasing certain forms of uncertainty may even be useful, with respect to task, data type and analysis scenario. This work suggests that uncertainty is a feature that can be both useful and problematic in visualization, and it is beneficial to augment an information visualization pipeline with a facility for visual uncertainty analysis.

Abstract Today real‐time sports performance analysis is a crucial aspect of matches in many major sports. For example, in soccer and rugby, team analysts may annotate videos during the matches by tagging specific actions and events, which typically result in some summary statistics and a large spreadsheet of recorded actions and events. To a coach, the summary statistics (e.g., the percentage of ball possession) lacks sufficient details, while reading the spreadsheet is time‐consuming and making decisions based on the spreadsheet in real‐time is thereby impossible. In this paper, we present a visualization solution to the current problem in real‐time sports performance analysis. We adopt a glyph‐based visual design to enable coaching staff and analysts to visualize actions and events “at a glance”. We discuss the relative merits of metaphoric glyphs in comparison with other types of glyph designs in this particular application. We describe an algorithm for managing the glyph layout at different spatial scales in interactive visualization. We demonstrate the use of this technical approach through its application in rugby, for which we delivered the visualization software,MatchPad, on a tablet computer. The MatchPad was used by the Welsh Rugby Union during the Rugby World Cup 2011. It successfully helped coaching staff and team analysts to examine actions and events in detail whilst maintaining a clear overview of the match, and assisted in their decision making during the matches. It also allows coaches to convey crucial information back to the players in a visually‐engaging manner to help improve their performance.

In written and spoken communications, figures of speech (e.g., metaphors and synecdoche) are often used as an aid to help convey abstract or less tangible concepts. However, the benefits of using rhetorical illustrations or embellishments in visualization have so far been inconclusive. In this work, we report an empirical study to evaluate hypotheses that visual embellishments may aid memorization, visual search and concept comprehension. One major departure from related experiments in the literature is that we make use of a dual-task methodology in our experiment. This design offers an abstraction of typical situations where viewers do not have their full attention focused on visualization (e.g., in meetings and lectures). The secondary task introduces "divided attention", and makes the effects of visual embellishments more observable. In addition, it also serves as additional masking in memory-based trials. The results of this study show that visual embellishments can help participants better remember the information depicted in visualization. On the other hand, visual embellishments can have a negative impact on the speed of visual search. The results show a complex pattern as to the benefits of visual embellishments in helping participants grasp key concepts from visualization.

Glyph-based visualization can offer elegant and concise presentation of multivariate information while enhancing speed and ease in visual search experienced by users. As with icon designs, glyphs are usually created based on the designers' experience and intuition, often in a spontaneous manner. Such a process does not scale well with the requirements of applications where a large number of concepts are to be encoded using glyphs. To alleviate such limitations, we propose a new systematic process for glyph design by exploring the parallel between the hierarchy of concept categorization and the ordering of discriminative capacity of visual channels. We examine the feasibility of this approach in an application where there is a pressing need for an efficient and effective means to visualize workflows of biological experiments. By processing thousands of workflow records in a public archive of biological experiments, we demonstrate that a cost-effective glyph design can be obtained by following a process of formulating a taxonomy with the aid of computation, identifying visual channels hierarchically, and defining application-specific abstraction and metaphors.

In this paper we investigate the challenge of 3D reconstruction from Snooker video data. We propose a system pipeline for intelligent filtering based on semantic importance in Snooker. The system can be divided into table detection and correction, followed by ball detection, classification and tracking. It is apparent from previous work that there are several challenges presented here. Firstly, previous methods tend to use a fixed top-down camera mounted above the table. To capture a full table view from this is challenging due to space limitations above the table. Instead, we capture video data from a tripod and correct the viewpoint through processing. Secondly, previous methods tend to simply detect the balls without considering other interfering objects such as player and cue. This becomes even more apparent when the player strikes the cue ball. Our intelligent filtering avoids such issues to give accurate 3D table reconstruction.

Abstract This work is concerned with a design study by an interdisciplinary team on visualizing a 10‐year record of seasonal and inter‐annual changes in frontal position (advance/retreat) of nearly 200 marine terminating glaciers in Greenland. Whilst the spatiotemporal nature of the raw data presents a challenge to develop a compact and intuitive visual design, the focus on coastal boundaries provides an opportunity for dimensional reduction. In this paper, we report the user‐centered design process carried out by the team, and present several visual encoding schemes that have met the requirements including compactness, intuitiveness, and ability to depict temporal changes and spatial relations. In particular, we designed a family of radial visualization, where radial lines correspond to different coastal locations, and nested rings represent the evolution of the temporal dimension from inner to outer circles. We developed an algorithm for mapping glacier terminus positions from Cartesian coordinates to angular coordinates. Instead of a naive uniform mapping, the algorithm maintains consistent spatial perception of the visually‐sensitive geographical references between their Cartesian and angular coordinates, and distributes other termini positions between primary locations based on coastal distance. This work has provided a useful solution to address the problem of inaccuracy in change evaluation based on pixel‐based visualization [ BPC*10 ].

Abstract We present a novel approach for the evaluation of 2D flow visualizations based on the visual reconstructability of the input vector fields. According to this metric, a visualization has high quality if the underlying data can be reliably reconstructed from the image. This approach provides visualization creators with a cost‐effective means to assess the quality of visualization results objectively. We present a vision‐based reconstruction system for the three most commonly‐used visual representations of vector fields, namely streamlines, arrow glyphs, and line integral convolution. To demonstrate the use of visual reconstructability as a quality metric, we consider a selection of vector fields obtained from numerical simulations, containing typical flow features. We apply the three types of visualization to each dataset, and compare the visualization results based on their visual reconstructability of the original vector field.

Abstract Over the past decade, computer scientists and psychologists have made great efforts to collect and analyze facial dynamics data that exhibit different expressions and emotions. Such data is commonly captured as videos and are transformed into feature‐based time‐series prior to any analysis. However, the analytical tasks, such as expression classification, have been hindered by the lack of understanding of the complex data space and the associated algorithm space. Conventional graph‐based time‐series visualization is also found inadequate to support such tasks. In this work, we adopt a visual analytics approach by visualizing the correlation between the algorithm space and our goal – classifying facial dynamics. We transform multiple feature‐based time‐series for each expression in measurement space to a multi‐dimensional representation in parameter space. This enables us to utilize parallel coordinates visualization to gain an understanding of the algorithm space, providing a fast and cost‐effective means to support the design of analytical algorithms.

Caricatures are a form of humorous visual art, usually created by skilled artists for the intention of amusement and entertainment. In this paper, we present a novel approach for automatic generation of digital caricatures from facial photographs, which capture artistic deformation styles from hand-drawn caricatures. We introduced a pseudo stress-strain model to encode the parameters of an artistic deformation style using "virtual" physical and material properties. We have also developed a software system for performing the caricaturistic deformation in 3D which eliminates the undesirable artifacts in 2D caricaturization. We employed a Multilevel Free-Form Deformation (MFFD) technique to optimize a 3D head model reconstructed from an input facial photograph, and for controlling the caricaturistic deformation. Our results demonstrated the effectiveness and usability of the proposed approach, which allows ordinary users to apply the captured and stored deformation styles to a variety of facial photographs.

Video storyboard, which is a form of video visualization, summarizes the major events in a video using illustrative visualization. There are three main technical challenges in creating a video storyboard, (a) event classification, (b) event selection and (c) event illustration. Among these challenges, (a) is highly application-dependent and requires a significant amount of application specific semantics to be encoded in a system or manually specified by users. This paper focuses on challenges (b) and (c). In particular, we present a framework for hierarchical event representation, and an importance-based selection algorithm for supporting the creation of a video storyboard from a video. We consider the storyboard to be an event summarization for the whole video, whilst each individual illustration on the board is also an event summarization but for a smaller time window. We utilized a 3D visualization template for depicting and annotating events in illustrations. To demonstrate the concepts and algorithms developed, we use Snooker video visualization as a case study, because it has a concrete and agreeable set of semantic definitions for events and can make use of existing techniques of event detection and 3D reconstruction in a reliable manner. Nevertheless, most of our concepts and algorithms developed for challenges (b) and (c) can be applied to other application areas.

Computerised facial aging estimation, which has the potential for many applications in human-computer interactions, has been investigated by many computer vision researchers in recent years. In this paper, a feature-based discriminant subspace is proposed to extract more discriminating and robust representations for aging estimation. After aligning all the faces by a piece-wise affine transform, orthogonal locality preserving projection (OLPP) is employed to project local binary patterns (LBP) from the faces into an age-discriminant subspace. The feature extracted from this manifold is more distinctive for age estimation compared with the features using in the state-of-the-art methods. Based on the public database FG-NET, the performance of the proposed feature is evaluated by using two different regression techniques, quadratic function and neural-network regression. The proposed feature subspace achieves the best performance based on both types of regression.

Abstract We present a feasibility study on using video visualization to aid snooker skill training. By involving the coaches and players in the loop of intelligent reasoning, our approach addresses the difficulties of automated semantic reasoning, while benefiting from mature video processing techniques. This work was conducted in conjunction with a snooker club and a sports scientist. In particular, we utilized the principal design of the VideoPerpetuoGram (VPG) to convey spatiotemporal information to the viewers through static visualization, removing the burden of repeated video viewing. We extended the VPG design to accommodate the need for depicting multiple video streams and respective temporal attribute fields, including silhouette extrusion, spatial attributes, and non‐spatial attributes. Our results and evaluation have shown that video visualization can provide snooker coaching with visually quantifiable and comparable summary records, and is thus a cost‐effective means for assessing skill levels and monitoring progress objectively and consistently.

Abstract Sound is an integral part of most movies and videos. In many situations, viewers of a video are unable to hear the sound track, for example, when watching it in a fast forward mode, viewing it by hearing‐impaired viewers or when the plot is given as a storyboard. In this paper, we present an automated visualization solution to such problems. The system first detects the common components (such as music, speech, rain, explosions, and so on) from a sound track, then maps them to a collection of programmable visual metaphors, and generates a composite visualization. This form of sound visualization, which is referred to as SoundRiver, can be also used to augment various forms of video abstraction and annotated key frames and to enhance graphical user interfaces for video handling software. The SoundRiver conveys more semantic information to the viewer than traditional graphical representations of sound illustration, such as phonoautographs, spectrograms or artistic audiovisual animations.

Abstract Salience detection is a principle mechanism to facilitate visual attention. A good visualization guides the observer's attention to the relevant aspects of the representation. Hence, the distribution of salience over a visualization image is an essential measure of the quality of the visualization. We describe a method for computing such a metric for a visualization image in the context of a given dataset. We show how this technique can be used to analyze a visualization's salience, improve an existing visualization, and choose the best representation from a set of alternatives. The usefulness of this proposed metric is illustrated using examples from information visualization, volume visualization and flow visualization.

Abstract With ever increasing computing power, it is possible to process ever more complex fluid simulations. However, a gap between data set sizes and our ability to visualize them remains. This is especially true for the field of flow visualization, which deals with large, time‐dependent, multivariate simulation data sets. In this paper, geometry‐based flow visualization techniques form the focus of discussion. Geometric flow visualization methods place discrete objects in the velocity field whose characteristics reflect the underlying properties of the flow. A great amount of progress has been made in this field over the last two decades. However, a number of challenges remain, including placement, speed of computation and perception. In this survey, we review and classify geometric flow visualization literature according to the most important challenges when considering such a visualization, a central theme being the seeding algorithm upon which they are based. This paper details our investigation into these techniques with discussions on their applicability and their relative merits and drawbacks. The result is an up‐to‐date overview of the current state‐of‐the‐art that highlights both solved and unsolved problems in this rapidly evolving branch of research. It also serves as a concise introduction to the field of flow visualization research.

Pixel-based visualization is a popular method of conveying large amounts of numerical data graphically. Application scenarios include business and finance, bioinformatics and remote sensing. In this work, we examined how the usability of such visual representations varied across different tasks and block resolutions. The main stimuli consisted of temporal pixel-based visualization with a white-red color map, simulating monthly temperature variation over a six-year period. In the first study, we included 5 separate tasks to exert different perceptual loads. We found that performance varied considerably as a function of task, ranging from 75% correct in low-load tasks to below 40% in high-load tasks. There was a small but consistent effect of resolution, with the uniform patch improving performance by around 6% relative to higher block resolution. In the second user study, we focused on a high-load task for evaluating month-to-month changes across different regions of the temperature range. We tested both CIE L*u*v* and RGB color spaces. We found that the nature of the change-evaluation errors related directly to the distance between the compared regions in the mapped color space. We were able to reduce such errors by using multiple color bands for the same data range. In a final study, we examined more fully the influence of block resolution on performance, and found block resolution had a limited impact on the effectiveness of pixel-based visualization.

In this paper, we examine whether or not information theory can be one of the theoretic frameworks for visualization. We formulate concepts and measurements for qualifying visual information. We illustrate these concepts with examples that manifest the intrinsic and implicit use of information theory in many existing visualization techniques. We outline the broad correlation between visualization and the major applications of information theory, while pointing out the difference in emphasis and some technical gaps. Our study provides compelling evidence that information theory can explain a significant number of phenomena or events in visualization, while no example has been found which is fundamentally in conflict with information theory. We also notice that the emphasis of some traditional applications of information theory, such as data compression or data communication, may not always suit visualization, as the former typically focuses on the efficient throughput of a communication channel, whilst the latter focuses on the effectiveness in aiding the perceptual and cognitive process for data understanding and knowledge discovery. These findings suggest that further theoretic developments are necessary for adopting and adapting information theory for visualization.

The availability of commodity volumetric displays provides ordinary users with a new means of visualizing 3D data. Many of these displays are in the class of isotropically emissive light devices, which are designed to directly illuminate voxels in a 3D frame buffer, producing X-ray-like visualizations. While this technology can offer intuitive insight into a 3D object, the visualizations are perceptually different from what a computer graphics or visualization system would render on a 2D screen. This paper formalizes rendering on isotropically emissive displays and introduces a novel technique that emulates traditional rendering effects on isotropically emissive volumetric displays, delivering results that are much closer to what is traditionally rendered on regular 2D screens. Such a technique can significantly broaden the capability and usage of isotropically emissive volumetric displays. Our method takes a 3D dataset or object as the input, creates an intermediate light field, and outputs a special 3D volume dataset called a lumi-volume. This lumi-volume encodes approximated rendering effects in a form suitable for display with accumulative integrals along unobtrusive rays. When a lumi-volume is fed directly into an isotropically emissive volumetric display, it creates a 3D visualization with surface shading effects that are familiar to the users. The key to this technique is an algorithm for creating a 3D lumi-volume from a 4D light field. In this paper, we discuss a number of technical issues, including transparency effects due to the dimension reduction and sampling rates for light fields and lumi-volumes. We show the effectiveness and usability of this technique with a selection of experimental results captured from an isotropically emissive volumetric display, and we demonstrate its potential capability and scalability with computer-simulated high-resolution results.

Abstract In this paper, we propose a novel approach for automatic generation of visualizations from domain‐specific data available on the web. We describe a general system pipeline that combines ontology mapping and probabilistic reasoning techniques. With this approach, a web page is first mapped to a Domain Ontology, which stores the semantics of a specific subject domain (e.g., music charts). The Domain Ontology is then mapped to one or more Visual Representation Ontologies, each of which captures the semantics of a visualization style (e.g., tree maps). To enable the mapping between these two ontologies, we establish a Semantic Bridging Ontology, which specifies the appropriateness of each semantic bridge. Finally each Visual Representation Ontology is mapped to a visualization using an external visualization toolkit. Using this approach, we have developed a prototype software tool, SemViz, as a realisation of this approach. By interfacing its Visual Representation Ontologies with public domain software such as ILOG Discovery and Prefuse, SemViz is able to generate appropriate visualizations automatically from a large collection of popular web pages for music charts without prior knowledge of these web pages.

Abstract Aging has considerable visual effects on the human face and is difficult to simulate using a universally‐applicable global model. In this paper, we focus on the hypothesis that the patterns of age progression (and regression) are related to the face concerned, as the latter implicitly captures the characteristics of gender, ethnic origin, and age group, as well as possibly the person‐specific development patterns of the individual. We use a data‐driven framework for automatic image‐based facial transformation in conjunction with a database of facial images. We build a novel parameterized model for encoding age‐transformation in addition with the traditional model for face description. We utilize evolutionary computing to learn the relationship between the two models. To support this work, we also developed a new image warping algorithm based on non‐uniform radial basis functions (NURBFs). Evolutionary computing was also used to handle the large parameter space associated with NURBFs. In comparison with several different methods, it consistently provides the best results against the ground truth.

One challenge in video processing is to detect actions and events, known or unknown, in video streams dynamically. This paper proposes a visualization solution, where a video stream is depicted as a series of snapshots at a relatively sparse interval, and detected actions are highlighted with continuous abstract illustrations. The combined imagery and illustrative visualization conveys multi-field information in a manner similar to electrocardiograms (ECG) and seismographs. We thus name this type of video visualization as VideoPerpetuoGram (VPG). In this paper, we describe a system that handles the aw and processed information of the video stream in a multi-field visualization pipeline. As examples, we consider the needs for highlighting several types of processed information, including detected actions in video streams, and estimated relationship between recognized objects. We examine the effective means for depicting multi-field information in VPG, and support our choice of visual mappings through a survey. Our GPU implementation facilitates the VPG-specific viewing specification through a sheared object space, as well as volume bricking and combinational rendering of volume data and glyphs.

Abstract A sampled object representation (SOR) defines a graphical model using data obtained from a sampling process, which takes a collection of samples at discrete positions in space in order to capture certain geometrical and physical properties of one or more objects of interest. Examples of SORs include images, videos, volume datasets and point datasets. Unlike many commonly used data representations in computer graphics, SORs lack in geometrical, topological and semantic information, which is much needed for controlling deformation and animation. Hence it poses a significant scientific and technical challenge to develop deformation and animation methods that operate upon SORs. Such methods can enable computer graphics and computer animation to benefit enormously from the advances of digital imaging technology. In this state of the art report, we survey a wide range of techniques that have been developed for manipulating, deforming and animating SORs. We consider a collection of elementary operations for manipulating SORs, which can serve as building blocks of deformation and animation techniques. We examine a collection of techniques that are designed to transform the geometry shape of deformable objects in sampled representations and pay particular attention to their deployment in surgical simulation. We review a collection of techniques for animating digital characters in SORs, focusing on recent developments in volume animation.

Splitting a volumetric object is a useful operation in volume graphics and its applications, but is not widely supported by existing systems for volume-based modeling and rendering. In this paper, we present an investigation into two main algorithmic approaches, namely, explicit and implicit splitting, for modeling and rendering splitting actions. We consider a generalized notion based on scalar fields, which encompasses discrete specifications (e.g., volume data sets) as well as procedural specifications (e.g., hypertextures) of volumetric objects. We examine the correctness, effectiveness, efficiency, and deficiencies of each approach in specifying and controlling a spatial and temporal specification of splitting. We propose methods for implementing these approaches and for overcoming their deficiencies. We present a modeling tool for creating specifications of splitting functions, and describe the use of volume scene graphs for facilitating direct rendering of volume splitting. We demonstrate the use of these approaches with examples of volume visualization, medical illustration, volume animation, and special effects.

Video visualization is a newly emerged technology drawing the concepts and methodologies from volume and flow visualization, image and video processing, and vision science. It extracts meaningful information from a video data set and conveys the extracted information to users in appropriate visual representations. It is not intended to provide fully automatic solutions to the traditional problems in video processing, but aims at offering an approach that involves human in the loop of intelligent reasoning while removing the burden of viewing videos. In this paper, we examine the technical scope of this new technology and relate it to the technical advances in video processing. In particular, we highlight the areas where video visualization can offer more generic solutions with minimal effort for adjusting scene-specific parameters and hard-coding application-specific logic, as well as areas where video visualization can benefit from a large collection of theories and algorithms which have already been developed for video processing. We also briefly discuss the potential applicability of advances in video processing to the broader spectrum of time-varying data visualization

Data management is the very first issue in handling very large datasets. Many existing out-of-core algorithms used in visualization are closely coupled with application-specific logic. This paper presents two knowledgebased out-of-core prefetching algorithms that do not use hard-coded rendering-related logic. They acquire the knowledge of the access history and patterns dynamically, and adapt their prefetching strategies accordingly. We have compared the algorithms with a demand-based algorithm, as well as a more domain-specific out-of-core algorithm. We carried out our evaluation in conjunction with an example application where rendering multiple point sets in a volume scene graph put a great strain on the rendering algorithm in terms of memory management. Our results have shown that the knowledge-based approach offers a better cache-hit to disk-access trade-off. This work demonstrates that it is possible to build an out-of-core prefetching algorithm without depending on rendering-related application-specific logic. The knowledge based approach has the advantage of being generic, efficient, flexible and self-adaptive.

Video visualization is a computation process that extracts meaningful information from original video data sets and conveys the extracted information to users in appropriate visual representations. This paper presents a broad treatment of the subject, following a typical research pipeline involving concept formulation, system development, a path-finding user study, and a field trial with real application data. In particular, we have conducted a fundamental study on the visualization of motion events in videos. We have, for the first time, deployed flow visualization techniques in video visualization. We have compared the effectiveness of different abstract visual representations of videos. We have conducted a user study to examine whether users are able to learn to recognize visual signatures of motions, and to assist in the evaluation of different visualization techniques. We have applied our understanding and the developed techniques to a set of application video clips. Our study has demonstrated that video visualization is both technically feasible and cost-effective. It has provided the first set of evidence confirming that ordinary users can be accustomed to the visual features depicted in video visualizations, and can learn to recognize visual signatures of a variety of motion events.

In this paper we describe a GPU-based technique for creating illustrative visualization through interactive manipulation of volumetric models. It is partly inspired by medical illustrations, where it is common to depict cuts and deformation in order to provide a better understanding of anatomical and biological structures or surgical processes, and partly motivated by the need for a real-time solution that supports the specification and visualization of such illustrative manipulation. We propose two new feature-aligned techniques, namely surface alignment and segment alignment, and compare them with the axis-aligned techniques which was reported in previous work on volume manipulation. We also present a mechanism for defining features using texture volumes, and methods for computing correct normals for the deformed volume in respect to different alignments. We describe a GPU-based implementation to achieve real-time performance of the techniques and a collection of manipulation operators including peelers, retractors, pliers and dilators which are adaptations of the metaphors and tools used in surgical procedures and medical illustrations. Our approach is directly applicable in medical and biological illustration, and we demonstrate how it works as an interactive tool for focus+context visualization, as well as a generic technique for volume graphics.

Colour realism plays an important role in computer graphics and visualization. In this paper, we present a new approach to direct volume rendering based on the Kubelka-Munk theory of diffuse reflectance. We show that not only the Kubelka-Munk theory facilitates a correct spectral volume rendering integral suitable for both solid objects and amorphous matters in volume datasets, but also provides volume visualization with more accurate optical effects than the traditional volume rendering integral based on the RGBα accumulation. We discuss the design of transfer functions for specifying absorption and scattering coefficients, and the use of post-illumination for integrating pre-processed reflectance images in real time. We demonstrate the optical realism achieved by this approach with a combination of several natural and artificial colour datasets.

Abstract If we were to have a Grid infrastructure for visualization, what technologies would be needed to build such an infrastructure, what kind of applications would benefit from it, and what challenges are we facing in order to accomplish this goal? In this survey paper, we make use of the term ‘visual supercomputing’ to encapsulate a subject domain concerning the infrastructural technology for visualization. We consider a broad range of scientific and technological advances in computer graphics and visualization, which are relevant to visual supercomputing. We identify the state‐of‐the‐art technologies that have prepared us for building such an infrastructure. We examine a collection of applications that would benefit enormously from such an infrastructure, and discuss their technical requirements. We propose a set of challenges that may guide our strategic efforts in the coming years.

Video data, generated by the entertainment industry, security and traffic cameras, video conferencing systems, video emails, and so on, is perhaps most time-consuming to process by human beings. In this paper, we present a novel methodology for "summarizing" video sequences using volume visualization techniques. We outline a system pipeline for capturing videos, extracting features, volume rendering video and feature data, and creating video visualization. We discuss a collection of image comparison metrics, including the linear dependence detector, for constructing "relative" and "absolute" difference volumes that represent the magnitude of variation between video frames. We describe the use of a few volume visualization techniques, including volume scene graphs and spatial transfer functions, for creating video visualization. In particular, we present a stream-based technique for processing and directly rendering video data in real time. With the aid of several examples, we demonstrate the effectiveness of using video visualization to convey meaningful information contained in video sequences.

Comparative evaluation of visualization and experimental results is a critical step in computational steering. In this paper, we present a study of image comparison metrics for quantifying the magnitude of difference between visualization of a computer simulation and a photographic image captured from an experiment. We examined eleven metrics, including three spatial domain, four spatial-frequency domain and four HVS (human-vision system) metrics. Among these metrics, a spatial-frequency domain metric called 2nd-order Fourier comparison was proposed specifically for this work. Our study consisted of two stages: base cases and field trials. The former is a general study on a controlled comparison space using purposely selected data, and the latter involves imagery results from computational fluid dynamics and a rheological experiment. This study has introduced a methodological framework for analyzing image-level methods used in comparative visualization. For the eleven metrics considered, it has offered a set of informative indicators as to the strengths and weaknesses of each metric. In particular, we have identified three image comparison metrics that are effective in separating "similar" and "different" image groups. Our 2nd-order Fourier comparison metric has compared favorably with others in two of the three tests, and has shown its potential to be used for steering computer simulation quantitatively.

Many graphical objects can be represented by swept volumes (including its subset — generalised cylinders) by sweeping 2D or 3D templates along 3D trajectories. In this paper, we present a new approach for constructing swept volumes using image templates. We utilise scalar fields as our underlying data type, and employ volume ray casting techniques for rendering swept volumes in their original sweeping specifications as well as in their voxelised approximations. In addition to some simple image-swept volumes, we also treat multi-channel image templates, video templates, generalised sweeps, and self-intersecting trajectories. This approach enables us to model swept volumes with heterogeneous interiors and amorphous effects. It also facilitates the use of constructive volume geometry for creating complex scenes in both modelling and rendering space.

This paper presents the design and implementation of artistic effects in modelling and rendering of volume datasets. Following different stages of a volume-based graphics pipeline, we examine various properties of volume data, and illustrate how expressive and non-photorealistic effects can be implemented. We demonstrate that the true 3D nature of volume data makes it particularly applicable for this use, allowing the addition of complex effects at the modelling stage as well as during rendering.

Concerns the development of non-photorealistic rendering techniques for volume visualisation. In particular, we present two pen-and-ink rendering methods, a 3D method based on non-photorealistic solid textures, and a 2/sup +/D method that involves two rendering phases in the object space and the image space respectively. As both techniques utilize volume- and image-based data representations, they can be built upon a traditional volume rendering pipeline, and can be integrated with the photorealistic methods available in such a pipeline. We demonstrate that such an integration facilitates an effective mechanism for enhancing visualisation and its interpretation.

We present an algebraic framework, called Constructive Volume Geometryn (CVG), for modelling complex spatial objects using combinational operations. By utilising scalar fields as fundamental building blocks, CVG provides high‐level algebraic representations of objects that are defined mathematically or built upon sampled or simulated datasets. It models amorphous phenomena as well as solid objects, and describes the interior as well as the exterior of objects. We also describe a hierarchical representation scheme for CVG, and a direct rendering method with a new approach for consistent sampling. The work has demonstrated the feasibility of combining a variety of graphics data types in a coherent modelling scheme.

This paper presents a method for modelling graphics scenes consisting of multiple volumetric objects. A two‐level hierarchical representation is employed, which enables the reduction of the overall storage consumption as well as rendering time. With this approach, different objects can be derived from the same volumetric dataset, and 2D images can be trivially integrated into a scene. The paper also describes an efficient algorithm for rendering such scenes on ordinary workstations, and addresses issues concerning memory requirements and disk swapping.

Presents three extensions to D.G. Kirkpatrick's (1983) point location algorithm. The extensions revolve around major focuses in the point location problem, namely the location of close successive query points and the dynamic and persistent implementations. The solutions described in this paper include (a) an algorithm that requires O(1) query time for reasonably close successive query points, with O(N) space bound and O(N) time to construct the search structure; (b) a dynamic implementation that require O(N) space, O(log N) query time and O(log N) update time; and (c) a persistent implementation of the same complexity as the dynamic interpretation. Two methods for modifying the search structure (the /spl Kscr/-structure) dynamically and persistently are also discussed. The space requirements and query times of the dynamic and persistent methods are identical to their static counterparts. To affirm the practicality, conceptual simplicity and efficiency of the new methods, we implemented all these extensions, performed computational tests and compared them with other alternative schemes. The results reflect the expected efficiencies of the underlying design.

Abstract This paper presents a new approach to the construction of a surface from a stack of contour slices. Unlike most existing methods, this new approach handles ambiguous conditions consistently without employing an algorithm to establish a correspondence between vertices on one contour and those on the next. It is easy to implement and fast to compute, requiring only basic geometric properties, namely closedness and simplicity, to be available with contour data. The advantages of this new approach have also been demonstrated with solutions to a few classical problems from the literature and some practical problems in medical imaging. It can also be applied to geographical surveying and keyframe animations.

Abstract Scientists and engineers use numerous kinds of graph notations to model various objects in the real world and their relationships in an abstract pictorial form. However, developing a specific graph editing tool for each of these notations is often very costly in terms of time and resources. This paper outlines a design of a suitable development environment for constructing graph‐based editing tools, based on the requirements analysis of such an environment derived from a survey of more than 60 different graph notations.