Deep Neural Network (DNN) evaluation is a classic topic in machine learning. Unlike existing methods that assess a DNN's general performance on pre-collected real-world data, we propose an approach to exploring a DNN's local performance on generated data. The core idea is to train a generative model (GM) on real-world data and visualize the DNN's performance over the GM's latent space (GLS). The GLS contains all generated samples (GSs) of the GM, capturing combinations of features and their variations present in the GM's training set, thereby enhancing the comprehensiveness of testing. Moreover, the plane organizes all GSs into a continuous space with regular variations in features, enabling users to conduct cross-scale exploration to understand the DNN's behavior across different levels of feature granularity. Our research focuses on two challenges in achieving and applying the idea. First, the GLS is unvisualizable due to its high dimensionality and unboundedness. We thus project the GLS onto a plane and visualize the DNN's performance on this plane. The plane serves as a surrogate for the GLS, providing rich visual cues for locating regions containing GSs involving performance-relevant patterns. Second, the plane alone is insufficient for performance exploration. To overcome this, we develop a system based on it. The system comprises two main views that integrate interactions, such as panning and zooming, to enable users to observe any plane region and assess the DNN's performance on the corresponding GSs. We also develop three auxiliary views to facilitate pattern exploration and to diagnose factors that lead to specific performance patterns. Experimental results demonstrate the usability and effectiveness of our approach.

Existing neural topic models often produce semantically ambiguous or low-quality topics, limiting their effectiveness in real-world applications. To address this, we propose ConTopic , a human-in-the-loop framework that integrates user-defined “must-link” and “cannot-link” constraints to improve topic quality. Our method employs an autoencoder-based neural network to jointly embed words, documents, and topics into a unified semantic space, enabling constraint-guided optimization via a dedicated loss function. We also introduce an interactive editing tool with three visualization strategies that help users assess topic quality, explore semantic relations, and refine topics with minimal cognitive effort. Experiments on real-world datasets, supported by quantitative evaluations and user studies, confirm the effectiveness and usability of ConTopic in enhancing topic modeling workflows.

The article presents a novel approach to visualizing adversarial robustness (called robustness below) of deep neural networks (DNNs). Traditional tests only return a value reflecting a DNN's overall robustness across a fixed number of test samples. Unlike them, we use test samples to train a generative model (GM) and render a DNN's robustness distribution over infinite generated samples within the GM's latent space. The approach extends test samples, enabling users to obtain new test samples to improve feature coverage constantly. Moreover, the distribution provides more information about a DNN's robustness, enabling users to understand a DNN's robustness comprehensively. We propose three methods to resolve the challenges of realizing the approach. Specifically, we (1) map a GM's high-dimensional latent space onto a plane with less information loss for visualization, (2) design a network to predict a DNN's robustness on massive samples to speed up the distribution rendering, and (3) develop a system to supports users to explore the distribution from multiple perspectives. Subjective and objective experiment results prove the usability and effectiveness of the approach.

Samples produced by generative models, called Generated Samples (GSs), have become a critical supplement to those collected from the real world in data-centric applications. Domain experts typically randomly collect many GSs and manually select a few of interest for applications. However, the methodology lacks guidance to locate desirable ones that exhibit specific features or adhere to application-oriented metrics among infinite generable candidates. These samples are generally concentrated in a few small regions of the generative model's latent space, called Generative Latent Space (GLS). This paper presents Latent Space Map that projects a GLS onto a plane to help users locate regions rich in desirable GSs. Our research revolves around two challenges in constructing the map. First, many GSs in a GLS are low-quality and useless for applications. Excluding them from the projection is challenging for their irregular distribution. We employ a Monte Carlo-based method to capture a manifold for projection, where high-quality GSs are mainly distributed. Second, the GLS is high-dimensional and unbounded, complicating the projection. We design a manifold projection method that endows the map with desirable characteristics to achieve high display accuracy and effective pattern perception for users freely observing the manifold. We further develop a system integrating Latent Space Map to aid in GS selection and refinement. Real-world cases, quantitative experiments, and feedback from domain experts confirm the usability and effectiveness of our approach.

This article presents a new Human-steerable Topic Modeling (HSTM) technique. Unlike existing techniques commonly relying on matrix decomposition-based topic models, we extend LDA as the fundamental component for extracting topics. LDA's high popularity and technical characteristics, such as better topic quality and no need to cherry-pick terms to construct the document-term matrix, ensure better applicability. Our research revolves around two inherent limitations of LDA. First, the principle of LDA is complex. Its calculation process is stochastic and difficult to control. We thus give a weighting method to incorporate users' refinements into the Gibbs sampling to control LDA. Second, LDA often runs on a corpus with massive terms and documents, forming a vast search space for users to find semantically relevant or irrelevant objects. We thus design a visual editing framework based on the coherence metric, proven to be the most consistent with human perception in assessing topic quality, to guide users' interactive refinements. Cases on two open real-world datasets, participants' performance in a user study, and quantitative experiment results demonstrate the usability and effectiveness of the proposed technique.

In recent years, AI-driven drug development has emerged as a prominent research topic in computer chemistry. A key focus is the application of generative models for molecule synthesis, which create extensive virtual libraries of chemical molecules based on latent spaces. However, locating molecules with desirable properties within the vast latent spaces remains a significant challenge. Large regions of invalid samples in the latent space, called “dead zones”, can impede the exploration efficiency. The process is always time-consuming and repetitive. Therefore, we aim to propose a visualization system to help experts identify potential molecules with desirable properties as they wander in the latent space. Specifically, we conducted a literature survey about the application of generative networks in drug synthesis to summarize the tasks and followed this with expert interviews to determine their requirements. Based on the above requirements, we introduce ChemNav, an interactive visual tool for navigating latent space for desirable molecules search. ChemNav incorporates a heuristic latent space interpolation path search algorithm to enhance the efficiency of valid molecule generation, and a similar sample search algorithm to accelerate the discovery of similar molecules. Evaluations of ChemNav through two case studies, a user study, and experiments demonstrated its effectiveness in inspiring researchers to explore the latent space for chemical molecule discovery.

We propose an approach to learning sample embedding for analyzing multi-dimensional datasets. The basic idea is to extract rules from the given dataset and learn the embedding for each sample based on the rules it satisfies. The approach can filter out pattern-irrelevant attributes, leading to significant visual structures of samples satisfying the same rules in the projection. In addition, analysts can understand a visual structure based on the rules that the involved samples satisfy, which improves the projection’s pattern interpretability. Our research involves two methods for achieving and applying the approach. First, we give a method to learn rule-based embedding for each sample. Second, we integrate the method into a system to achieve an analytical workflow. Cases on real-world dataset and quantitative experiment results show the usability and effectiveness of our approach.

Generative models provide an efficient way to analyze and understand unlabeled data, creating the latent space for data modeling and generation. Since the interpretation of latent space usually requires implicit expert knowledge, this human-centered feature makes visual analytic methods effective. In the filed of computer chemistry, some research have applied generative models to generate chemical spaces and generated new molecules by sampling in the latent space. However, the latent space is typically high and sparse, and there may be a large number of “dead zones”, which may lead to decoding sample points from the latent space are noisy or invalid. Therefore, it is extremely challenging to efficiently search and traverse the latent space and generate new molecules with the desired properties. This paper aims to propose a visualization system for interactive exploration of latent space, which inspires the researchers to design new potential molecules with desired properties. The main work of this paper is as follows: First, we investigate a series of literature on the application of generative networks to drug design and synthesis, and interview experts with computer chemistry background to summarize the requirements and tasks. Second, based on the above requirements and tasks, we propose a graph-based latent space traversal and interpolation algorithms and neighborhood sampling algorithms. This can improve the number of generated potential molecules and the speed of discovery of similar molecules. Then, we conduct comparison experiments to verify the effectiveness of the algorithms. Finally, we design visualization system and then conduct the case study and user study to verify the effectiveness of the visualization system.

Embedding is a common technique for analyzing multi-dimensional data. However, the embedding projection cannot always form significant and interpretable visual structures that foreshadow underlying data patterns. We propose an approach that incorporates human knowledge into data embeddings to improve pattern significance and interpretability. The core idea is (1) externalizing tacit human knowledge as explicit sample labels and (2) adding a classification loss in the embedding network to encode samples' classes. The approach pulls samples of the same class with similar data features closer in the projection, leading to more compact (significant) and class-consistent (interpretable) visual structures. We give an embedding network with a customized classification loss to implement the idea and integrate the network into a visualization system to form a workflow that supports flexible class creation and pattern exploration. Patterns found on open datasets in case studies, subjects' performance in a user study, and quantitative experiment results illustrate the general usability and effectiveness of the approach.

Drug molecule design is a classic research topic. Drug experts traditionally design molecules relying on their experience. Manual drug design is time-consuming and may produce low-efficacy and off-target molecules. With the popularity of deep learning, drug experts are beginning to use generative models to design drug molecules. A well-trained generative model can learn the distribution of training samples and infinitely generate drug-like molecules similar to the training samples. The automatic process improves design efficiency. However, most existing methods focus on proposing and optimizing generative models. How to discover ideal molecules from massive candidates is still an unresolved challenge. We propose a visualization system to discover ideal drug molecules generated by generative models. In this paper, we investigated the requirements and issues of drug design experts when using generative models, i.e., generating molecular structures with specific constraint and finding other molecular structures similar to potential drug molecular structures. We formalized the first problem as an optimization problem and proposed to use a genetic algorithm to solve it. For the second problem, we proposed to use a neighborhood sampling algorithm based on the continuity of the latent space to find solutions. We integrated the proposed algorithms into a visualization tool, and a case study for discovering potential drug molecules to make KOR agonists and experiments demonstrated the utility of our approach.

We propose an approach to underpin interactive visual exploration of large data volumes by training Learned Visualization Index (LVI). Knowing in advance the data, the aggregation functions that are used for visualization, the visual encoding, and available interactive operations for data selection, LVI allows to avoid time-consuming data retrieval and processing of raw data in response to user’s interactions. Instead, LVI directly predicts aggregates of interest for the user’s data selection. We demonstrate the efficiency of the proposed approach in application to two use cases of spatio-temporal data at different scales.

Abstract Multi‐dimensional data exploration is a classic research topic in visualization. Most existing approaches are designed for identifying record patterns in dimensional space or subspace. In this paper, we propose a visual analytics approach to exploring subset patterns. The core of the approach is a subset embedding network (SEN) that represents a group of subsets as uniformly‐formatted embeddings. We implement the SEN as multiple subnets with separate loss functions. The design enables to handle arbitrary subsets and capture the similarity of subsets on single features, thus achieving accurate pattern exploration, which in most cases is searching for subsets having similar values on few features. Moreover, each subnet is a fully‐connected neural network with one hidden layer. The simple structure brings high training efficiency. We integrate the SEN into a visualization system that achieves a 3‐step workflow. Specifically, analysts (1) partition the given dataset into subsets, (2) select portions in a projected latent space created using the SEN, and (3) determine the existence of patterns within selected subsets. Generally, the system combines visualizations, interactions, automatic methods, and quantitative measures to balance the exploration flexibility and operation efficiency, and improve the interpretability and faithfulness of the identified patterns. Case studies and quantitative experiments on multiple open datasets demonstrate the general applicability and effectiveness of our approach.

In various domains, there are abundant streams or sequences of multi-item data of various kinds, e.g. streams of news and social media texts, sequences of genes and sports events, etc. Comparison is an important and general task in data analysis. For comparing data streams involving multiple items (e.g., words in texts, actors or action types in action sequences, visited places in itineraries, etc.), we propose Co-Bridges, a visual design involving connection and comparison techniques that reveal similarities and differences between two streams. Co-Bridges use river and bridge metaphors, where two sides of a river represent data streams, and bridges connect temporally or sequentially aligned segments of streams. Commonalities and differences between these segments in terms of involvement of various items are shown on the bridges. Interactive query tools support the selection of particular stream subsets for focused exploration. The visualization supports both qualitative (common and distinct items) and quantitative (stream volume, amount of item involvement) comparisons. We further propose Comparison-of-Comparisons, in which two or more Co-Bridges corresponding to different selections are juxtaposed. We test the applicability of the Co-Bridges in different domains, including social media text streams and sports event sequences. We perform an evaluation of the users' capability to understand and use Co-Bridges. The results confirm that Co-Bridges is effective for supporting pair-wise visual comparisons in a wide range of applications.

Visual analytics usually deals with complex data and uses sophisticated algorithmic, visual, and interactive techniques supporting the analysis. Findings and results of the analysis often need to be communicated to an audience that lacks visual analytics expertise. This requires analysis outcomes to be presented in simpler ways than that are typically used in visual analytics systems. However, not only analytical visualizations may be too complex for target audiences but also the information that needs to be presented. Analysis results may consist of multiple components, which may involve multiple heterogeneous facets. Hence, there exists a gap on the path from obtaining analysis findings to communicating them, within which two main challenges lie: information complexity and display complexity. We address this problem by proposing a general framework where data analysis and result presentation are linked by story synthesis, in which the analyst creates and organises story contents. Unlike previous research, where analytic findings are represented by stored display states, we treat findings as data constructs. We focus on selecting, assembling and organizing findings for further presentation rather than on tracking analysis history and enabling dual (i.e., explorative and communicative) use of data displays. In story synthesis, findings are selected, assembled, and arranged in meaningful layouts that take into account the structure of information and inherent properties of its components. We propose a workflow for applying the proposed conceptual framework in designing visual analytics systems and demonstrate the generality of the approach by applying it to two diverse domains, social media and movement analysis.

We propose an approach to analyzing data in which texts are associated with spatial and temporal references with the aim to understand how the text semantics vary over space and time. To represent the semantics, we apply probabilistic topic modeling. After extracting a set of topics and representing the texts by vectors of topic weights, we aggregate the data into a data cube with the dimensions corresponding to the set of topics, the set of spatial locations (e.g., regions), and the time divided into suitable intervals according to the scale of the planned analysis. Each cube cell corresponds to a combination (topic, location, time interval) and contains aggregate measures characterizing the subset of the texts concerning this topic and having the spatial and temporal references within these location and interval. Based on this structure, we systematically describe the space of analysis tasks on exploring the interrelationships among the three heterogeneous information facets, semantics, space, and time. We introduce the operations of projecting and slicing the cube, which are used to decompose complex tasks into simpler subtasks. We then present a design of a visual analytics system intended to support these subtasks. To reduce the complexity of the user interface, we apply the principles of structural, visual, and operational uniformity while respecting the specific properties of each facet. The aggregated data are represented in three parallel views corresponding to the three facets and providing different complementary perspectives on the data. The views have similar look-and-feel to the extent allowed by the facet specifics. Uniform interactive operations applicable to any view support establishing links between the facets. The uniformity principle is also applied in supporting the projecting and slicing operations on the data cube. We evaluate the feasibility and utility of the approach by applying it in two analysis scenarios using geolocated social media data for studying people's reactions to social and natural events of different spatial and temporal scales.

Word cloud is a text visualization technique which is widely applied in helping improve semantic understanding about target materials. One of the most important features is the font size, which represents words frequencies of a document. As the result, in this paper, we explore how to set font sizes of words, and its influence on semantic understanding through people's performance with qualitative and controlled experiments. Adopting an machine learning algorithm LDA (Latent Dirichlet Allocation) topic model, we quantify semantics of the document and judge participants' accuracy performance. The experimental results show the influence of different font size on semantic understanding performance and provide insights for ways in promoting semantic understanding of word cloud.

Spatial time series is a common type of data dealt with in many domains, such as economic statistics and environmental science. There have been many studies focusing on finding and analyzing various kinds of events in time series; the term 'event' refers to significant changes or occurrences of particular patterns formed by consecutive attribute values. We focus on a further step in event analysis: discover temporal relationship patterns between event locations, i.e., repeated cases when there is a specific temporal relationship (same time, before, or after) between events occurring at two locations. This can provide important clues for understanding the formation and spreading mechanisms of events and interdependencies among spatial locations. We propose a visual exploration framework COPE (Co-Occurrence Pattern Exploration), which allows users to extract events of interest from data and detect various co-occurrence patterns among them. Case studies and expert reviews were conducted to verify the effectiveness and scalability of COPE using two real-world datasets.

Shape grammars are powerful in specifying and generating an infinite number of designs by repeatedly applying predefined transformation rules to initial shapes. However, the intractable nature of shape grammars limits its wide use in many applications. Moreover, shape grammars lack a parsing method to automatically validate generated designs. In this paper, we present an approach capable of generating tractable designs and validating generated designs. Our approach transforms polygonal shapes and shape rules in shape grammars to graphs and then uses the spatial graph grammar (SGG) formalism to generate and parse designs. We have evaluated our approach by transforming shapes and shape rules originally specified in a shape grammar for Turkish houses to our graph representation and validating designs with the transformed graph rules on VEGGIE, a graph grammar specification and induction tool. Other shape grammars can be potentially integrated with our approach to extend their capability in generating and validating designs.

Classic geospatial network visualization tends to limit itself to 2D representation by organizing edges and nodes on a 2D map or the external surface of a traditional 3D globe model. Visual clutters and occlusions due to edge crossings and node-edge overlaps make efficient and effective exploration of geospatial networks a challenge. This paper proposes an interactive visualization approach for the intuitive exploration of geospatial networks inside a spherical virtual reality environment. To reduce visual clutter and reveal network patterns, we also propose a parameterized 5-step 3D edge-bundling algorithm and a set of techniques to avoid collision of network edges with the viewpoint. Our spherical interaction and 3D edge-bundling approach have been implemented in an Oculus Rift VR system. We demonstrate the usefulness of our approach with two case studies on real-world network data and usability experiments.

We present a new metaphor-based approach to exploring the scatterplot consisting of huge amount of points. To reduce clutter, we group the data points having similar attribute distributions together and utilize the sunflower metaphor to represent the points of one group, thus reducing the original data points to several sunflowers. With this approach, the user is able to detect stable states, recurring states, outlier points, and gain knowledge about the transitions between different states. The components of our approach are normalization, grouping, dimensionality reduction and visualization. The effectiveness of the approach is shown by applying it to an actual dataset.

This paper presents a multi-view approach to analyzing driving styles from vehicle experiment data with multi-dimensional and time-series characteristics. Our approach integrates four visualizations, specialized in revealing different aspects of the driving styles. As the main view, overview view is used to identify the high-level features of the experiment data, providing the analyst a compact and intuitive view for understanding temporal and multi-dimensional patterns at the same time. Other three visualization techniques, providing complementary views, focusing on temporal features, relationships between different attributes and driving action variations. Several case studies have been conducted using a real-world dataset to verify the effectiveness and scalability of the proposed approach.

Geoscience observation data refers to the datasets consisting of time series of multiple parameters generated from the sensors at fixed locations. Although a few works have attempted to visualize features of these data, none of them views these data as a specific type and attempts to show the overview in all the space, time and attribute aspects. It is important for domain experts to select interested subsets from huge amounts of observation data according to the high level patterns shown in the overview. We present a novel approach to visualizing geoscience observation data in a compact radial view. Our solution consists of three visual elements. A map showing the spatial aspect is in the center of the visualization, while temporal and attribute aspects are seamlessly combined with the spatial information. Our approach is equipped with interactive mechanisms for highlighting the selected features, adjusting the display range, as well as interactively generating a fisheye view. We demonstrate the effectiveness and usability of our approach with a usability experiment. Eye tracking records and user feedbacks obtained in the experiment prove the effectiveness of our approach.

Trajectory data visualizations are usually 3-Dimensional. Due to the potential cluttering and occlusion problems, slow speed and poor interactivity as well as the influence of perspective to the effects of display, 3D visualization techniques have been controversial in the field of information visualization. To address these problems, this paper proposes a novel 3D trajectory visualization method employing the immersive virtual reality technology. Compared with the traditional 3D trajectory visualization methods, our method breaks the limitations of the traditional 2D display of 3D visualization, and engages users in a virtual world with optional perspective to observe 3D visualization without over-plotting. Through practical experiments, our work can reduce the over-plotting of 3D visualization results, and provide smooth transition between visualization views and interactions.