Association rule mining typically focuses on discovering global rules valid across the entire dataset. Yet local rules valid for subsets of the dataset, while significantly different from global rules, are often also of tremendous importance to analysts. In this work, we tackle this overlooked problem of online mining of localized asso-ciation rules. We provide support for analysts to interactively mine rules that are hidden in a global context yet are locally significant. To tackle this problem we design a compact multidimensional itemset-based data partitioning (MIP-index). MIP-index offers ef-ficient mining performance by utilizing precomputed results, while still allowing the user the flexibility of selecting any data subset of interest at run-time. We design a suite of alternative execu-tion strategies for processing such localized mining requests. Op-timization principles such as selection push-up, supported R-tree filter and differential treatment of contained and partially over-lapped MIPs are proposed. We analytically and experimentally demonstrate that different execution strategies are effective for dif-ferent query scenarios. Given a localized mining query, our CO-LARM query optimizer takes a cost-based approach to identify the best strategy for execution. Through extensive experiments using benchmark data sets we demonstrate that the COLARM optimizer is highly accurate in online plan selection and discovering local-ized rules (otherwise hidden in the global context) in a diversity of localized mining requests.

Abstract Linear models are commonly used to identify trends in data. While it is an easy task to build linear models using pre‐selected variables, it is challenging to select the best variables from a large number of alternatives. Most metrics for selecting variables are global in nature, and thus not useful for identifying local patterns. In this work, we present an integrated framework with visual representations that allows the user to incrementally build and verify models in three model spaces that support local pattern discovery and summarization: model complementarity, model diversity, and model representivity. Visual representations are designed and implemented for each of the model spaces. Our visualizations enable the discovery of complementary variables, i.e., those that perform well in modeling different subsets of data points. They also support the isolation of local models based on a diversity measure. Furthermore, the system integrates a hierarchical representation to identify the outlier local trends and the local trends that share similar directions in the model space. A case study on financial risk analysis is discussed, followed by a user study.

We demonstrate our SPIRE technology for supporting interactive mining of both positive and negative rules at the speed of thought. It is often misleading to learn only about positive rules, yet extremely revealing to find strongly supported negative rules. Key technical contributions of SPIRE including region-wise abstractions of rules, positive-negative rule relationship analysis, rule redundancy management and rule visualization supporting novel exploratory queries will be showcased. The audience can interactively explore complex rule relationships in a visual manner, such as comparing negative rules with their positive counterparts, that would otherwise take prohibitive time. Overall, our SPIRE system provides data analysts with rich insights into rules and rule relationships while significantly reducing manual effort and time investment required.

While significant strides have been made on efficient association rule mining, the usability of mining systems woefully lags behind. In particular, the usability of rule mining systems is limited by the lack of support for interactive exploration of the relationships among rule results produced with various parameter settings. Based on a novel parameter space-driven approach, our proposed Framework for Interactive Rule Exploration (FIRE) addresses the usability shortcoming. FIRE features innovative visual displays and effective interactions that enable analysts to conduct rule exploration at the speed of thought. Particularly, the parameter space view (PSpace) displays the distribution of rules produced for diverse parameter settings. This not only facilitates user parameter selection but also empowers analyst's to understand rule relationships in the parameter space context. Our user study with 22 subjects establishes the usability and effectiveness of the proposed features and interactions of FIRE using benchmark datasets. Overall, this research encompasses significant contributions at the intersection of data mining, knowledge management and visual analytics.

We demonstrate our SPHINX system that not only derives but also visualizes evidence-hypotheses relationships on a parameter space of belief and plausibility. SPHINX facilitates the analyst to interactively explore the contribution of different pieces of evidence towards the hypotheses. The key technical contributions of SPHINX include both computational and visual dimensions. The computational contributions cover (a.) flexible computational model selection; and (b.) real-time incremental strength computations. The visual contributions include (a.) sense-making over parameter space; (b.) filtering and abstraction options; (c.) novel visual displays such as evidence glyph and skyline views. Using two real datasets, we will demonstrate that the SPHINX system provides the analysts with rich insights into evidence-hypothesis relationships facilitating the discovery and decision making process.

We demonstrate our PARAS technology for supporting interactive association mining at near real-time speeds. Key technical innovations of PARAS, in particular, stable region abstractions and rule redundancy management supporting novel parameter space-centric exploratory queries will be showcased. The audience will be able to interactively explore the parameter space view of rules. They will experience near real-time speeds achieved by PARAS for operations, such as comparing rule sets mined using different parameter values, that would otherwise take hours of computation and much manual investigation. Overall, we will demonstrate that the PARAS system provides a rich experience to data analysts through parameter tuning recommendations while significantly reducing the trial-and-error interactions.

Association rule mining is known to be computationally intensive, yet real-time decision-making applications are increasingly intolerant to delays. In this paper, we introduce the parameter space model, called PARAS. PARAS enables efficient rule mining by compactly maintaining the final rulesets. The PARAS model is based on the notion of stable region abstractions that form the coarse granularity ruleset space. Based on new insights on the redundancy relationships among rules, PARAS establishes a surprisingly compact representation of complex redundancy relationships while enabling efficient redundancy resolution at query-time. Besides the classical rule mining requests, the PARAS model supports three novel classes of exploratory queries. Using the proposed PSpace index, these exploratory query classes can all be answered with near real-time responsiveness. Our experimental evaluation using several benchmark datasets demonstrates that PARAS achieves 2 to 5 orders of magnitude improvement over state-of-the-art approaches in online association rule mining.

We will demonstrate the visual analytics system V istreamT, that supports interactive mining of complex patterns within and across live data streams and stream pattern archives. Our system is equipped with both computational pattern mining and visualization techniques, which allow it to not only efficiently discover and manage patterns but also effectively convey the mining results to human analysts through visual displays. In our demonstration, we will illustrate that with V istreamT, analysts can easily submit, monitor and interact with a broad range of query types for pattern mining. This includes novel strategies for extracting complex patterns from streams in real time, summarizing neighbour-based patterns using multi-resolution compression strategies, selectively pushing patterns into the stream archive, validating the popularity or rarity of stream patterns by stream archive matching, and pattern evolution tracking to link patterns across time.

Visual Analytics (VA) is an evolving field that, at its core, is directed to the science of analytical reasoning supported by highly interactive visual interfaces. The IEEE Conference on Visual Analytics Science and Technology (IEEE VAST), founded in 2006 as the IEEE Symposium on Visual Analytics Science and Technology, is the first international conference dedicated to advances in Visual Analytics Science and Technology. The IEEE Transactions on Visualization and Computer Graphics (TVCG) has recognized and honored the importance of Visual Analytics from the beginning, and invites the authors of the best conference papers to submit substantively extended versions of VAST papers to the journal. For these papers, TVCG applies the usual standard in asking for more than 30% new material and insights compared to the conference paper. This special section presents the extended versions of the best papers of IEEE VAST 2011, which took place in October 2011 in Providence, Rhode Island, USA. These papers were selected together with the best paper award selection committee, which was composed of three members who reviewed the top papers and their peer reviews. The three selected papers went through the regular and standard reviewing process of TVCG. The papers presented here reflect the diversity of the growing field of visual analytics. Collectively, the set of papers exemplify three components that are central to visual analytics as a field. the Guest Editors then provide an overview of the technical articles and features presented in this issue.

In diverse applications ranging from stock trading to traffic monitoring, data streams are continuously monitored by multiple analysts for extracting patterns of interest in real time. These analysts often submit similar pattern mining requests yet customized with different parameter settings. In this work, we present shared execution strategies for processing a large number of neighbor-based pattern mining requests of the same type yet with arbitrary parameter settings. Such neighbor-based pattern mining requests cover a broad range of popular mining query types, including detection of clusters, outliers, and nearest neighbors. Given the high algorithmic complexity of the mining process, serving multiple such queries in a single system is extremely resource intensive. The naive method of detecting and maintaining patterns for different queries independently is often infeasible in practice, as its demands on system resources increase dramatically with the cardinality of the query workload. In order to maximize the efficiency of the system resource utilization for executing multiple queries simultaneously, we analyze the commonalities of the neighbor-based pattern mining queries, and identify several general optimization principles which lead to significant system resource sharing among multiple queries. In particular, as a preliminary sharing effort, we observe that the computation needed for the range query searches (the process of searching the neighbors for each object) can be shared among multiple queries and thus saves the CPU consumption. Then we analyze the interrelations between the patterns identified by queries with different parameters settings, including both pattern-specific and window-specific parameters. For that, we first introduce an incremental pattern representation, which represents the patterns identified by queries with different pattern-specific parameters within a single compact structure. This enables integrated pattern maintenance for multiple queries. Second, by leveraging the potential overlaps among sliding windows, we propose a metaquery strategy which utilizes a single query to answer multiple queries with different window-specific parameters. By combining these three techniques, namely the range query search sharing, integrated pattern maintenance, and metaquery strategy, our framework realizes fully shared execution of multiple queries with arbitrary parameter settings. It achieves significant savings of computational and memory resources due to shared execution. Our comprehensive experimental study, using real data streams from domains of stock trades and moving object monitoring, demonstrates that our solution is significantly faster than the independent execution strategy, while using only a small portion of memory space compared to the independent execution. We also show that our solution scales in handling large numbers of queries in the order of hundreds or even thousands under high input data rates.

A continuous top-k query retrieves the k most preferred objects in a data stream according to a given preference function. These queries are important for a broad spectrum of applications ranging from web-based advertising to financial analysis. In various streaming applications, a large number of such continuous top-k queries need to be executed simultaneously against a common popular input stream. To efficiently handle such top-k query workload, we present a comprehensive framework, called MTopS.Within this MTopS framework, several computational components work collaboratively to first analyze the commonalities across the workload; organize the workload for maximized sharing opportunities; execute the workload queries simultaneously in a shared manner; and output query results whenever any input query requires. In particular, MTopS supports two proposed algorithms, MTopBand and MTopList, which both incrementally maintain the top-k objects over time for multiple queries. As the foundation, we first identify the minimal object set from the data stream that is both necessary and sufficient for accurately answering all top-k queries in the workload. Then, the MTopBand algorithm is presented to incrementally maintain such minimum object set and eliminate the need for any recomputation from scratch. To further optimize MTop-Band, we design the second algorithm, MTopList which organizes the progressive top-k results of workload queries in a compact structure. MTopList is shown to be memory optimal and also more efficient in terms of CPU time usage than MTopBand. Our experimental study, using real data streams from domains of stock trades and moving object monitoring, demonstrates that both the efficiency and scalability of our proposed techniques are clearly superior to the state-of-the-art solutions.

Although various mining algorithms have been proposed in the literature to efficiently compute clusters, few strides have been made to date in helping analysts to interactively explore such patterns in the stream context. We present a framework called CLUES to both computationally and visually support the process of real-time mining of density-based clusters. CLUES is composed of three major components. First, as foundation of CLUES, we develop an evolution model of density-based clusters in data streams that captures the complete spectrum of cluster evolution types across streaming windows. Second, to equip CLUES with the capability of efficiently tracking cluster evolution, we design a novel algorithm to piggy-back the evolution tracking process into the underlying cluster detection process. Third, CLUES organizes the detected clusters and their evolution interrelationships into a multidimensional pattern space - presenting clusters at different time horizons and across different abstraction levels. It provides a rich set of visualization and interaction techniques to allow the analyst to explore this multi-dimensional pattern space in real-time. Our experimental evaluation, including performance studies and a user study, using real streams from ground group movement monitoring and from stock transaction domains confirm both the efficiency and effectiveness of our proposed CLUES framework.

Continuous top-k queries, which report a certain number (k) of top preferred objects from data streams, are important for a broad class of real-time applications, ranging from financial analysis to network traffic monitoring. Existing solutions for tackling this problem aim to reduce the computational costs by incrementally updating the top-k results upon each window slide. However, they all suffer from the performance bottleneck of periodically requiring a complete recomputation of the top-k results from scratch. Such an operation is not only computationally expensive but also causes significant memory consumption, as it requires keeping all objects alive in the query window. To solve this problem, we identify the "Minimal Top-K candidate set" (MTK), namely the subset of stream objects that is both necessary and sufficient for continuous top-k monitoring. Based on this theoretical foundation, we design the MinTopk algorithm that elegantly maintains MTK and thus eliminates the need for recomputation. We prove the optimality of the MinTopk algorithm in both CPU and memory utilization for continuous top-k monitoring. Our experimental study shows that both the efficiency and scalability of our proposed algorithm is clearly superior to the state-of-the-art solutions.

Discovering interesting patterns in datasets is a very important data mining task. Subgroup patterns are local findings identifying the subgroups of a population with some unusual, unexpected, or deviating distribution of a target attribute. However, this pattern discovery task poses several compelling challenges. First, computational data mining techniques can generally only discover and extract pre-defined patterns. Second, since the extracted patterns are typically multi-dimensional arbitrary-shaped regions, it is very difficult to convey in an easily interpretable manner. Finally, in order to assist analysts in exploring their discoveries and understanding the relationships among patterns, as well as connections between patterns and the underlying data instances, an integrated visualization system is greatly needed. In this paper, we present a novel subgroup pattern extraction and visualization system, called the Nugget Browser, that takes advantage of both data mining methods and interactive visual exploration. The system accepts analysts' mining queries interactively, converts the query results into an understandable form, builds visual representations, and supports navigation and exploration for further analyses.

Abstract Time‐series data is a common target for visual analytics, as they appear in a wide range of application domains. Typical tasks in analyzing time‐series data include identifying cyclic behavior, outliers, trends, and periods of time that share distinctive shape characteristics. Many methods for visualizing time series data exist, generally mapping the data values to positions or colors. While each can be used to perform a subset of the above tasks, none to date is a complete solution. In this paper we present a novel approach to time‐series data visualization, namely creating multivariate data records out of short subsequences of the data and then using multivariate visualization methods to display and explore the data in the resulting shape space. We borrow ideas from text analysis, where the use of N‐grams is a common approach to decomposing and processing unstructured text. By mapping each temporal N‐gram to a glyph, and then positioning the glyphs via PCA (basically a projection in shape space), many different kinds of patterns in the sequence can be readily identified. Interactive selection via brushing, in conjunction with linking to other visualizations, provides a wide range of tools for exploring the data. We validate the usefulness of this approach with examples from several application domains and tasks, comparing our methods with traditional time‐series visualizations.

Density-based cluster mining is known to serve a broad range of applications ranging from stock trade analysis to moving object monitoring. Although methods for efficient extraction of density-based clusters have been studied in the literature, the problem of summarizing and matching of such clusters with arbitrary shapes and complex cluster structures remains unsolved. Therefore, the goal of our work is to extend the state-of-art of density-based cluster mining in streams from cluster extraction only to now also support analysis and management of the extracted clusters. Our work solves three major technical challenges. First, we propose a novel multi-resolution cluster summarization method, called Skeletal Grid Summarization (SGS), which captures the key features of density-based clusters, covering both their external shape and internal cluster structures. Second, in order to summarize the extracted clusters in real-time, we present an integrated computation strategy C-SGS, which piggybacks the generation of cluster summarizations within the online clustering process. Lastly, we design a mechanism to efficiently execute cluster matching queries, which identify similar clusters for given cluster of analyst's interest from clusters extracted earlier in the stream history. Our experimental study using real streaming data shows the clear superiority of our proposed methods in both efficiency and effectiveness for cluster summarization and cluster matching queries to other potential alternatives.

We will demonstrate our system, called V iStream, supporting interactive visual exploration of neighbor-based patterns [7] in data streams. V iStream does not only apply innovative multi-query strategies to compute a broad range of popular patterns, such as clusters and outliers, in a highly efficient manner, but it also provides a rich set of visual interfaces and interactions to enable real-time pattern exploration. With ViStream, analysts can easily interact with pattern mining processes by navigating along the time horizons, abstraction levels and parameter spaces, and thus better understand the phenomena of interest.

The discovery of complex patterns such as clusters, outliers, and associations from huge volumes of streaming data has been recognized as critical for many domains. However, pattern detection with sliding window semantics, as required by applications ranging from stock market analysis to moving object tracking remains largely unexplored. Applying static pattern detection algorithms from scratch to every window is prohibitively expensive due to their high algorithmic complexity. This work tackles this problem by developing the first solution for incremental detection of neighbor-based patterns specific to sliding window scenarios. The specific pattern types covered in this work include density-based clusters and distance-based outliers. Incremental pattern computation in highly dynamic streaming environments is challenging, because purging a large amount of to-be-expired data from previously formed patterns may cause complex pattern changes including migration, splitting, merging and termination of these patterns. Previous incremental neighbor-based pattern detection algorithms, which were typically not designed to handle sliding windows, such as incremental DBSCAN, are not able to solve this problem efficiently in terms of both CPU and memory consumption. To overcome this, we exploit the "predictability" property of sliding windows to elegantly discount the effect of expiring objects on the remaining pattern structures. Our solution achieves minimal CPU utilization, while still keeping the memory utilization linear in the number of objects in the window. Our comprehensive experimental study, using both synthetic as well as real data from domains of stock trades and moving object monitoring, demonstrates superiority of our proposed strategies over alternate methods in both CPU and memory utilization.

In diverse applications ranging from stock trading to traffic monitoring, popular data streams are typically monitored by multiple analysts for patterns of interest. These analysts may submit similar pattern mining requests, such as cluster detection queries, yet customized with different parameter settings. In this work, we present an efficient shared execution strategy for processing a large number of density-based cluster detection queries with arbitrary parameter settings. Given the high algorithmic complexity of the clustering process and the real-time responsiveness required by streaming applications, serving multiple such queries in a single system is extremely resource intensive. The naive method of detecting and maintaining clusters for different queries independently is often in-feasible in practice, as its demands on system resources increase dramatically with the cardinality of the query workload. To overcome this, we analyze the interrelations between the cluster sets identified by queries with different parameters settings, including both pattern-specific and window-specific parameters. We introduce the notion of the growth property among the cluster sets identified by different queries, and characterize the conditions under which it holds. By exploiting this growth property we propose a uniform solution, called Chandi , which represents identified cluster sets as one single compact structure and performs integrated maintenance on them -- resulting in significant sharing of computational and memory resources. Our comprehensive experimental study, using real data streams from domains of stock trades and moving object monitoring, demonstrates that Chandi is on average four times faster than the best alternative methods, while using 85% less memory space in our test cases. It also shows that Chandi scales in handling large numbers of queries on the order of hundreds or even thousands under high input data rates.

Queries issued by casual users or specialists exploring a dataset often point us to important subsets of the data, be it clusters, outliers or other meaningful features. Capturing and caching such queries (henceforth called nuggets) has many potential benefits, including the optimization of the system performance and the search experience of users. Unfortunately, current visual exploration systems have not yet tapped into this potential resource of identifying and sharing important queries. In this paper, we introduce a query consolidation strategy aimed at solving the general problem of isolating important queries from the potentially huge amount of queries submitted. Our solution clusters redundant queries caused by exploration-style query specification, which is prevalent in data exploration systems. To measure the similarity between queries, we designed an effective distance metric that incorporates both the query specification and the actual query result. To overcome its high complexity when comparing queries with large result sets, we designed an approximation method, which is efficient while still providing excellent accuracy. A user study conducted on multivariate data sets comparing our proposed technique to others in the literature confirms that the proposed distance metric indeed matches well with users' intuition. As proof of feasibility, we integrated our proposed query consolidation solution into the Nugget Management System (NMS) framework [22], which is based on a visual exploration system XmdvTool. A second user study indicates that both the efficiency and accuracy of users' visual exploration are enhanced when supported by NMS.

In this work, we describe our approach for making the interactive data exploration system, called XmdvTool, quality-aware to assure informed decision-making. XmdvToolQ, makes quality or lack thereof explicit for all stages of the data exploration process from raw data, to abstracted data, to the final visual displays, allowing users to query and navigate through data-, structure- and quality-spaces.

Few existing visualization systems can handle large data sets with hundreds of dimensions, since high-dimensional data sets cause clutter on the display and large response time in interactive exploration. In this paper, we present a significantly improved multidimensional visualization approach named Value and Relation (VaR) display that allows users to effectively and efficiently explore large data sets with several hundred dimensions. In the VaR display, data values and dimension relationships are explicitly visualized in the same display by using dimension glyphs to explicitly represent values in dimensions and glyph layout to explicitly convey dimension relationships. In particular, pixel-oriented techniques and density-based scatterplots are used to create dimension glyphs to convey values. Multidimensional scaling, Jigsaw map hierarchy visualization techniques, and an animation metaphor named Rainfall are used to convey relationships among dimensions. A rich set of interaction tools has been provided to allow users to interactively detect patterns of interest in the VaR display. A prototype of the VaR display has been fully implemented. The case studies presented in this paper show how the prototype supports interactive exploration of data sets of several hundred dimensions. A user study evaluating the prototype is also reported in this paper.

Data abstraction techniques are widely used in multiresolution visualization systems to reduce visual clutter and facilitate analysis from overview to detail. However, analysts are usually unaware of how well the abstracted data represent the original dataset, which can impact the reliability of results gleaned from the abstractions. In this paper, we define two data abstraction quality measures for computing the degree to which the abstraction conveys the original dataset: the Histogram Difference Measure and the Nearest Neighbor Measure. They have been integrated within XmdvTool, a public-domain multiresolution visualization system for multivariate data analysis that supports sampling as well as clustering to simplify data. Several interactive operations are provided, including adjusting the data abstraction level, changing selected regions, and setting the acceptable data abstraction quality level. Conducting these operations, analysts can select an optimal data abstraction level. Also, analysts can compare different abstraction methods using the measures to see how well relative data density and outliers are maintained, and then select an abstraction method that meets the requirement of their analytic tasks.

THREE papers in this issue of IEEE Transactions on Visualization and Computer Graphics (TVCG) are expanded versions of ones presented at InfoVis 2005. These examples of the cutting edge of information visualization research showcase the diversity and depth of the field, illustrating new display techniques as well as novel application domains for information visualization systems. The three papers focus on the visualization of three different styles of data: graph-based data, time series data, and categorical data. The techniques developed and described in these papers may also be applicable to data from a variety of problem areas and the authors include both design motivations in their work as well as illustrative examples of the application of the techniques. “Drawing Directed Graphs Using Quadratic Programming,” by Tim Dwyer, Yehuda Koren, and Kim Marriott, won the InfoVis 2005 Best Paper Award. In this paper, the authors introduce a new method for drawing directed graphs that combines constraint programming techniques with a high performance force-directed placement algorithm. The technique is useful for highlighting hierarchies in directed graphs while retaining beneficial properties of force-directed placement strategies such as proximity and symmetry relations. The authors also describe experiments that show this new visualization technique can convey the structure of large digraphs better than the most widely used hierarchical graph drawing method. “Designing for Social Data Analysis,” by Martin Wattenberg and Jesse Kriss, explores how an information visualization tool may become part of a dynamic online social environment. The authors focus on the area of baby naming and provide a delightful tool called the NameVoyager, a Web-based system that allows people to explore historical trends in the names that parents give to their children. The NameVoyager garnered huge interest on the Web when it was deployed and the authors explore how the system facilitates a form of social data analysis. The paper describes design decisions and implementation issues that arose for the system and it considers some of the reasons why the system became so popular. The paper concludes by discussing the design of an extension to the system for a more complex data set. “Parallel Sets: Interactive Exploration and Visual Analysis of Categorical Data,” by Robert Kosara, Fabian Bendix, and Helwig Hauser, applies a variation of the well-known parallel coordinates visualization technique for representing categorical data. The introduced technique shows data frequencies instead of individual data points and uses boxes and parallelograms within the parallel coordinates style plot. The authors include a rich set of interaction techniques with the visualization that allow viewers to examine many different perspectives on the data. They illustrate the power of their visualization through sample analysis scenarios with two example data sets.

Modern computer applications from business decision support to scientific data analysis use visualization techniques. However, visual exploration tools do not scale well for large data sets due to screen clutter. Visualization tools have thus been extended to support hierarchical views of the data, with support for focusing and drilling-down using interactive brushes. We now investigate how best to couple such a near real-time responsive visualization tool with database support. For this, we have developed a tree labeling method, called MinMax tree, that allows the movement of the on-line recursive processing of visual user interactions on hierarchical data sets into an off-line precomputation step. Using MinMax tree we map the recursive processing at the interface level to two dimensional range queries that can be answered efficiently using spatial indexes. We also employ caching and prefetching at the client side to cope with the real-time response requirements. The techniques have been incorporated into XmdvTool, a free software package for multi-variate data visualization and exploration. Our experimental results show 70% to 80% reduction in response time latency even with limited system resources.

HREE papers in this issue of TVCG are expanded versions of ones presented at InfoVis 2004. These examples of the cutting edge of information visualization research show the maturing of the field, using novel methods for designing, developing, and evaluating tools that help users solve real problems on large and complex data sets, rather than just creating interesting displays. “Knowledge Precepts for Design and Evaluation of Information Visualizations” by Robert A. Amar and John T. Stasko won the InfoVis 2004 Best Paper Award. In this paper, the authors argue that efforts to date to design effective visualization tools often fail because they concentrate mostly on presenting data, as opposed to supporting analysis. Limitations of existing systems are grouped into two gaps in the analysis process; the Worldview Gap consists of the difference between what is shown and what is needed to make decisions, while the Rationale Gap consists of the difference between relationships viewed and the certainty and utility of the relationship. Analysis of existing tools using this framework reveals many ways in which techniques can be improved to better support high-level analysis. “An Insight-Based Methodology for Evaluating Bioinformatics Visualizations” by Purvi Saraiya, Chris North, and Karen Duca presents a novel method for assessing and comparing the effectiveness of visualization tools, namely, the number and type of insights discovered. The authors define an insight as a unit of discovery and identify several attributes and categories of insights. They then report the results from controlled experiments used to evaluate five software tools for visual exploration of microarray data. The results shed many insights into what makes a visualization tool effective.

Modern computer applications, from business decision support to scientific data analysis, utilize data visualization tools to support exploratory activities. Visual exploration tools typically do not scale well when applied to huge data sets, partially because being interactive necessitates real-time responses. However, we observe that interactive visual explorations exhibit several properties that can be exploited for data access optimization, including locality of exploration, contiguous queries, and significant delays between user operations. We thus apply semantic caching of active query sets on the client side to exploit some of the above characteristics. We also introduce several prefetching strategies, each exploiting characteristics of our visual exploration environment. We have incorporated caching and prefetching strategies into XmdvTool, a public-domain tool for visual exploration of multivariate data sets. Experimental studies using synthetic as well as real user traces are conducted. Our results demonstrate that these proposed optimization techniques achieve significant performance improvements in our exploratory analysis system.

Accessing data stored in persistent memory represents a bottleneck for current visual exploration applications. Semantic caching of frequent queries at the client-side along with prefetching can improve performance of such systems. However, a prefetching setup that only uses one prefetching strategy may be insufficient because (1) different users have different exploration patterns, and (2) a user's pattern may be changing within the same session. To solve this, existing research focuses on refining a single prefetching strategy. We, on the other hand, now propose a framework wherein prefetching strategies are adaptively selected over time across and within one user session. This work is the first to study adaptive prefetching in the context of visual data exploration. Specifically, we have implemented our proposed approach within XmdvTool, a freeware visualization system for multivariate data, and evaluated it using real user traces. Our results confirm that our approach improves system performance by dynamically selecting the most appropriate combination of prefetching strategies that adapts to the user's changing patterns.

Must we continue to define a difference between information and scientific visualization? Scientific visualization evolved first in the late 1980’s while information visualization matured in the mid-1990’s. Scientific visualization is frequently considered to focus on the visual display of spatial data associated with scientific processes such as the bonding of molecules in computational chemistry. Information visualization examines developing visual metaphors for non-inherently spatial data such as the exploration of text-based document databases. This panel examines the effective, productive, and perhaps confusing tension between these subfields of visualization by highlighting the following issues:

Interactive selection is a critical component in exploratory visualization, allowing users to isolate subsets of the displayed information for highlighting, deleting, analysis, or focused investigation. Brushing, a popular method for implementing the selection process, has traditionally been performed in either screen space or data space. In this paper, we introduce an alternate, and potentially powerful, mode of selection that we term structure-based brushing, for selection in data sets with natural or imposed structure. Our initial implementation has focused on hierarchically structured data, specifically very large multivariate data sets structured via hierarchical clustering and partitioning algorithms. The structure-based brush allows users to navigate hierarchies by specifying focal extents and level-of-detail on a visual representation of the structure. Proximity-based coloring, which maps similar colors to data that are closely related within the structure, helps convey both structural relationships and anomalies. We describe the design and implementation of our structure-based brushing tool. We also validate its usefulness using two distinct hierarchical visualization techniques, namely hierarchical parallel coordinates and tree-maps. Finally, we discuss relationships between different classes of brushes and identify methods by which structure-based brushing could be extended to alternate data structures.

Our ability to accumulate large, complex (multivariate) data sets has far exceeded our ability to effectively process them in searching for patterns, anomalies and other interesting features. Conventional multivariate visualization techniques generally do not scale well with respect to the size of the data set. The focus of this paper is on the interactive visualization of large multivariate data sets based on a number of novel extensions to the parallel coordinates display technique. We develop a multi-resolution view of the data via hierarchical clustering, and use a variation of parallel coordinates to convey aggregation information for the resulting clusters. Users can then navigate the resulting structure until the desired focus region and level of detail is reached, using our suite of navigational and filtering tools. We describe the design and implementation of our hierarchical parallel coordinates system which is based on extending the XmdvTool system. Lastly, we show examples of the tools and techniques applied to large (hundreds of thousands of records) multivariate data sets.

Recent advances in fire science and computer modeling of fires allow scientists to predict fire growth and spread through structures. In this paper we describe a variety of visualizations of simulated room fires for use by both fire protection engineers and fire suppression personnel. We also introduce the concept of fuzzy visualization, which results from the superposition of data from several separate simulations into a single visualization.

This paper presents a paradigm for the interactive selection (querying) of data from a structured grid of data points for exploratory visualization. The paradigm is based on specifying and iteratively adjusting the Focus, Extent, and Density (FED) of the data attributes. The FED model supports highly complex queries of structured data in an intuitive fashion, and is augmented with a visual interface composed of a set of simple yet powerful user interface controls for query specification. In addition, statistical aggregations are supported by the model. Finally, the FED model is compared to the SQL paradigm, and is shown to be well suited for mapping to a direct-manipulation graphical interface.

A problem in teaching large introductory computer science courses is to overcome the impersonality of the large lecture class and to provide more personal attention to individual students.Our approach as to use peer [earning experiences to instill in students the need to take responsibility for their learning and for ihe learning of those around them.Recent work has shown that educational quality for students and productivity for faculty can be enhanced through use of peer-learning environments where students do not just learn and faculty do not just teach.The novel aspects of our work are to apply group learning in a large introductory computer science class setting and to expect more responsibility on the part of students for their learning.In support of these goals we have introduced the use of upper-level undergraduate students to help facilitate student group interaction.In addition, we have developed software to minimize the administrative overhead of handling many groups and for students to electronically record group learning activities.

Much of the attention in visualization research has focussed on data rooted in physical phenomena, which is generally limited to three or four dimensions. However, many sources of data do not share this dimensional restriction. A critical problem in the analysis of such data is providing researchers with tools to gain insights into characteristics of the data, such as anomalies and patterns. Several visualization methods have been developed to address this problem, and each has its strengths and weaknesses. This paper describes a system named XmdvTool which integrates several of the most common methods for projecting multivariate data onto a two-dimensional screen. This integration allows users to explore their data in a variety of formats with ease. A view enhancement mechanism called an N-dimensional brush is also described. The brush allows users to gain insights into spatial relationships over N dimensions by highlighting data which falls within a user-specified subspace.>

This paper describes a portable software package implementing a variation of the dot-matrix plot for genetic sequence comparison in conjunction with highly interactive image manipulation and examination techniques. Visualization is a qualitative method of analyzing and exploring data by presenting data in a pictorial form with an effective interface. Visualization techniques can be applied to molecular sequence comparisons to highlight both the similarities and the differences in a qualitative and descriptive manner. The method used for displaying the sequence comparisons is the correlation image (CI). Among the topics explored in this paper are the generation of correlation images, filtering CIs to visually enhance certain sequence relationships, the handling of large correlation images, and providing an effective interface for the manipulation and exploration of the sequences and their resulting images.

A shadow identification and classification method for real images is developed. The method is based on extensive analysis of shadow intensity and shadow geometry. The procedure for identifying shadows is divided into low-level, middle-level, and high-level processes. The low-level extracts dark regions from images. The middle-level process performs feature analysis on dark regions, including detecting vertices on the outlines of dark regions, identifying penumbrae in dark regions, assigning the subregions in dark regions as self-shadows and cast shadows, and finding object regions adjacent to dark regions. The high-level process integrates the information derived from the previous processes and confirms shadows among the dark regions.>

A method of visual comparison is described, that provides the scientist with a unique tool to study the qualitative relationships between three sequences of numbers or symbols. The program displays a 3D shape containing the sequence similarities and differences, which manifest themselves as simple geometric shapes and colors that a human observer can easily detect and classify. The method presents all possible correlations to the user, giving it a considerable advantage over existing sequence comparison tools that only search for a programmed subset of all possible correlations. Thus, using this technique, researchers may detect sequence similarities that other analytic methods might completely overlook. The program can also filter out undesirable or insignificant correlations. The technique is easily adapted to a wide range of applications.>

The authors present a tool for the display and analysis of N-dimensional data based on a technique called dimensional stacking. This technique is described. The primary goal is to create a tool that enables the user to project data of arbitrary dimensions onto a two-dimensional image. Of equal importance is the ability to control the viewing parameters, so that one can interactively adjust what ranges of values each dimension takes and the form in which the dimensions are displayed. This will allow an intuitive feel for the data to be developed as the database is explored. The system uses dimensional stacking, to collapse and N-dimension space down into a 2-D space and then render the values contained therein. Each value can then be represented as a pixel or rectangular region on a 2-D screen whose intensity corresponds to the data value at that point.>

A description is given of progress toward implementing a rule-based visual inspection system. The target product for inspection is integrated-circuit lead frames. A frame-based representation system is used to describe the product and defect categories. The system uses three levels of visual inspection to identify generic and specific inspection targets and defect categories.>