This paper proposes a method of abstractive summarization designed to scale to document collections instead of individual documents. Our approach combines semantic clustering, document size reduction within topic clusters, semantic chunking of a cluster's documents, GPT-based summarization and concatenation, and a combined sentiment and text visualization of each topic to support exploratory data analysis. A statistical comparison of our results with those of existing state-of-the-art systems, including BART, BRIO, PEGASUS, and MoCa, using ROUGE and METEOR summary scores showed statistically equivalent performance with BART and PEGASUS in the CNN/Daily Mail test dataset and with BART in the Gigaword test dataset. This finding is promising, since we view document collection summarization as more challenging than individual document summarization. We conclude with a discussion of how issues of scale are being addressed in the GPTlarge language model, then suggest potential areas for future work.

Stereo matching is a classic challenging problem in computer vision, which has recently witnessed remarkable progress by Deep Neural Networks (DNNs). This paradigm shift leads to two interesting and entangled questions that have not been addressed well. First, it is unclear whether stereo matching DNNs that are trained from scratch really learn to perform matching well. This paper studies this problem from the lens of white-box adversarial attacks. It presents a method of learning stereo-constrained photometrically-consistent attacks, which by design are weaker adversarial attacks, and yet can cause catastrophic performance drop for those DNNs. This observation suggests that they may not actually learn to perform matching well in the sense that they should otherwise achieve potentially even better after stereo-constrained perturbations are introduced. Second, stereo matching DNNs are typically trained under the simulation-to-real (Sim2Real) pipeline due to the data hungriness of DNNs. Thus, alleviating the impacts of the Sim2Real photometric gap in stereo matching DNNs becomes a pressing need. Towards joint adversarially robust and domain generalizable stereo matching, this paper proposes to learn DNN-contextualized binary-pattern-driven non-parametric cost-volumes. It leverages the perspective of learning the cost aggregation via DNNs, and presents a simple yet expressive design that is fully end-to-end trainable, without resorting to specific aggregation inductive biases. In experiments, the proposed method is tested in the SceneFlow dataset, the KITTI2015 dataset, and the Middlebury dataset. It significantly improves the adversarial robustness, while retaining accuracy performance comparable to state-of-the-art methods. It also shows a better Sim2Real generalizability. Our code and pretrained models are released at \href{https://github.com/kelkelcheng/AdversariallyRobustStereo}{this Github Repo}.

Abstract This paper describes the design and implementation of a web‐based system to visualize large collections of text conversations integrated into a hierarchical four‐level‐of‐detail design. Viewers can visualize conversations: (1) in a streamgraph topic overview for a user‐specified time period; (2) as emotion patterns for a topic chosen from the streamgraph; (3) as semantic sequences for a user‐selected emotion pattern, and (4) as an emotion‐driven conversation graph for a single conversation. We collaborated with the Live Chatcustomer service group at SAS Institute to design and evaluate our system's strengths and limitations.

Deep neural networks (DNNs) have made tremendous progress in many different areas in recent years. How these networks function internally, however, is often not well understood. Advances in under-standing DNNs will benefit and accelerate the development of the field. We present TNNVis, a visualization system that supports un-derstanding of deep neural networks specifically designed to analyze text. TNNVis focuses on DNNs composed of fully connected and convolutional layers. It integrates visual encodings and interaction techniques chosen specifically for our tasks. The tool allows users to: (1) visually explore DNN models with arbitrary input using a combination of node-link diagrams and matrix representation; (2) quickly identify activation values, weights, and feature map patterns within a network; (3) flexibly focus on visual information of interest with threshold, inspection, insight query, and tooltip operations; (4) discover network activation and training patterns through animation; and (5) compare differences between internal activation patterns for different inputs to the DNN. These functions allow neural network researchers to examine their DNN models from new perspectives, producing insights on how these models function. Clustering and summarization techniques are employed to support large convolutional and fully connected layers. Based on several part of speech models with different structure and size, we present multiple use cases where visualization facilitates an understanding of the models.

An ensemble is a collection of related datasets, called members, built from a series of runs of a simulation or an experiment. Ensembles are large, temporal, multidimensional, and multivariate, making them difficult to analyze. Another important challenge is visualizing ensembles that vary both in space and time. Initial visualization techniques displayed ensembles with a small number of members, or presented an overview of an entire ensemble, but without potentially important details. Recently, researchers have suggested combining these two directions, allowing users to choose subsets of members to visualization. This manual selection process places the burden on the user to identify which members to explore. We first introduce a static ensemble visualization system that automatically helps users locate interesting subsets of members to visualize. We next extend the system to support analysis and visualization of temporal ensembles. We employ 3D shape comparison, cluster tree visualization, and glyph based visualization to represent different levels of detail within an ensemble. This strategy is used to provide two approaches for temporal ensemble analysis: (1) segment based ensemble analysis, to capture important shape transition time-steps, clusters groups of similar members, and identify common shape changes over time across multiple members; and (2) time-step based ensemble analysis, which assumes ensemble members are aligned in time by combining similar shapes at common time-steps. Both approaches enable users to interactively visualize and analyze a temporal ensemble from different perspectives at different levels of detail. We demonstrate our techniques on an ensemble studying matter transition from hadronic gas to quark-gluon plasma during gold-on-gold particle collisions.

Network security analysis and ensemble data visualization are two active research areas. Although they are treated as separate domains, they share many common challenges and characteristics. Both focus on scalability, time-dependent data analytics, and exploration of patterns and unusual behaviors in large datasets. These overlaps provide an opportunity to apply ensemble visualization research to improve network security analysis. To study this goal, we propose methods to interpret network security alerts and flow traffic as ensemble members. We can then apply ensemble visualization techniques in a network analysis environment to produce a network ensemble visualization system. Including ensemble representations provide new, in-depth insights into relationships between alerts and flow traffic. Analysts can cluster traffic with similar behavior and identify traffic with unusual patterns, something that is difficult to achieve with high-level overviews of large network datasets. Furthermore, our ensemble approach facilitates analysis of relationships between alerts and flow traffic, improves scalability, maintains accessibility and configurability, and is designed to fit our analysts' working environment, mental models, and problem solving strategies.

This paper describes a web-based visualization system designed for network security analysts at the U.S. Army Research Laboratory (ARL). Our goal is to provide visual support to the analysts as they investigate security alerts for malicious activity within their systems. Our ARL collaborators identified a number of important requirements for any candidate visualization system. These relate to the analyst's mental models and working environment, and to the visualization tool's configurability, accessibility, scalability, and "fit" with existing analysis strategies. To meet these requirements, we designed and implement a web-based tool that uses different types of charts as its core representation framework. A JavaScript charting library (RGraph) was extended to provide the interface flexibility and correlation capabilities needed to support analysts as they explore different hypotheses about a potential attack. We describe key elements of our design, explain how an analyst's intent is used to generate different visualizations, and show how the system's interface allows an analyst to rapidly produce a sequence of visualizations to explore specific details about a potential attack as they arise. We conclude with a discussion of plans to further improve the system, and to collect feedback from our ARL colleagues on its strengths and limitations in real-world analysis scenarios.

This article describes a perceptual level-of-detail approach for visualizing data. Properties of a dataset that cannot be resolved in the current display environment need not be shown, for example, when too few pixels are used to render a data element, or when the element's subtended visual angle falls below the acuity limits of our visual system. To identify these situations, we asked: (1) What type of information can a human user perceive in a particular display environment? (2) Can we design visualizations that control what they represent relative to these limits? and (3) Is it possible to dynamically update a visualization as the display environment changes, to continue to effectively utilize our perceptual abilities? To answer these questions, we conducted controlled experiments that identified the pixel resolution and subtended visual angle needed to distinguish different values of luminance, hue, size, and orientation. This information is summarized in a perceptual display hierarchy, a formalization describing how many pixels— resolution —and how much physical area on a viewer's retina— visual angle —is required for an element's visual properties to be readily seen. We demonstrate our theoretical results by visualizing historical climatology data from the International Panel for Climate Change.

This paper describes a new method to explore and discover within a large data set. We apply techniques from preference elicitation to automatically identify data elements that are of potential interest to the viewer. These "elements of interest (EOI)" are bundled into spatially local clusters, and connected together to form a graph. The graph is used to build camera paths that allow viewers to "tour" areas of interest (AOI) within their data. It is also visualized to provide wayfinding cues. Our preference model uses Bayesian classification to tag elements in a data set as interesting or not interesting to the viewer. The model responds in real time, updating the elements of interest based on a viewer's actions. This allows us to track a viewer's interests as they change during exploration and analysis. Viewers can also interact directly with interest rules the preference model defines. We demonstrate our theoretical results by visualizing historical climatology data collected at locations throughout the world.

A fundamental goal of visualization is to produce images of data that support visual analysis, exploration, and discovery of novel insights. An important consideration during visualization design is the role of human visual perception. How we "see" details in an image can directly impact a viewer's efficiency and effectiveness. This paper surveys research on attention and visual perception, with a specific focus on results that have direct relevance to visualization and visual analytics. We discuss theories of low-level visual perception, then show how these findings form a foundation for more recent work on visual memory and visual attention. We conclude with a brief overview of how knowledge of visual attention and visual memory is being applied in visualization and graphics. We also discuss how challenges in visualization are motivating research in psychophysics.

Visualization has become integral to the knowledge discovery process across various domains. However, challenges remain in the effective use of visualization techniques, especially when displaying, exploring and analyzing large, multidimensional datasets, such as weather and meteorological data. Direct visualizations of such datasets tend to produce images that are cluttered with excess detail and are ineffective at communicating information at higher levels of abstraction. To address this problem we provide a visual summarization framework to intuitively reduce the data to its important and relevant characteristics. Summarization is performed in three broad steps. First, high-relevance data elements and clusters of similar data attributes are identified to reduce a dataset's size and dimensionality. Next, patterns, relationships and outliers are extracted from the reduced data. Finally, the extracted summary characteristics are visualized to the user. Such visualizations reduce excess visual detail and are more suited to the rapid comprehension of complex data. Users can interactively guide the summarization process gaining insight into both how and why the summary results are produced. Our framework improves the benefits of mathematical analysis and interactive visualization by combining the strengths of the computer and the user to generate high-quality summaries. Initial results from applying our framework to large weather datasets have been positive, suggesting that our approach could be beneficial for a wide range of domains and applications.

This paper describes the integration of perceptual guidelines from human vision with an AI-based mixed-initiative search strategy. The result is a visualization assistant called ViA, a system that collaborates with its users to identify perceptually salient visualizations for large, multidimensional datasets. ViA applies knowledge of low-level human vision to: (1) evaluate the effectiveness of a particular visualization for a given dataset and analysis tasks; and (2) rapidly direct its search towards new visualizations that are most likely to offer improvements over those seen to date. Context, domain expertise, and a high-level understanding of a dataset are critical to identifying effective visualizations. We apply a mixed-initiative strategy that allows ViA and its users to share their different strengths and continually improve ViA's understanding of a user's preferences. We visualize historical weather conditions to compare ViA's search strategy to exhaustive analysis, simulated annealing, and reactive tabu search, and to measure the improvement provided by mixed-initiative interaction. We also visualize intelligent agents competing in a simulated online auction to evaluate ViA's perceptual guidelines. Results from each study are positive, suggesting that ViA can construct high-quality visualizations for a range of real-world datasets.

In many applications, it is important to understand the individual values of, and relationships between, multiple related scalar variables defined across a common domain. Several approaches have been proposed for representing data in these situations. In this paper we focus on strategies for the visualization of multivariate data that rely on color mixing. In particular, through a series of controlled observer experiments, we seek to establish a fundamental understanding of the information-carrying capacities of two alternative methods for encoding multivariate information using color: color blending and color weaving. We begin with a baseline experiment in which we assess participants' abilities to accurately read numerical data encoded in six different basic color scales defined in the L*a*b* color space. We then assess participants' abilities to read combinations of 2, 3, 4 and 6 different data values represented in a common region of the domain, encoded using either color blending or color weaving. In color blending a single mixed color is formed via linear combination of the individual values in L*a*b* space, and in color weaving the original individual colors are displayed side-by-side in a high frequency texture that fills the region. A third experiment was conducted to clarify some of the trends regarding the color contrast and its effect on the magnitude of the error that was observed in the second experiment. The results indicate that when the component colors are represented side-by-side in a high frequency texture, most participants' abilities to infer the values of individual components are significantly improved, relative to when the colors are blended. Participants' performance was significantly better with color weaving particularly when more than 2 colors were used, and even when the individual colors subtended only 3 minutes of visual angle in the texture. However, the information-carrying capacity of the color weaving approach has its limits. We found that participants' abilities to accurately interpret each of the individual components in a high frequency color texture typically falls off as the number of components increases from 4 to 6. We found no significant advantages, in either color blending or color weaving, to using color scales based on component hues thatare more widely separated in the L*a*b* color space. Furthermore, we found some indications that extra difficulties may arise when opponent hues are employed.

This paper describes an experimental study of three perceptual properties of motion: flicker, direction, and velocity. Our goal is to understand how to apply these properties to represent data in a visualization environment. Results from our experiments show that all three properties can encode multiple data values, but that minimum visual differences are needed to ensure rapid and accurate target detection: flicker must be coherent and must have a cycle length of 120 milliseconds or greater, direction must differ by at least 20/spl deg/, and velocity must differ by at least 0.43/spl deg/ of subtended visual angle. We conclude with an overview of how we are applying our results to real-world data, and then discuss future work we plan to pursue.

An important problem in the area of computer graphics is the visualization of large, complex information spaces. Datasets of this type have grown rapidly in recent years, both in number and in size. Images of the data stored in these collections must support rapid and accurate exploration and analysis. This article presents a method for constructing visualizations that are both effective and aesthetic. Our approach uses techniques from master paintings and human perception to visualize a multidimensional dataset. Individual data elements are drawn with one or more brush strokes that vary their appearance to represent the element's attribute values. The result is a nonphotorealistic visualization of information stored in the dataset. Our research extends existing glyph-based and nonphotorealistic techniques by applying perceptual guidelines to build an effective representation of the underlying data. The nonphotorealistic properties the strokes employ are selected from studies of the history and theory of Impressionist art. We show that these properties are similar to visual features that are detected by the low-level human visual system. This correspondence allows us to manage the strokes to produce perceptually salient visualizations. Psychophysical experiments confirm a strong relationship between the expressive power of our nonphotorealistic properties and previous findings on the use of perceptual color and texture patterns for data display. Results from these studies are used to produce effective nonphotorealistic visualizations. We conclude by applying our techniques to a large, multidimensional weather dataset to demonstrate their viability in a practical, real-world setting.

This paper describes a system for exploring and selecting entries from a music database through a visualization interface. The system is designed for deployment in situations in which the user's attention is a tightly limited resource. The system combines research topics in intelligent user interfaces, visualization techniques, and cognitive modeling. Informal evaluation of the system has given us useful insights into the design tradeoffs that developers may face when building visual interfaces for off-the-desktop applications.

This paper presents a new technique for visualizing large, complex collections of data. The size and dimensionality of these datasets make them challenging to display in an effective manner. The images must show the global structure of spatial relationships within the dataset, yet at the same time accurately represent the local detail of each data element being visualized. We propose combining ideas from information and scientific visualization together with a navigation assistant, a software system designed to help users identify and explore areas of interest within their data. The assistant locates data elements of potential importance to the user, clusters them into spatial regions, and builds underlying graph structures to connect the regions and the elements they contain. Graph traversal algorithms, constraint-based viewpoint construction, and intelligent camera planning techniques can then be used to design animated tours of these regions. In this way, the navigation assistant can help users to explore any of the areas of interest within their data. We conclude by demonstrating how our assistant is being used to visualize a multidimensional weather dataset.

This paper describes a set of visualization techniques for interactive planning in a physical force simulation called AFS. We have developed a 3D environment in which textures are overlaid on a simulated landscape to convey information about environmental properties, agent actions, and possible strategies. Scenes are presented, via automated camera planning, such that some simple agent goals can be induced visually with little effort. These two areas of visualization functionality in AFS exploit properties of human low-level and intermediate-level vision, respectively. This paper presents AFS, its visualization environment, and studies we have run to explore the relationship between AFS visualizations and the high-level planning process.

The paper describes a novel application of feature preserving mesh simplification to the problem of managing large, multidimensional datasets during scientific visualization. To allow this, we view a scientific dataset as a triangulated mesh of data elements, where the attributes embedded in each element form a set of properties arrayed across the surface of the mesh. Existing simplification techniques were not designed to address the high dimensionality that exists in these types of datasets. In addition, vertex operations that relocate, insert, or remove data elements may need to be modified or restricted. Principal component analysis provides an algorithm-independent method for compressing a dataset's dimensionality during simplification. Vertex locking forces certain data elements to maintain their spatial locations; this technique is also used to guarantee a minimum density in the simplified dataset. The result is a visualization that significantly reduces the number of data elements to display, while at the same time ensuring that high-variance regions of potential interest remain intact. We apply our techniques to a number of well-known feature preserving algorithms, and demonstrate their applicability in a real-world context by simplifying a multidimensional weather dataset. Our results show a significant improvement in execution time with only a small reduction in accuracy; even when the dataset was simplified to 10% of its original size, average per attribute error was less than 1%.

Scientific datasets are often difficult to analyse or visualize, due to their large size and high dimensionality. A multistep approach to address this problem is proposed. Data management techniques are used to identify areas of interest within the dataset. This allows the reduction of a dataset's size and dimensionality, and the estimation of missing values or correction of erroneous entries. The results are displayed using visualization techniques based on perceptual rules. The visualization tools are designed to exploit the power of the low-level human visual system. The result is a set of displays that allow users to perform rapid and accurate exploratory data analysis. In order to demonstrate the techniques, an environmental dataset being used to model salmon growth and migration patterns was visualized. Data mining was used to identify significant attributes and to provide accurate estimates of plankton density. Colour and texture were used to visualize the significant attributes and estimated plankton densities for each month for the years 1956-1964. Experiments run in the laboratory showed that the chosen colours and textures support rapid and accurate element identification, boundary detection, region tracking and estimation. The result is a visualization tool that allows users to quickly locate specific plankton densities and the boundaries they form. Users can compare plankton densities to other environmental conditions like sea surface temperature and current strength. Finally, users can track changes in any of the dataset's attributes on a monthly or yearly basis.

We present a new method for using texture and color to visualize multivariate data elements arranged on an underlying height field. We combine simple texture patterns with perceptually uniform colors to increase the number of attribute values we can display simultaneously. Our technique builds multicolored perceptual texture elements (or pexels) to represent each data element. Attribute values encoded in an element are used to vary the appearance of its pexel. Texture and color patterns that form when the pexels are displayed can be used to rapidly and accurately explore the dataset. Our pexels are built by varying three separate texture dimensions: height, density, and regularity. Results from computer graphics, computer vision, and human visual psychophysics have identified these dimensions as important for the formation of perceptual texture patterns. The pexels are colored using a selection technique that controls color distance, linear separation, and color category. Proper use of these criteria guarantees colors that are equally distinguishable from one another. We describe a set of controlled experiments that demonstrate the effectiveness of our texture dimensions and color selection criteria. We then discuss new work that studies how texture and color can be used simultaneously in a single display.

Presents a new method for using texture to visualize multi-dimensional data elements arranged on an underlying 3D height field. We hope to use simple texture patterns in combination with other visual features like hue and intensity to increase the number of attribute values we can display simultaneously. Our technique builds perceptual texture elements (or pexels) to represent each data element. Attribute values encoded in the data element are used to vary the appearance of a corresponding pexel. Texture patterns that form when the pexels are displayed can be used to rapidly and accurately explore the dataset. Our pexels are built by controlling three separate texture dimensions: height, density and regularity. Results from computer graphics, computer vision and cognitive psychology have identified these dimensions as important for the formation of perceptual texture patterns. We conducted a set of controlled experiments to measure the effectiveness of these dimensions, and to identify any visual interference that may occur when all three are displayed simultaneously at the same spatial location. Results from our experiments show that these dimensions can be used in specific combinations to form perceptual textures for visualizing multidimensional datasets. We demonstrate the effectiveness of our technique by applying it to two real-world visualization environments: tracking typhoon activity in southeast Asia, and analyzing ocean conditions in the northern Pacific.

One well known application area of volume rendering is the reconstruction and visualization of output from medical scanners like computed tomography (CT). 2D greyscale slices produced by these scanners can be reconstructed and displayed onscreen as a 3D model. Volume visualization of medical images must address two important issues. First, it is difficult to segment medical scans into individual materials based only on intensity values. Second, although greyscale images are the normal method for displaying medical volumes, these types of images are not necessarily appropriate for highlighting regions of interest within the volume. Studies of the human visual system have shown that individual intensity values are difficult to detect in a greyscale image. In these situations colour is a more effective visual feature. We addressed both problems during the visualization of CT scans of abdominal aortic aneurysms. We have developed a classification method that empirically segments regions of interest in each of the 2D slices. We use a perceptual colour selection technique to identify each region of interest in both the 2D slices and the 3D reconstructed volumes. The result is a colourized volume that the radiologists are using to rapidly and accurately identify the locations and spatial interactions of different materials from their scans. Our technique is being used in an experimental post operative environment to help to evaluate the results of surgery designed to prevent the rupture of the aneurysm. In the future, we hope to use the technique during the planning of placement of support grafts prior to the actual operation.

We describe a technique for choosing multiple colours for use during data visualization. Our goal is a systematic method for maximizing the total number of colours available for use, while still allowing an observer to rapidly and accurately search a display for any one of the given colours. Previous research suggests that we need to consider three separate effects during colour selection: colour distance, linear separation, and colour category. We describe a simple method for measuring and controlling all of these effects. Our method was tested by performing a set of target identification studies; we analysed the ability of thirty eight observers to find a colour target in displays that contained differently coloured background elements. Results showed our method can be used to select a group of colours that will provide good differentiation between data elements during data visualization.

A new method is presented for performing rapid and accurate numerical estimation. The method is derived from an area of human cognitive psychology called preattentive processing. Preattentive processing refers to an initial organization of the visual field based on cognitive operations believed to be rapid, automatic, and spatially parallel. Examples of visual features that can be detected in this way include hue, intensity, orientation, size, and motion. We beleive that studies from preattentive vision should be used to assist in the design of visualization tools, especially those for which high-speed target detection, boundary identification, and region detection are important. In our present study, we investigated two known preattentive features (hue and orientation) in the context of a new task (numerical estimation) in order to see whether preattentive estimation was possible. Our experiments tested displays that were designed to visualize data from salmon migration simulations. The results showed that rapid and accurate estimation was indeed possible using either hue or orientation. Furthermore, random variation in one of these features resulted in no interference when subjects estimated the percentage of the other. To test the generality of our results, we varied two important display parameters—display duration and feature difference—and found boundary conditions for each. Implications of our results for application to real-world data and tasks are discussed.