Deep reinforcement learning (RL), where a policy represented by a deep neural network is trained, has shown some success in playing video games and chess. However, applying RL to real-world tasks like robot control is still challenging. Because generating a massive number of samples to train control policies using RL on real robots is very expensive, hence impractical, it is common to train in simulations, and then transfer to real environments. The trained policy, however, may fail in the real world because of the difference between the training and the real environments, especially the difference in dynamics. To diagnose the problems, it is crucial for experts to understand (1) how the trained policy behaves under different dynamics settings, (2) which part of the policy affects the behaviors the most when the dynamics setting changes, and (3) how to adjust the training procedure to make the policy robust.This paper presents DynamicsExplorer, a visual analytics tool to diagnose the trained policy on robot control tasks under different dynamics settings. DynamicsExplorer allows experts to overview the results of multiple tests with different dynamics-related parameter settings so experts can visually detect failures and analyze the sensitivity of different parameters. Experts can further examine the internal activations of the policy for selected tests and compare the activations between success and failure tests. Such comparisons help experts form hypotheses about the policy and allows them to verify the hypotheses via DynamicsExplorer. Multiple use cases are presented to demonstrate the utility of DynamicsExplorer.

Development of object detectors for video in applications such as autonomous driving requires an iterative training process with data that initially requires human labeling. Later stages of development require tuning a large set of parameters that may not have labeled data available. For each training iteration and parameter selection decision, insight is needed into object detector performance. This work presents a visualization method called Space-Time Slicing to assist a human developer in the development of object detectors for driving applications without requiring labeled data. Space-Time Slicing reveals patterns in the detection data that can suggest the presence of false positives and false negatives. It may be used to set such parameters as image pixel size in data preprocessing and confidence thresholds for object classifiers by comparing performance across different conditions.

A well-known limitation of classic continuous treemaps is that they generally provide two (or at most a few) visual mappings for data variables apart from the hierarchical relationships. Typically, one variable maps to cell area; another maps to color. However, many data-centric tasks require human users to consider multiple variables simultaneously. The current work introduces the concept of equal-height, variable-width cells in treemaps, which affords the packing of multiple variables into the cell areas of the terminals of the hierarchy. We demonstrate how color and some largely width-invariant graphs can be utilized in the cell areas to add additional visual information in a multi-variate treemap. Examples come from machine learning and from finance applications.

Riffling the pages of a book, perhaps in the search for a specific image, is an example of Rapid Serial Visual Presentation (RSVP). Even at a pace of 10 images per second, successful search is often possible. Interest in RSVP arises because a digital embodiment of RSVP has many applications.There are many possible 'modes' of RSVP. However, a mode can be especially helpful if, after the appearance of an image, and without delaying the arrival of other images, it can remain in view for a second or two to allow a user to confirm that a desired image has been found. Moreover, if a collection of images is presented in such a way as to be perceived as moving in 3D space, it is thought that the search for an individual image can thereby be enhanced by comparison with a 2D presentation.To test this conjecture we devise and use the "Deep-Flat" visual illusion whereby a column of moving images magnifying in size is perceived as approaching the viewer as in a 3D space. When the images are presented in an equivalent way horizontally as a row, the viewer tends to see this as images growing in size, but now on a flat (2D) plane. We tested comparable RSVP designs in these two illusions to ascertain the relative effects of 2D and 3D style presentation under precisely controlled conditions. Elicited data included both performance measures (e.g., recognition success), and user preferences and opinions.We established the effectiveness of RSVP using the illusion. When tested under directly comparable conditions, we concluded that performance is not significantly affected by the illusion of depth, but that the inclusion of certain background cues can have a significantly detrimental effect on performance.

Treemaps have the desirable property of presenting overviews along with details of data and thus are of interest in visualizations of multi-attribute tabular data with attribute hierarchies. However, the original treemap algorithms and most subsequent variations are hampered in making parallel structures in a hierarchical data structure visually comparable. Structurally parallel elements are not aligned, making it difficult to compare them visually. We propose a method that allows for proportional and non-proportional subdivisions of subtrees while preserving visual alignment of parallel structures. We extend the framework so that other types of data visualizations can be placed within the graphical areas of a treemap to allow for the visual comparison of a broad collection of data types including temporal data.

Electrical grid planning aims at optimizing the grid through the control of the performance and placement of electrical assets in order to minimize failures or vulnerabilities. With this purpose, grid planners carry out an initial stage of data exploration using a large volume of incident and equipment data collected over extensive time periods. In current practice these tasks are performed manually, which makes it very difficult to recognize patterns and gain insights into the data. In this paper, we propose a parallel multivariate visualization technique as a suitable approach for improving the existing practice. Based on the usage of an interactive visualization tool called BarExam, we demonstrate the feasibility of this visualization technique for displaying the dataset and present example insights that this visualization technique can provide to grid planners.

Visualization of set-valued attributes in multi-dimensional information visualization systems remains a relatively unexplored problem. Here we introduce a novel method for visualizing set-valued attributes that we call the singleton set distribution view and integrate it into an interactive multi-dimensional attribute visualization tool utilizing parallel bargrams (aka equal-height histograms) as its main visual motif. We discuss our design rationale and report on the results of an evaluation study.

As our ability to measure the world around us improves, we are quickly generating massive quantities of high-dimensional, spatial-temporal data. In this paper, we concern ourselves with datasets in which the spatial characteristics are relatively static but many dimensions prevail and data is sampled over different time periods. Example applications include building energy management and HVAC unit diagnostics. We present methods employed in our Wakame visualization system to support such tasks as discovering anomalies and comparing performance across multiple time series. Novel methods include animated transitions that relate data in spatially located 3D views with conventional 2D graphs. Additionally, several components of our prototype employ analytics to guide the user to "interesting" portions of the dataset.

In this paper, we present an interactive visualization method for set-valued attributes that maintains the advantages of item-oriented views and interactions found in parallel multivariate visualizations such as bargrams (equal-height histograms). The challenge is to accommodate renderings of an item when it appears multiple times in set-valued attribute views while at the same time preserving value- and item-based selection, brushing, and filtering. Such techniques can help users derive particular types of insights into data based on distributions and correlations of attribute values.

This paper describes a long-term project exploring advanced visual interfaces for antenna design. MERL developed three successive prototypes that embodied an evolution towards larger scales and more concrete semantics for visualization of large sets of candidate designs and then winnowing them down. We experimented with multidimensional scaling and then collective line graphs before settling on linked scatterplots to visualize performance in a design space of up to 10 million antennas at a time. In the end, the scatterplot solution was most successful at balancing intelligibility with visualization of the space as a whole. The design allows for adding more 1D or 2D linked feature visualizations if needed, and it smoothly transitions to other "details on demand" views for final tweaking.

The need for effective search for television content is growing as the number of choices for TV viewing and/or recording explodes. In this paper we describe a preliminary prototype of a multimodal Speech-In List-Out (SILO) interface in which users' input is unrestricted by vocabulary or grammar. We report on usability testing with a sample of six users. The prototype enables search through video content metadata downloaded from an electronic program guide (EPG) service. Our setup for testing included adding a microphone to a TV remote control and running an application on a PC whose visual interface was displayed on a TV.

Recent advances in computer video projection open up new possibilities for real-time interactive, persuasive displays. Now a display can continuously adapt to a viewer so as to maximize its effectiveness. However, by the very nature of persuasion, these displays must be both immersive and subtle. We have been working on technologies that support this application including multi-projector and implicit interaction techniques. These technologies have been used to create a series of interactive persuasive displays that are described.

In this paper we propose a new model for a class of rapid serial visual presentation (RSVP) interfaces [16] in the context of consumer video devices. The basic spatial layout explodes a sequence of image frames into a 3D trail in order to provide more context for a spatial/temporal presentation. As the user plays forward or back, the trail advances or recedes while the image in the foreground focus position is replaced. The design is able to incorporate a variety of methods for analyzing or highlighting images in the trail. Our hypotheses are that users can navigate more quickly and precisely to points of interest when compared to conventional consumer-based browsing, channel flipping, or fast-forwarding techniques. We report on an experiment testing our hypotheses in which we found that subjects were more accurate but not faster in browsing to a target of interest in recorded television content with a TV remote.

In this paper we introduce multidimensional visualization and interaction techniques that are an extension to related work in parallel histograms and dynamic querying. Bargrams are, in effect, histograms whose bars have been tipped over and lined up end-to-end. We discuss affordances of parallel bargrams in the context of systems that support consumer-based information exploration and choice based on the attributes of the items in the choice set. Our tool called EZChooser has enabled a number of prototypes in such domains as Internet shopping, investment decisions, college choice, and so on, and a limited version has been deployed for car shopping. Evaluations of the techniques include an experiment indicating that trained users prefer EZChooser over static tables for choice tasks among sets of 50 items with 7-9 attributes.

In this paper we propose new visual interface technology to address multidimensional data exploration and browsing tasks. MultiNav, a prototype from GTE Laboratories, is based upon a multidimensional information model that affords new data exploration and semantically structured browsing interactions. The primary visual metaphor is based on sliding rods, each of which is associated with an information dimension from the underlying model. Users can interactively select value ranges along the rods in order to reveal hidden relationships as well as query and restrict the set through direct manipulation. A novel focus+context view is afforded in which detail about individual items is revealed within the context of the global multidimensional attribute space. We propose a novel interaction technique to change focus, which is based on dragging rods from side to side. We relate this work on multidimensional information visualization to other research in the area, including Parallel Coordinates, Dynamic Histograms, Dynamic Queries, and focus+context tables.*.

In this paper we consider the role of rapid-fire presentation of images in the service of navigation in information spaces. We presume a model of information navigation in which the user performs a cycle of (pre)viewing, selecting, and moving. Our hypothesis is that images presented to the user in rapid succession can significantly enhance the previewing step, thus optimizing the selection step and improving navigability. We discuss two prototypes for navigation tools in Web information spaces in which images are used as the primary means for presenting meta-information about "upcoming " Web pages. The presentation is modeled as a flow of information streaming to the user, and orientation is visualized through positions in ordered sequences. KEYWORDS: Information navigation, visualization, images, previewing INTRODUCTION George Furnas [4] has developed a theory of navigation in information spaces that serves as a useful springboard for positioning research on this sub...

This paper describes an approach to the automatic presentation of multimedia documents based on parsing and syntax-directed translation using Relational Grammars. This translation is followed by a constraint solving mechanism to create the final layout. Grammatical rules provide the mechanism for mapping from a representation of the content of a presentation to forms that specify the media objects to be realized. These realization forms include sets of spatial and temporal constraints between elements of the presentation. Individual grammars encapsulate the "look and feel" of a presentation and can be used as generators of that style. By making the grammars sensitive to the requirements of the output medium, parsing can introduce flexibility into the information realization process. Keywords: Automatic design, grammar-directed design, visual languages, relational grammars, parsing, constraints 2: Introduction A fully functioning multimedia system requires a wide range of stages to achi...

Motivation for including relational constraints other than equality within grammatical formalisms has come from discontinuous constituency and partially free word order for natural languages as well as from the need to define combinatory operations at the most basic level for languages with a two-dimensional syntax (e.g., mathematical notation, chemical equations, and various diagramming languages). This paper presents F-PATR, a generalization of the PATR-II unification-based formalism, which incorporates relational constraints expressed as user-defined functions. An operational semantics is given for unification that is an adaptation and extension of the approach taken by Ait-Kaci and Nasr (1989). It is designed particularly for unification-based formalisms implemented in functional programming environments such as Lisp. The application of unification in a chart parser for relational set languages is discussed briefly.

In this paper, we report on our use of zero morphemes in Unification-Based Combinatory Categorial Grammar. After illustrating the benefits of this approach with several examples, we describe the algorithm for compiling zero morphemes into unary rules, which allows us to use zero morphemes more efficiently in natural language processing. Then, we discuss the question of equivalence of a grammar with these unary rules to the original grammar. Lastly, we compare our approach to zero morphemes with possible alternatives.

Steedman (1985, 1987) and others have proposed that Categorial Grammar, a theory of syntax in which grammatical categories are viewed as functions, be augmented with operators such as functional composition and type raising in order to analyze "noncanonical" syntactic constructions such as wh- extraction and node raising. A consequence of these augmentations is an explosion of semantically equivalent derivations admitted by the grammar. The present work proposes a method for circumventing this spurious ambiguity problem. It involves deriving new, specialized combinators and replacing the orginal basic combinators with these derived ones. In this paper, examples of these predictive combinators are offered and their effects illustrated. An algorithm for deriving them, as well as a discussion of their semantics, will be presented in forthcoming work.