Many potential benefits of data physicalizations are thought to stem from their tangible nature and their presence in a shared space, which allows for physical interaction in a social environment. Prior research has studied where observers touch data physicalizations and how these touches depend on task. However, limited research explores the moment-by-moment details of touches, gestures, and verbal dialogue and how these interactions contribute to the larger data physicalization sensemaking process. This case study offers a new data analysis from a previous study to provide fine-grained accounting of one participant’s touches, gestures, and conversations around a data object. We identify four types of touches and gestures and describe relational patterns between individual interactions. This work provides a foundation for further exploring the reasons people touch or gesture with data physicalizations, connecting these efforts with gesture studies research, and identifying design implications for data physicalization.

International audience

This article explores how the ability to recall information in data visualizations depends on the presentation technology. Participants viewed 10 Isotype visualizations on a 2D screen, in 3D, in Virtual Reality (VR) and in Mixed Reality (MR). To provide a fair comparison between the three 3D conditions, we used LIDAR to capture the details of the physical rooms, and used this information to create our textured 3D models. For all environments, we measured the number of visualizations recalled and their order (2D) or spatial location (3D, VR, MR). We also measured the number of syntactic and semantic features recalled. Results of our study show increased recall and greater richness of data understanding in the MR condition. Not only did participants recall more visualizations and ordinal/spatial positions in MR, but they also remembered more details about graph axes and data mappings, and more information about the shape of the data. We discuss how differences in the spatial and kinesthetic cues provided in these different environments could contribute to these results, and reasons why we did not observe comparable performance in the 3D and VR conditions.

Traces of touch provide valuable insight into how we interact with the physical world. Measuring touch behavior, however, is expensive and imprecise. Utilizing a fluorescent UV tracer powder, we developed a low-cost analog method to capture persistent, high-contrast touch records on arbitrary objects. We describe our process for selecting a tracer, methods for capturing, enhancing, and aggregating traces, and approaches to examining qualitative aspects of the user experience. Three user studies demonstrate key features of this method. First, we show that it provides clear and durable traces on objects representative of scientific visualization, physicalization, and product design. Second, we demonstrate how this method could be used to study touch perception, by measuring how task and narrative framing elicit different touch behaviors on the same object. Third, we demonstrate how this method can be used to evaluate data physicalizations by observing how participants touch two different physicalizations of COVID-19 time-series data.

Geographical maps encoded with rainbow color scales are widely used by climate scientists. Despite a plethora of evidence from the visualization and vision sciences literature about the shortcomings of the rainbow color scale, they continue to be preferred over perceptually optimal alternatives. To study and analyze this mismatch between theory and practice, we present a web-based user study that compares the effect of color scales on performance accuracy for climate-modeling tasks. In this study, we used pairs of continuous geographical maps generated using climatological metrics for quantifying pairwise magnitude difference and spatial similarity. For each pair of maps, 39 scientist-observers judged: i) the magnitude of their difference, ii) their degree of spatial similarity, and iii) the region of greatest dissimilarity between them. Besides the rainbow color scale, two other continuous color scales were chosen such that all three of them covaried two dimensions (luminance monotonicity and hue banding), hypothesized to have an impact on task performance. We also analyzed subjective performance measures, such as user confidence, perceived accuracy, preference, and familiarity in using the different color scales. We found that monotonic luminance scales produced significantly more accurate judgments of magnitude difference but were not superior in spatial comparison tasks, and that hue banding had differential effects based on the task and conditions. Scientists expressed the highest preference and perceived confidence and accuracy with the rainbow, despite its poor performance on the magnitude comparison tasks. We also report on interesting interactions among stimulus conditions, tasks, and color scales, that lead to open research questions.

Many graph layout algorithms optimize visual characteristics to achieve useful representations. Implicitly, their goal is to create visual representations that are more intuitive to human observers. In this paper, we asked users to explicitly manipulate nodes in a network diagram to create layouts that they felt best captured the relationships in the data. This allowed us to measure organizational behavior directly, allowing us to evaluate the perceptual importance of particular visual features, such as edge crossings and edge-lengths uniformity. We also manipulated the interior structure of the node relationships by designing data sets that contained clusters, that is, sets of nodes that are strongly interconnected. By varying the degree to which these clusters were "masked" by extraneous edges we were able to measure observers' sensitivity to the existence of clusters and how they revealed them in the network diagram. Based on these measurements we found that observers are able to recover cluster structure, that the distance between clusters is inversely related to the strength of the clustering, and that users exhibit the tendency to use edges to visually delineate perceptual groups. These results demonstrate the role of perceptual organization in representing graph data and provide concrete recommendations for graph layout algorithms.

We propose a new approach for image segmentation that is based on low-level features for color and texture. It is aimed at segmentation of natural scenes, in which the color and texture of each segment does not typically exhibit uniform statistical characteristics. The proposed approach combines knowledge of human perception with an understanding of signal characteristics in order to segment natural scenes into perceptually/semantically uniform regions. The proposed approach is based on two types of spatially adaptive low-level features. The first describes the local color composition in terms of spatially adaptive dominant colors, and the second describes the spatial characteristics of the grayscale component of the texture. Together, they provide a simple and effective characterization of texture that the proposed algorithm uses to obtain robust and, at the same time, accurate and precise segmentations. The resulting segmentations convey semantic information that can be used for content-based retrieval. The performance of the proposed algorithms is demonstrated in the domain of photographic images, including low-resolution, degraded, and compressed images.

We present a perceptually-tuned multiscale image segmentation algorithm that is based on spatially adaptive color and texture features. The proposed algorithm extends a previously proposed approach to include multiple texture scales. The determination of the multiscale texture features is based on perceptual considerations. We also examine the perceptual tuning of the algorithm and how it is affected by the presence of different texture scales. The multiscale extension is necessary for segmenting higher resolution images and is particularly effective in segmenting objects shown in different perspectives. The performance of the proposed algorithm is demonstrated in the domain of photographic images.

We propose a method for semantic categorization and retrieval of photographic images based on low-level image descriptors derived from perceptual experiments. The method applies multidimensional scaling and hierarchical clustering analysis to identify candidate semantic categories into which human observers organize images. Through a series of subjective experiments we refine our definition of these categories and select a set of low-level image features that uniquely describe them. We then devise a new image similarity metric and develop a prototype system, which identifies the semantic category of the image and retrieves similar images from the database. We tested the metric on a set of new images and compared the categorization results with that of human observers. Our results provide a good match to human performance, thus validating the use of human judgments to develop semantic descriptors. Our method can be used for the enhancement of current retrieval methods, better organization of image/video databases, and the development of more intuitive navigation schemes, browsing methods and user interfaces.

An image segmentation algorithm that is based on spatially adaptive color and texture features is presented. The proposed algorithm is based on a previously proposed algorithm but introduces a number of new elements. We use a new set of texture features based on a steerable filter decomposition. The steerable filters combined with a new spatial texture segmentation scheme provide a finer and more robust segmentation into texture classes. The proposed algorithm includes an elaborate border estimation procedure, which extends the idea of Pappas (1992) adaptive clustering segmentation algorithm to color texture. The performance of the proposed algorithm is demonstrated in the domain of photographic images, including low resolution compressed images.

We propose an image segmentation algorithm that is based on spatially adaptive color and texture features. The features are first developed independently, and then combined to obtain an overall segmentation. Texture feature estimation requires a finite neighborhood which limits the spatial resolution of texture segmentation, while color segmentation provides accurate and precise edge localization. We combine a previously proposed adaptive clustering algorithm for color segmentation with a simple but effective texture segmentation approach to obtain an overall image segmentation. Our focus is in the domain of photographic images with an essentially unlimited range of topics. The images are assumed to be of relatively low resolution and may be degraded or compressed.

We propose a method for semantic categorization and retrieval of photographic images based on low-level image descriptors. In this method, we first use multidimensional scaling (MDS) and hierarchical cluster analysis (HCA) to model the semantic categories into which human observers organize images. Through a series of psychophysical experiments and analyses, we refine our definition of these semantic categories, and use these results to discover a set of low-level image features to describe each category. We then devise an image similarity metric that embodies our results, and develop a prototype system, which identifies the semantic category of the image and retrieves the most similar images from the database. We tested the metric on a new set of images, and compared the categorization results with that of human observers. Our results provide a good match to human performance, thus validating the use of human judgments to develop semantic descriptors.

We have developed a fast, perceptual method for selecting color scales for data visualization that takes advantage of our sensitivity to luminance variations in human faces. To do so, we conducted experiments in which we mapped various color scales onto the intensity values of a digitized photograph of a face and asked observers to rate each image. We found a very strong correlation between the perceived naturalness of the images and the degree to which the underlying color scales increased monotonically in luminance. Color scales that did not include a monotonically increasing luminance component produced no positive rating scores. Since color scales with monotonic luminance profiles are widely recommended for visualizing continuous scalar data, a purely visual technique for identifying such color scales could be very useful, especially in situations where color calibration is not integrated into the visualization environment, such as over the Internet.

WEAVE (Workbench Environment for Analysis and Visual Exploration) is an environment for creating interactive visualization applications. WEAVE differs from previous systems in that it provides transparent linking between custom 3D visualizations and multidimensional statistical representations, and provides interactive color brushing between all visualizations. The authors demonstrate how WEAVE can be used to rapidly prototype a biomedical application, weaving together simulation data, measurement data, and 3D anatomical data concerning the propagation of excitation in the heart. These linked statistical and custom three-dimensional visualizations of the heart can allow scientists to more effectively study the correspondence of structure and behavior.

Analyzing options is a complex, multi-variate process. Option behavior depends on a variety of market conditions which vary over the time course of the option. The goal of this project is to provide an interactive visual environment which allows the analyst to explore these complex interactions, and to select and construct specific views for communicating information to non-analysts (e.g., marketing managers and customers). In this paper we describe an environment for exploring 2- and 3-dimensional representations of options data, dynamically varying parameters, examining how multi-variate relationships develop over time, and exploring the likelihood of the development of different outcomes over the life of the option. We also demonstrate how this tool has been used by analysts to communicate to non-analysts how particular options no longer deliver the behavior they were originally intended to provide.

The paper presents an interactive approach for guiding the user's select of colormaps in visualization. PRAVDAColor, implemented as a module in the IBM Visualization Data Explorer, provides the user a selection of appropriate colormaps given the data type and spatial frequency, the user's task, and properties of the human perceptual system.

In Rogowitz and Treinish (1993), we introduced an architecture for incorporating perceptual rules into the visualization process. In this architecture, higher-level descriptors of the data, metadata, flow to perceptual rules, which constrain visualization operations. In this paper, we develop a deeper analysis of the rules, the prerequisite metadata, and the system for enabling their operation.>