Abstract This article investigates when and how video games enable players to create externalizations in a diverse sample of 388 video games. We follow a grounded‐theory approach, extracting externalizations from video games to explore design ideas and relate them to practices in visualization. Video games often engage players in problem‐solving activities, like solving a murder mystery or optimizing a strategy, requiring players to interpret heterogeneous data—much like tasks in the visualization domain. In many cases, externalizations can help reduce a user's mental load by making tangible what otherwise only lives in their head, acting as external storage or a visual playground. Over five coding phases, we created a hierarchy of 277 tags to describe the video games in our collection, from which we extracted 169 externalizations. We characterize these externalizations along nine dimensions like mental load, visual encodings, and motivations, resulting in 13 categories divided into four clusters: quick access, storage, sensemaking, and communication. We formulate considerations to guide future work, looking at tasks and challenges, naming potentials for inspiration, and discussing which topics could advance the state of externalization.

Abstract Eye tracking provides a unique perspective on the inherently visual discourse between visualisation systems and their users, and has recently become sufficiently precise and affordable to be integrated as regular input into workstations and virtual or augmented reality headsets alike. As such, real‐time eye tracking can now contribute significantly towards the development of gaze‐aware visualisations that infer and monitor users' needs to actively support their activities. To facilitate such systems we make three contributions. First, we structure and discuss design considerations for gaze‐aware visualisations along four axes: measurable data; inferable data; opportunities for support; and limiting factors to beware. Second, we distill visualisation research challenges that preclude such systems. Finally, we show via three usage scenarios how to apply these design considerations to imagine how existing systems can benefit from real‐time eye tracking. We combined a structured literature analysis, a consideration of suitable places for eye‐tracking integration in the typical visualisation ecosystem, and design space modelling. Eye tracking has significant potential to improve the interactive visual analysis of data across many visualisation domains. Our paper attempts to provide a comprehensive, general survey and conceptual discussion in this promising field, outlining the state‐of‐the‐art and future research opportunities.

We present results of a replication study on smartwatch visualizations with adults aged 65 and older. The older adult population is rising globally, coinciding with their increasing interest in using small wearable devices, such as smartwatches, to track and view data. Smartwatches, however, pose challenges to this population: fonts and visualizations are often small and meant to be seen at a glance. How concise design on smartwatches interacts with aging-related changes in perception and cognition, however, is not well understood. We replicate a study that investigated how visualization type and number of data points affect glanceable perception. We observe strong evidence of differences for participants aged 75 and older, sparking interesting questions regarding the study of visualization and older adults. We discuss first steps toward better understanding and supporting an older population of smartwatch wearers and reflect on our experiences working with this population. Supplementary materials are available at https://osf.io/7x4hq/.

TimeSeriesMaker is an open-source application to visually compose time series data in an intuitive and shareable manner. Visualization researchers often use time series data in studies about perceptual or cognitive phenomena and many other contexts. However, finding or generating time series data that fits a given scenario is not always easy. Using a component-based architecture, TimeSeriesMaker allows analysts to compose time series data with complex patterns by combining different components, such as noise, a linear trend or a seasonal pattern. An interactive compositor tree of these components lets analysts explore their combinations using different operators. We support reproducibility and transparency by including functionalities that allow analysts to export and share their configuration, which others can use to reload and modify the same time series. In a qualitative online study with visualization researchers, we found that our approach enables them to create a time series based on an example image or their own requirements. However, system usability could be further improved when interacting with the compositor tree. TimeSeriesMaker can be found here: https://unistuttgart-visus.github.io/time-series-maker/.

With two studies, we assess how different walking trajectories (straight line, circular, and infinity) and speeds (2 km/h, 4 km/h, and 6 km/h) influence the accuracy and response time of participants reading micro visualizations on a smartwatch. We showed our participants common watch face micro visualizations including date, time, weather information, and four complications showing progress charts of fitness data. Our findings suggest that while walking trajectories did not significantly affect reading performance, overall walking activity, especially at high speeds, hurt reading accuracy and, to some extent, response time. Supplemental material is available at: https://osf.io/u78s6/.

We present three studies that investigate the effectiveness of multiple glanceable part-to-whole proportion representations on smartwatch faces. Our goal was to understand how quickly and accurately people can make judgments about their progress toward multiple goals displayed in a small space. We designed our three studies with increasing external validity. The first study compared bar charts, radial bar charts, and text representations—shown with a digital time display. The second study added an analog time dial as a distractor to increase the complexity of the watch face. To emulate realistic viewing conditions, the third study investigated the effect of viewing angles. In Study 1 bar and radial bar charts outperformed text representations, in Study 2 adding an analog time dial as a distractor did not affect task performance, and in Study 3 only the most extreme angle led to some performance decrease. Supplementary material is available at https://osf.io/ad2z7/.

We present the findings of four studies related to the visualization of sleep data on wearables with two form factors: smartwatches and fitness bands. Our goal was to understand the interests, preferences, and effectiveness of different sleep visualizations by form factor. In a survey, we showed that wearers were mostly interested in weekly sleep duration, and nightly sleep phase data. Visualizations of this data were generally preferred over purely text-based representations, and the preferred chart type for fitness bands, and smartwatches was often the same. In one in-person pilot study, and two crowdsourced studies, we then tested the effectiveness of the most preferred representations for different tasks, and found that participants performed simple tasks effectively on both form factors but more complex tasks benefited from the larger smartwatch size. Lastly, we reflect on our crowdsourced study methodology for testing the effectiveness of visualizations for wearables. Supplementary material is available at https://osf.io/yz8ar/.

When visualizations are used to augment text, geographical information on a map, or on mobile devices, there often is not enough space to show complex data with approaches like juxtaposed visualizations or coordinated views. To alleviate this issue, we propose the use of animated transitions between several small-scale visualizations. We discuss design considerations for animated transitions in small-scale visualizations. We further present the results of a study on the effectiveness of those animated transitions in conveying information and attribute relations, and the mental load of following the animated transitions.

In this paper, we present three visualizations showing heart rate (HR) data collected over time. Two visualizations present a summary chart (bar or radial chart), summarizing the amount of time spent per HR zone (i.e., low, moderate, high intensity). We conducted a pilot study with five participants to evaluate the efficiency of the visualizations when monitoring the intensity of an activity while playing a tennis-like Virtual Reality game. Preliminary results show that participants were performing (with respect to time and accuracy) better with and preferred the bar chart summary.

We describe a design space of view manipulation interactions for small data-driven contextual visualizations (word-scale visualizations). These interaction techniques support an active reading experience and engage readers through exploration of embedded visualizations whose placement and content connect them to specific terms in a document. A reader could, for example, use our proposed interaction techniques to explore word-scale visualizations of stock market trends for companies listed in a market overview article. When readers wish to engage more deeply with the data, they can collect, arrange, compare, and navigate the document using the embedded word-scale visualizations, permitting more visualization-centric analyses. We support our design space with a concrete implementation, illustrate it with examples from three application domains, and report results from two experiments. The experiments show how view manipulation interactions helped readers examine embedded visualizations more quickly and with less scrolling and yielded qualitative feedback on usability and future opportunities.

We introduce an approach for the visual analysis of eye movement data from two people playing competitive virtual board games. Our approach provides methods to temporally synchronize and spatially register gaze and mouse recordings from two eye tracking devices. Analysts can examine such fused data visually with a combination of techniques: attention maps and gaze plots as well as a temporal summary of the distance between gaze positions and mouse events of the two players. We show different game scenarios from the competitive game Go, which is especially complex for analyzing strategies of individual players, to demonstrate our methods. In general, our visual analysis approach can provide analysts with insights into strategies, learning processes, and means of communication between people.

In this paper, we analyze eye movement data of 26 participants using a quantitative and qualitative approach to investigate how people read natural language text in comparison to source code. In particular, we use the radial transition graph visualization to explore strategies of participants during these reading tasks and extract common patterns amongst participants. We illustrate via examples how visualization can play a role at uncovering behavior of people while reading natural language text versus source code. Our results show that the linear reading order of natural text is only partially applicable to source code reading. We found patterns representing a linear order and also patterns that represent reading of the source code in execution order. Participants also focus more on those areas that are important to comprehend core functionality and we found that they skip unimportant constructs such as brackets.

This eye tracking study examines participants' visual attention when solving algorithmic problems in the form of programming problems. The stimuli consisted of a problem statement, example output, and a set of multiple-choice questions regarding variables, data types, and operations needed to solve the programming problems. We recorded eye movements of students and performed an Area of Interest (Aol) sequence analysis to identify reading strategies in terms of participants' performance and visual effort. Using classical eye tracking metrics and a visual Aol sequence analysis we identified two main groups of participants---effective and ineffective problem solvers. This indicates that diversity of participants' mental schemas leads to a difference in their performance. Therefore, identifying how participants' reading behavior varies at a finer level of granularity warrants further investigation.

Visual analytics (VA) research provides helpful solutions for interactive visual data analysis when exploring large and complex datasets. Due to recent advances in eye tracking technology, promising opportunities arise to extend these traditional VA approaches. Therefore, we discuss foundations for eye tracking support in VA systems. We first review and discuss the structure and range of typical VA systems. Based on a widely used VA model, we present five comprehensive examples that cover a wide range of usage scenarios. Then, we demonstrate that the VA model can be used to systematically explore how concrete VA systems could be extended with eye tracking, to create supportive and adaptive analytics systems. This allows us to identify general research and application opportunities, and classify them into research themes. In a call for action, we map the road for future research to broaden the use of eye tracking and advance visual analytics.

We present the results of two perception studies to assess how quickly people can perform a simple data comparison task for small-scale visualizations on a smartwatch. The main goal of these studies is to extend our understanding of design constraints for smartwatch visualizations. Previous work has shown that a vast majority of smartwatch interactions last under 5 s. It is still unknown what people can actually perceive from visualizations during such short glances, in particular with such a limited display space of smartwatches. To shed light on this question, we conducted two perception studies that assessed the lower bounds of task time for a simple data comparison task. We tested three chart types common on smartwatches: bar charts, donut charts, and radial bar charts with three different data sizes: 7, 12, and 24 data values. In our first study, we controlled the differences of the two target bars to be compared, while the second study varied the difference randomly. For both studies, we found that participants performed the task on average in <300 ms for the bar chart, <220 ms for the donut chart, and in <1780 ms for the radial bar chart. Thresholds in the second study per chart type were on average 1.14-1.35× higher than in the first study. Our results show that bar and donut charts should be preferred on smartwatch displays when quick data comparisons are necessary.

We investigate how people discover the functionality of an interactive visualization that was designed for the general public. While interactive visualizations are increasingly available for public use, we still know little about how the general public discovers what they can do with these visualizations and what interactions are available. Developing a better understanding of this discovery process can help inform the design of visualizations for the general public, which in turn can help make data more accessible. To unpack this problem, we conducted a lab study in which participants were free to use their own methods to discover the functionality of a connected set of interactive visualizations of public energy data. We collected eye movement data and interaction logs as well as video and audio recordings. By analyzing this combined data, we extract exploration strategies that the participants employed to discover the functionality in these interactive visualizations. These exploration strategies illuminate possible design directions for improving the discoverability of a visualization's functionality.

In this paper, we contribute an eye tracking study conducted with pedestrians and cyclists. We apply a visual analytics-based method to inspect pedestrians' and cyclists' gaze behavior as well as video recordings and accelerometer data. This method using multi-modal data allows us to explore patterns and extract common eye movement strategies. Our results are that participants paid most attention to the path itself; advertisements do not distract participants; participants focus more on pedestrians than on cyclists; pedestrians perform more shoulder checks than cyclists do; and we extracted common gaze sequences. Such an experiment in a real-world traffic environment allows us to understand realistic behavior of pedestrians and cyclists better.

In this paper, we contribute an eye tracking study conducted with pedestrians and cyclists. We apply a visual analytics-based method to inspect pedestrians' and cyclists' gaze behavior as well as video recordings and accelerometer data. This method using multi-modal data allows us to explore patterns and extract common eye movement strategies. Our results are that participants paid most attention to the path itself; advertisements do not distract participants; participants focus more on pedestrians than on cyclists; pedestrians perform more shoulder checks than cyclists do; and we extracted common gaze sequences. Such an experiment in a real-world traffic environment allows us to understand realistic behavior of pedestrians and cyclists better.

Abstract This survey provides an introduction into eye tracking visualization with an overview of existing techniques. Eye tracking is important for evaluating user behaviour. Analysing eye tracking data is typically done quantitatively, applying statistical methods. However, in recent years, researchers have been increasingly using qualitative and exploratory analysis methods based on visualization techniques. For this state‐of‐the‐art report, we investigated about 110 research papers presenting visualization techniques for eye tracking data. We classified these visualization techniques and identified two main categories: point‐based methods and methods based on areas of interest. Additionally, we conducted an expert review asking leading eye tracking experts how they apply visualization techniques in their analysis of eye tracking data. Based on the experts' feedback, we identified challenges that have to be tackled in the future so that visualizations will become even more widely applied in eye tracking research.

Abstract In eye tracking research, finding eye movement patterns and similar strategies between participants’ eye movements is important to understand task solving strategies and obstacles. In this application paper, we present a graph comparison method using radial graphs that show Areas of Interest (AOIs) and their transitions. An analyst investigates a single graph based on dwell times, directed transitions, and temporal AOI sequences. Two graphs can be compared directly and temporal changes may be analyzed. A list and matrix approach facilitate the analyst to contrast more than two graphs guided by visually encoded graph similarities. We evaluated our approach in case studies with three eye tracking and visualization experts. They identified temporal transition patterns of eye movements across participants, groups of participants, and outliers.

In information visualization, evaluation plays a crucial role during the development of a new visualization technique. In recent years, eye tracking has become one means to analyze how users perceive and understand a new visualization system. Since most visualizations are highly interactive nowadays, a study should take interaction, in terms of user-input, into account as well. In addition, think aloud data gives insights into cognitive processes of participants using a visualization system. Typically, researchers evaluate these data sources separately. However, we think it is beneficial to correlate eye tracking, interaction, and think aloud data for deeper analyses. In this paper, we present challenges and possible solutions in triangulating user behavior using multiple evaluation data sources. We describe how the data is collected, synchronized, and analyzed using a string-based and a visualization-based approach founded on experiences from our current research. We suggest methods how to tackle these issues and discuss benefits and disadvantages. Thus, the contribution of our work is twofold. On the one hand, we present our approach and the experiences we gained during our research. On the other hand, we investigate additional methods that can be used to analyze this multi-source data.

In eye tracking studies a complex visual stimulus requires the definition of many areas of interest (AOIs). Often these AOIs have an inherent, nested hierarchical structure that can be utilized to facilitate analysis tasks. We discuss how this hierarchical AOI structure in combination with appropriate visualization techniques can be applied to analyze fixation sequences on differently aggregated levels. An AOI View, AOI Tree, AOI Matrix, and AOI Graph enable a bottom-up and top-down evaluation of fixation sequences. We conducted an expert review and compared our techniques to current state-of-the-art visualization techniques in eye movement research to further improve and extend our approach. To show how our approach is used in practice, we evaluate fixation sequences collected during a study where 101 AOIs are organized hierarchically.

Evaluation has become a fundamental part of visualization research and researchers have employed many approaches from the field of human-computer interaction like measures of task performance, thinking aloud protocols, and analysis of interaction logs. Recently, eye tracking has also become popular to analyze visual strategies of users in this context. This has added another modality and more data, which requires special visualization techniques to analyze this data. However, only few approaches exist that aim at an integrated analysis of multiple concurrent evaluation procedures. The variety, complexity, and sheer amount of such coupled multi-source data streams require a visual analytics approach. Our approach provides a highly interactive visualization environment to display and analyze thinking aloud, interaction, and eye movement data in close relation. Automatic pattern finding algorithms allow an efficient exploratory search and support the reasoning process to derive common eye-interaction-thinking patterns between participants. In addition, our tool equips researchers with mechanisms for searching and verifying expected usage patterns. We apply our approach to a user study involving a visual analytics application and we discuss insights gained from this joint analysis. We anticipate our approach to be applicable to other combinations of evaluation techniques and a broad class of visualization applications.

Recording of eye movement data can help to understand where and at what participants look. However, analyzing eye movement data is a time consuming task. Using visualization techniques in the analysis process can help to uncover concealed relationships within the data and can therefore be seen as one means in the analysis of eye movement data. The most well known visualization techniques in eye tracking are heat maps or scanpaths. In recent years more visualization techniques have been developed, as for example scaled traces [Goldberg and Helfman 2010], eyePatterns [West et al. 2006], or eSeeTrack [Tsang et al. 2010].

During eye tracking studies, vast amounts of spatio-temporal data in the form of eye gaze trajectories are recorded. Finding insights into these time-varying data sets is a challenging task. Visualization techniques such as heat maps or gaze plots help find patterns in the data but highly aggregate the data (heat maps) or are difficult to read due to overplotting (gaze plots). In this paper, we propose transforming eye movement data into a dynamic graph data structure to explore the visualization problem from a new perspective. By aggregating gaze trajectories of participants over time periods or Areas of Interest (AOIs), a fair trade-off between aggregation and details is achieved. We show that existing dynamic graph visualizations can be used to display the transformed data and illustrate the approach by applying it to eye tracking data recorded for investigating the readability of tree diagrams.

Several algorithms, approaches, and implementations have been developed to support comparison of scan paths and finding of interesting scan path structures. In this work we contribute a visual approach to support scan path comparison. A key feature of this approach is the combination of a clustering algorithm using Levenshtein distance with the parallel scan path visualization technique. The combination of computational methods with an interactive visualization allows us to use both the power of pattern finding algorithms and the human ability to visually recognize patterns. To use the concept in practice we implemented the approach in a prototype and show its application in two scan path analysis scenarios from automobile usability testing and visualization research.

Several algorithms, approaches, and implementations have been developed to support comparison of scan paths and finding of interesting scan path structures. In this work we contribute a visual approach to support scan path comparison. A key feature of this approach is the combination of a clustering algorithm using Levenshtein distance with the parallel scan path visualization technique. The combination of computational methods with an interactive visualization allows us to use both the power of pattern finding algorithms and the human ability to visually recognize patterns. To use the concept in practice we implemented the approach in a prototype and show its application in two scan path analysis scenarios from automobile usability testing and visualization research.