We introduce a temporal arrangement framework for embedding visualizations in sports videos with a focus on swimming. Our work is inspired by strategies used in current TV broadcasts, where visualizations are selectively displayed to provide meaningful and engaging information to audiences. We began with a systematic review of TV broadcast practices, through which we identified recurring temporal combinations of visualizations and competition statuses, which we define as patterns of temporal arrangement for embedded visualizations. To move beyond the constraints of existing broadcast practices, we then conducted a formative study with a general population. Based on this broader perspective, we designed a configuration framework that allows us to formally specify when and for how long, related to swimming context metadata, visualizations appear in a video. We instantiate the framework in a technology probe, SwimChrono, for applications with real-world swimming context videos. Through audience-customized configurations, SwimChrono supports novel arrangements beyond those used in existing professional settings, is adaptable to various swimming contexts, including different lengths and swimming styles, and key events. Furthermore, we conduct user studies and contribute use cases to illustrate how our framework can be well applied for diverse needs.

Abstract Shot maps are popular in many sports as they typically plot events and player positions in the way they are collected, using a pitch or a table as an absolute coordinate system. We introduce a variation of a table tennis shot map that shifts the point of view from the table to the player. This results in a new reference system to plot incoming balls relative to the player's position rather than on the table. This approach aligns with how table tennis tactical analysis is conducted, focusing on identifying empty spaces and weak spots around the players. We describe the motivation behind this work, built through close collaboration with two table tennis experts, and demonstrate how this approach aligns with the way they analyze games to reveal key tactical aspects. We also present the design rationale and the computer vision pipeline used to accurately collect data from broadcast videos. Our findings show that the technique enables capturing insights that were not visible with the absolute coordinate system, particularly in understanding regions that are reachable and those close to the pivot area of the player.

We report on challenges and considerations for supporting design processes for visualizations in motion embedded in sports videos. We derive our insights from analyzing swimming race visualizations and motion-related data, building a technology probe, as well as a study with designers. Understanding how to design situated visualizations in motion is important for a variety of contexts. Competitive sports coverage, in particular, increasingly includes information on athlete or team statistics and records. Although moving visual representations attached to athletes or other targets are starting to appear, systematic investigations on how to best support their design process in the context of sports videos are still missing. Our work makes several contributions in identifying opportunities for visualizations to be added to swimming competition coverage but, most importantly, in identifying requirements and challenges for designing situated visualizations in motion. Our investigations include the analysis of a survey with swimming enthusiasts on their motion-related information needs, an ideation workshop to collect designs and elicit design challenges, the design of a technology probe that allows to create embedded visualizations in motion based on real data (Fig. 1), and an evaluation with visualization designers that aimed to understand the benefits of designing directly on videos.

Automatic sports field registration aims at projecting a given image taken with unknown camera parameters to a known 3D coordinate system in order to obtain higher-level information like the position and speed of players. Existing methods generally detect specific visual landmarks on the field and then use an iterative refinement to get closer to the desired calibration. They are usually only compared in terms of precision on a standard benchmark without considering other metrics. However, execution speed is also important, mainly in the context of live broadcast TV and sports analysis. This work introduces a new automatic field registration method achieving excellent performance on the WorldCup Soccer benchmark, while neither depending on specific visible landmarks nor any refinement, resulting in a very high execution speed one-shot model. Finally, to complement the usual Soccer benchmark, we introduce a new Swimming Pool registration benchmark which is more challenging for the task at hand. Code and dataset available at https://github.com/njacquelin/sportsfieldregistration.

Visual Question Answering systems target answering open-ended textual questions given input images. They are a testbed for learning high-level reasoning with a primary use in HCI, for instance assistance for the visually impaired. Recent research has shown that state-of-the-art models tend to produce answers exploiting biases and shortcuts in the training data, and sometimes do not even look at the input image, instead of performing the required reasoning steps. We present VisQA, a visual analytics tool that explores this question of reasoning vs. bias exploitation. It exposes the key element of state-of-the-art neural models - attention maps in transformers. Our working hypothesis is that reasoning steps leading to model predictions are observable from attention distributions, which are particularly useful for visualization. The design process of VisQA was motivated by well-known bias examples from the fields of deep learning and vision-language reasoning and evaluated in two ways. First, as a result of a collaboration of three fields, machine learning, vision and language reasoning, and data analytics, the work lead to a better understanding of bias exploitation of neural models for VQA, which eventually resulted in an impact on its design and training through the proposition of a method for the transfer of reasoning patterns from an oracle model. Second, we also report on the design of VisQA, and a goal-oriented evaluation of VisQA targeting the analysis of a model decision process from multiple experts, providing evidence that it makes the inner workings of models accessible to users.

We contribute a research agenda for visualization in motion and two experiments to understand how well viewers can read data from moving visualizations. We define visualizations in motion as visual data representations that are used in contexts that exhibit relative motion between a viewer and an entire visualization. Sports analytics, video games, wearable devices, or data physicalizations are example contexts that involve different types of relative motion between a viewer and a visualization. To analyze the opportunities and challenges for designing visualization in motion, we show example scenarios and outline a first research agenda. Motivated primarily by the prevalence of and opportunities for visualizations in sports and video games we started to investigate a small aspect of our research agenda: the impact of two important characteristics of motion-speed and trajectory on a stationary viewer's ability to read data from moving donut and bar charts. We found that increasing speed and trajectory complexity did negatively affect the accuracy of reading values from the charts and that bar charts were more negatively impacted. In practice, however, this impact was small: both charts were still read fairly accurately.

Since its inception, Visual Question Answering (VQA) is notoriously known as a task, where models are prone to exploit biases in datasets to find shortcuts instead of performing high-level reasoning. Classical methods address this by removing biases from training data, or adding branches to models to detect and remove biases. In this paper, we argue that uncertainty in vision is a dominating factor preventing the successful learning of reasoning in vision and language problems. We train a visual oracle and in a large scale study provide experimental evidence that it is much less prone to exploiting spurious dataset biases compared to standard models. We propose to study the attention mechanisms at work in the visual oracle and compare them with a SOTA Transformer-based model. We provide an in-depth analysis and visualizations of reasoning patterns obtained with an online visualization tool which we make publicly available1. We exploit these insights by transferring reasoning patterns from the oracle to a SOTA Transformer-based VQA model taking standard noisy visual inputs via fine-tuning. In experiments we report higher overall accuracy, as well as accuracy on infrequent answers for each question type, which provides evidence for improved generalization and a decrease of the dependency on dataset biases.

Abstract We propose to take an artifact‐centric approach to design studies by leveraging the concept of boundary object. Design studies typically focus on processes and articulate design decisions in a project‐specific context with a goal of transferability. We argue that design studies could benefit from paying attention to the material conditions in which teams collaborate to reach design outcomes. We report on a design study of isochrone maps following cartographic generalization principles. Focusing on boundary objects enables us to characterize five categories of artifacts and tools that facilitated collaboration between actors involved in the design process (structured collections, structuring artifacts, process‐centric artifacts, generative artifacts, and bridging artifacts). We found that artifacts such as layered maps and map collections played a unifying role for our inter‐disciplinary team. We discuss how such artifacts can be pivotal in the design process. Finally, we discuss how considering boundary objects could improve the transferability of design study results, and support reflection on inter‐disciplinary collaboration in the domain of Information Visualization.

We present ReViVD, a tool for exploring and filtering large trajectory-based datasets using virtual reality. ReViVD’s novelty lies in using simple 3D shapes—such as cuboids, spheres and cylinders—as queries for users to select and filter groups of trajectories. Building on this simple paradigm, more complex queries can be created by combining previously made selection groups through a system of user-created Boolean operations. We demonstrate the use of ReViVD in different application domains, from GPS position tracking to simulated data (e. g., turbulent particle flows and traffic simulation). Our results show the ease of use and expressiveness of the 3D geometric shapes in a broad range of exploratory tasks. Re- ViVD was found to be particularly useful for progressively refining selections to isolate outlying behaviors. It also acts as a powerful communication tool for conveying the structure of normally abstract datasets to an audience.

Abstract We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co‐correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explain‐ability in the field of visual analytics.

Visualizing 3D trajectories to extract insights about their similarities and spatial configuration is a critical task in several domains. Air traffic controllers for example deal with large quantities of aircrafts routes to optimize safety in airspace and neuroscientists attempt to understand neuronal pathways in the human brain by visualizing bundles of fibers from DTI images. Extracting insights from masses of 3D trajectories is challenging as the multiple three dimensional lines have complex geometries, may overlap, cross or even merge with each other, making it impossible to follow individual ones in dense areas. As trajectories are inherently spatial and three dimensional, we propose FiberClay: a system to display and interact with 3D trajectories in immersive environments. FiberClay renders a large quantity of trajectories in real time using GP-GPU techniques. FiberClay also introduces a new set of interactive techniques for composing complex queries in 3D space leveraging immersive environment controllers and user position. These techniques enable an analyst to select and compare sets of trajectories with specific geometries and data properties. We conclude by discussing insights found using FiberClay with domain experts in air traffic control and neurology.

Abstract In this report, we organize and reflect on recent advances and challenges in the field of sports data visualization. The exponentially‐growing body of visualization research based on sports data is a prime indication of the importance and timeliness of this report. Sports data visualization research encompasses the breadth of visualization tasks and goals: exploring the design of new visualization techniques; adapting existing visualizations to a novel domain; and conducting design studies and evaluations in close collaboration with experts, including practitioners, enthusiasts, and journalists. Frequently this research has impact beyond sports in both academia and in industry because it is i) grounded in realistic, highly heterogeneous data, ii) applied to real‐world problems, and iii) designed in close collaboration with domain experts. In this report, we analyze current research contributions through the lens of three categories of sports data: box score data (data containing statistical summaries of a sport event such as a game), tracking data (data about in‐game actions and trajectories), and meta‐data (data about the sport and its participants but not necessarily a given game). We conclude this report with a high‐level discussion of sports visualization research informed by our analysis—identifying critical research gaps and valuable opportunities for the visualization community. More information is available at the STAR's website: https://sportsdataviz.github.io/ .

Mock-ups are rapid, low fidelity prototypes, that are used in many design-related fields to generate and share ideas. While their creation is supported by many mature methods and tools, surprisingly few are suited for the needs of information visualization. In this article, we introduce a novel approach to creating visualizations mock-ups, based on a dialogue between graphic design and parametric toolkit explorations. Our approach consists in iteratively subdividing the display space, while progressively informing each division with realistic data. We show that a wealth of mock-ups can easily be created using only temporary data attributes, as we wait for more realistic data to become available. We describe the implementation of this approach in a D3-based toolkit, which we use to highlight its generative power, and we discuss the potential for transitioning towards higher fidelity prototypes.

We introduce a novel time navigation technique to update ranking tables by direct manipulation. The technique allows users to drag a table's cells to change the time period, while a line chart overlays on top of the table to provide an overview of the changes. The line chart is also a visual hint to control the pace at which data are updated. We explore the design and usability of this technique for table variations in size, time spans and data variability. We report the results of a usability study, using academic citation rankings and economic complexity datasets, and discuss design implications coming with real-world scenarios such as missing data and affordance.

This article introduces A Table!, an enhanced soccer ranking table providing temporal navigation by combining two novel interaction techniques. Ranking tables order soccer teams represented as rows, according to values of columns containing attributes e.g., accumulated points, or number of scored goals. Because they represent a snapshot of a championship at a time t, tables are regularly updated with new results. Such updates usually change the rows order, which makes the tracking of a specified team over time difficult. We observed that the tables available on the web do not support tracking such changes very well, are generally hard to read, and lack interactions. This contrasts with the extensive use of comments on temporal trends found in soccer analysts articles. To better support such analyzes, the two interactive techniques presented allow exploration of time, and are designed to preserve users' flow: DRAG-CELL is based on direct manipulation of values to browse ranks; VIZ-RANK uses a transient line chart of team ranks to visually explore a championship. An on-line evaluation with 143 participants shows that each technique efficiently supports a set of important temporal tasks not supported by current ranking tables. This paves the way for introducing efficient advanced visual exploration techniques to millions of soccer enthusiasts who use tables everyday.

Understanding relationships between sets is an important analysis task that has received widespread attention in the visualization community. The major challenge in this context is the combinatorial explosion of the number of set intersections if the number of sets exceeds a trivial threshold. In this paper we introduce UpSet, a novel visualization technique for the quantitative analysis of sets, their intersections, and aggregates of intersections. UpSet is focused on creating task-driven aggregates, communicating the size and properties of aggregates and intersections, and a duality between the visualization of the elements in a dataset and their set membership. UpSet visualizes set intersections in a matrix layout and introduces aggregates based on groupings and queries. The matrix layout enables the effective representation of associated data, such as the number of elements in the aggregates and intersections, as well as additional summary statistics derived from subset or element attributes. Sorting according to various measures enables a task-driven analysis of relevant intersections and aggregates. The elements represented in the sets and their associated attributes are visualized in a separate view. Queries based on containment in specific intersections, aggregates or driven by attribute filters are propagated between both views. We also introduce several advanced visual encodings and interaction methods to overcome the problems of varying scales and to address scalability. UpSet is web-based and open source. We demonstrate its general utility in multiple use cases from various domains.

From financial statistics to nutritional values, we are frequently exposed to quantitative information expressed in measures of either extreme magnitudes or unfamiliar units, or both. A common practice used to comprehend such complex measures is to relate, re-express, and compare them through visual depictions using magnitudes and units that are easier to grasp. Through this practice, we create a new graphic composition that we refer to as a concrete scale. To the best of our knowledge, there are no design guidelines that exist for concrete scales despite their common use in communication, educational, and decision-making settings. We attempt to fill this void by introducing a novel framework that would serve as a practical guide for their analysis and design. Informed by a thorough analysis of graphic compositions involving complex measures and an extensive literature review of scale cognition mechanisms, our framework outlines the design space of various measure relations--specifically relations involving the re-expression of complex measures to more familiar concepts--and their visual representations as graphic compositions.

We introduce Visual Sedimentation, a novel design metaphor for visualizing data streams directly inspired by the physical process of sedimentation. Visualizing data streams (e. g., Tweets, RSS, Emails) is challenging as incoming data arrive at unpredictable rates and have to remain readable. For data streams, clearly expressing chronological order while avoiding clutter, and keeping aging data visible, are important. The metaphor is drawn from the real-world sedimentation processes: objects fall due to gravity, and aggregate into strata over time. Inspired by this metaphor, data is visually depicted as falling objects using a force model to land on a surface, aggregating into strata over time. In this paper, we discuss how this metaphor addresses the specific challenge of smoothing the transition between incoming and aging data. We describe the metaphor's design space, a toolkit developed to facilitate its implementation, and example applications to a range of case studies. We then explore the generative capabilities of the design space through our toolkit. We finally illustrate creative extensions of the metaphor when applied to real streams of data.

This article presents SoccerStories, a visualization interface to support analysts in exploring soccer data and communicating interesting insights. Currently, most analyses on such data relate to statistics on individual players or teams. However, soccer analysts we collaborated with consider that quantitative analysis alone does not convey the right picture of the game, as context, player positions and phases of player actions are the most relevant aspects. We designed SoccerStories to support the current practice of soccer analysts and to enrich it, both in the analysis and communication stages. Our system provides an overview+detail interface of game phases, and their aggregation into a series of connected visualizations, each visualization being tailored for actions such as a series of passes or a goal attempt. To evaluate our tool, we ran two qualitative user studies on recent games using SoccerStories with data from one of the world's leading live sports data providers. The first study resulted in a series of four articles on soccer tactics, by a tactics analyst, who said he would not have been able to write these otherwise. The second study consisted in an exploratory follow-up to investigate design alternatives for embedding soccer phases into word-sized graphics. For both experiments, we received a very enthusiastic feedback and participants consider further use of SoccerStories to enhance their current workflow.

Information Visualization is a challenging field, enabling a better use of humans' visual and cognitive system, to make sense of very large datasets. This paper aims at improving the current Information Visualizations design workflow, by enabling a better cooperation among programmers, designers and users, in a one-to-one and community oriented fashion. Our contribution is a web-based interface, to create visualization flows that can be edited and shared, between actors within communities. We detail a real case study where programmers, designers and users successfully worked together to quickly design and improve an interactive image visualization interface, based on images similarities.