We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

Hyperparameter optimization (HPO) is a long-running process that can span hours or even days. While recent Human-in-the-Loop HPO systems enable monitoring and steering of the process, they are typically designed for desktop environments, which limits their effectiveness in managing prolonged experiments in practice. To address these limitations, we present HyPockeTuner, an interactive mobile system that enables users to monitor, steer, and reflect on HPO experiments anytime, anywhere from smartphones. Its mobile-tailored interface supports tracking experiment history and visualizing the relationship between user interventions and performance changes. HyPockeTuner also employs a notification workflow that alerts users to important events, reducing the burden of constant monitoring while enabling timely interventions. In a pilot study, we validated that users could readily identify critical events, such as performance improvements and intervention points, through our visualization. Furthermore, two five-day deployment studies with follow-up reflection sessions demonstrated that users could integrate experiment management into their daily routines and reflect on past decisions, generating insights for future improvement.

Combining conversational AI with refreshable tactile displays (RTDs) offers significant potential for creating accessible data visualization for people who are blind or have low vision (BLV). To support researchers and developers building accessible data visualizations with RTDs, we present a multimodal data interaction architecture along with an open-source reference implementation. Our system is the first to combine touch input with a conversational agent on an RTD, enabling deictic queries that fuse touch context with spoken language, such as "what is the trend between these points?" The architecture addresses key technical challenges, including touch sensing on RTDs, visual-to-tactile encoding, integrating touch context with conversational AI, and synchronizing multimodal output. Our contributions are twofold: (1) a technical architecture integrating RTD hardware, external touch sensing, and conversational AI to enable multimodal data interaction; and (2) an open-source reference implementation demonstrating its feasibility. This work provides a technical foundation to support future research in multimodal accessible data visualization.

Traditional statistical graphics are precise but often lack the visual appeal, memorability, and engagement of pictorial charts. We present a generative framework for the automated synthesis of pictorial charts that bridges the gap between semantic expression and structural faithfulness. Rather than treating charts merely as images to be stylized, we frame the problem as a dual-conditioned generation task guided by two parallel external control signals: a text prompt capturing the semantic context of the editing intent, and a context image providing the abstract statistical chart's global structure. To reinforce these controls within a Multi-Modal Diffusion Transformer, we introduce two complementary feature-level mechanisms: structural alignment to anchor spatial layouts to the input chart, and semantic alignment to transfer expressive textures from reference images. Generalizing across major visual channels (i.e., length, area, angle, and position) and diverse semantic domains, our method produces pictorial charts that are both artistically compelling and structurally consistent. Extensive quantitative evaluations and perceptual user studies demonstrate that our framework outperforms traditional controllable generation and image editing baselines, providing a foundation for high-fidelity, data-driven generative modeling in expressive visual storytelling. Project page: https://ssalign.github.io/.

Tactile graphics are widely used to present maps and statistical diagrams to blind and low vision (BLV) people, with accessibility guidelines recommending their use for graphics where spatial relationships are important. Their use is expected to grow with the advent of commodity refreshable tactile displays. However, in stark contrast to visual information graphics, we lack a clear understanding of the benefts that well-designed tactile information graphics offer over text descriptions for BLV people. To address this gap, we introduce a framework considering the three components of encoding, perception and cognition to examine the known benefts for visual information graphics and explore their applicability to tactile information graphics. This work establishes a preliminary theoretical foundation for the tactile-frst design of information graphics and identifes future research avenues.

We present PiCCL (Pictorial Chart Composition Language), a new language that enables users to easily create pictorial charts using a set of simple operators. To support systematic construction while addressing the main challenge of expressive pictorial chart authoring-manual composition and fine-tuning of visual properties-PiCCL introduces a parametric representation that integrates data-driven chart generation with graphical composition. It also employs a lazy data-binding mechanism that automatically synthesizes charts. PiCCL is grounded in a comprehensive analysis of real-world pictorial chart examples. We describe PiCCL's design and its implementation as piccl.js, a JavaScript-based library. To evaluate PiCCL, we showcase a gallery that demonstrates its expressiveness and report findings from a user study assessing the usability of piccl.js. We conclude with a discussion of PiCCL's limitations and potential, as well as future research directions.

Figure 1: With Data Formulator 2, analysts can iterate on a previous design by (1) selecting a chart from data threads and (2) providing combined natural language and graphical user interface inputs in the chart builder to specify the new design.The AI model generates code to transform the data and update the chart.Data threads are updated with new charts for future use.

Woosung Koh, Janghan Yoon, MinHyung Lee, Youngjin Song, Jaegwan Cho, Jaehyun Kang, Taehyeon Kim, Se-Young Yun, Youngjae Yu, Bongshin Lee. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Despite the recent surge of research efforts to make data visualizations accessible to people who are blind or have low vision (BLV), how to support BLV people's data analysis remains an important and challenging question. As refreshable tactile displays (RTDs) become cheaper and conversational agents continue to improve, their combination provides a promising approach to support BLV people's interactive data exploration and analysis. To understand how BLV people would use and react to a system combining an RTD with a conversational agent, we conducted a Wizard-of-Oz study with 11 BLV participants, where they interacted with line charts, bar charts, and isarithmic maps. Our analysis of participants' interactions led to the identification of nine distinct patterns. We also learned that the choice of modalities depended on the type of task and prior experience with tactile graphics, and that participants strongly preferred the combination of RTD and speech to a single modality. In addition, participants with more tactile experience described how tactile images facilitated a deeper engagement with the data and supported independent interpretation. Our findings will inform the design of interfaces for such interactive mixed-modality systems.

The 2024 VGTC Visualization Technical Achievement Award goes to Han-Wei Shen for his research on extreme-scale and multivariate time-varying data visualization, uncertainty visualization, and novel approaches to inclusion of AI in scientific workflows. The 2024 VGTC Visualization Technical Achievement Award goes to Bongshin Lee for her groundbreaking contributions in advancing visualization technologies through intuitive interaction designs, data-driven storytelling, and integration of machine learning.

We present an authoring tool, called CAST+ (Canis Studio Plus), that enables the interactive creation of chart animations through the direct manipulation of keyframes. It introduces the visual specification of chart animations consisting of keyframes that can be played sequentially or simultaneously, and animation parameters (e.g., duration, delay). Building on Canis (Ge et al. 2020), a declarative chart animation grammar that leverages data-enriched SVG charts, CAST+ supports auto-completion for constructing both keyframes and keyframe sequences. It also enables users to refine the animation specification (e.g., aligning keyframes across tracks to play them together, adjusting delay) with direct manipulation. We report a user study conducted to assess the visual specification and system usability with its initial version. We enhanced the system's expressiveness and usability: CAST+ now supports the animation of multiple types of visual marks in the same keyframe group with new auto-completion algorithms based on generalized selection. This enables the creation of more expressive animations, while reducing the number of interactions needed to create comparable animations. We present a gallery of examples and four usage scenarios to demonstrate the expressiveness of CAST+. Finally, we discuss the limitations, comparison, and potentials of CAST+ as well as directions for future research.

Creating an animated data video with audio narration is a time-consuming and complex task that requires expertise. It involves designing complex animations, turning written scripts into audio narrations, and synchronizing visual changes with the narrations. This paper presents WonderFlow, an interactive authoring tool, that facilitates narration-centric design of animated data videos. WonderFlow allows authors to easily specify semantic links between text and the corresponding chart elements. Then it automatically generates audio narration by leveraging text-to-speech techniques and aligns the narration with an animation. WonderFlow provides a structure-aware animation library designed to ease chart animation creation, enabling authors to apply pre-designed animation effects to common visualization components. Additionally, authors can preview and refine their data videos within the same system, without having to switch between different creation tools. A series of evaluation results confirmed that WonderFlow is easy to use and simplifies the creation of data videos with narration-animation interplay.

We introduce ChartA11y, an app developed to enable accessible 2-D visualizations on smartphones for blind users through a participatory and iterative design process involving 13 sessions with two blind partners. We also present a design journey for making accessible touch experiences that go beyond simple auditory feedback, incorporating multimodal interactions and multisensory data representations. Together, ChartA11y aimed at providing direct chart accessing and comprehensive chart understanding by applying a two-mode setting: a semantic navigation framework mode and a direct touch mapping mode. By re-designing traditional touch-to-audio interactions, ChartA11y also extends to accessible scatter plots, addressing the under-explored challenges posed by their non-linear data distribution. Our main contributions encompass the detailed participatory design process and the resulting system, ChartA11y, offering a novel approach for blind users to access visualizations on their smartphones.

Spreadsheets are widely used for storing, manipulating, analyzing, and visualizing data. Features such as conditional formatting, formulas, sorting, and filtering play an important role when understanding and analyzing data in spreadsheets. They employ visual cues, but we have little understanding of the experiences of blind screen reader (SR) users with such features. We conducted a study with 12 blind SR users to gain insights into their challenges, workarounds, and strategies in understanding and extracting information from a spreadsheet consisting of multiple tables that incorporated data analysis features. We identified five factors that impact blind SR users’ experiences: cognitive overload, time-information trade-off, lack of awareness and expertise, inadequate system feedback, and delayed and absent SR responses. Drawn from these findings, we discuss design suggestions and future research agenda to improve SR users’ spreadsheet experiences.

This paper investigates new data exploration experiences that enable blind users to interact with statistical data visualizations—bar plots, heat maps, box plots, and scatter plots—leveraging multimodal data representations. In addition to sonification and textual descriptions that are commonly employed by existing accessible visualizations, our MAIDR (multimodal access and interactive data representation) system incorporates two additional modalities (braille and review) that offer complementary benefits. It also provides blind users with the autonomy and control to interactively access and understand data visualizations. In a user study involving 11 blind participants, we found the MAIDR system facilitated the accurate interpretation of statistical visualizations. Participants exhibited a range of strategies in combining multiple modalities, influenced by their past interactions and experiences with data visualizations. This work accentuates the overlooked potential of combining refreshable tactile representation with other modalities and elevates the discussion on the importance of user autonomy when designing accessible data visualizations.

Activity tracking has the potential to promote active lifestyles among older adults. However, current activity tracking technologies may inadvertently perpetuate ageism by focusing on age-related health risks. Advocating for a personalized approach in activity tracking technology, we sought to understand what activities older adults find meaningful to track and the underlying values of those activities. We conducted a reflective interview study following a 7-day activity journaling with 13 participants. We identified various underlying values motivating participants to track activities they deemed meaningful. These values, whether competing or aligned, shape the desirability of activities. Older adults appreciate low-exertion activities, but they are difficult to track. We discuss how these activities can become central in designing activity tracking systems. Our research offers insights for creating value-driven, personalized activity trackers that resonate more fully with the meaningful activities of older adults.

To facilitate the reuse of existing charts, previous research has examined how to obtain a semantic understanding of a chart by deconstructing its visual representation into reusable components, such as encodings. However, existing deconstruction approaches primarily focus on chart styles, handling only basic layouts. In this paper, we investigate how to deconstruct chart layouts, focusing on rectangle-based ones, as they cover not only 17 chart types but also advanced layouts (e.g., small multiples, nested layouts). We develop an interactive tool, called Mystique, adopting a mixed-initiative approach to extract the axes and legend, and deconstruct a chart's layout into four semantic components: mark groups, spatial relationships, data encodings, and graphical constraints. Mystique employs a wizard interface that guides chart authors through a series of steps to specify how the deconstructed components map to their own data. On 150 rectangle-based SVG charts, Mystique achieves above 85% accuracy for axis and legend extraction and 96% accuracy for layout deconstruction. In a chart reproduction study, participants could easily reuse existing charts on new datasets. We discuss the current limitations of Mystique and future research directions.

With most modern visualization tools, authors need to transform their data into tidy formats to create visualizations they want. Because this requires experience with programming or separate data processing tools, data transformation remains a barrier in visualization authoring. To address this challenge, we present a new visualization paradigm, concept binding, that separates high-level visualization intents and low-level data transformation steps, leveraging an AI agent. We realize this paradigm in Data Formulator, an interactive visualization authoring tool. With Data Formulator, authors first define data concepts they plan to visualize using natural languages or examples, and then bind them to visual channels. Data Formulator then dispatches its AI-agent to automatically transform the input data to surface these concepts and generate desired visualizations. When presenting the results (transformed table and output visualizations) from the AI agent, Data Formulator provides feedback to help authors inspect and understand them. A user study with 10 participants shows that participants could learn and use Data Formulator to create visualizations that involve challenging data transformations, and presents interesting future research directions.

Ambient Information Systems (AIS) have shown some success when used as a notification towards users’ health-related activities. But in the actual busy lives of users, ambient notifications might be forgotten or even missed, nullifying the original notification. Could a system use multiple levels of noticeability to ensure its message is received, and how could this concept be effectively portrayed? To examine these questions, we took a Research through Design approach and created plant-mimicking Shape-Changing Interface (S-CI) artifacts, then conducted interviews with 10 participants who currently used a reminder system for health-related activities. We report findings on acceptable scenarios to disrupting people for health-related activities, and participants’ reactions to our design choices, including how using naturalistic aesthetics led to interpretations of the uncanny and morose, and which ways system physicality affected imagined uses. We offer design suggestions in health-related notification systems and S-CIs, and discuss future work in ambient-to-disruptive technology.

Even though screen readers are a core accessibility tool for blind and low vision individuals (BLVIs), most visualizations are incompatible with screen readers. To improve accessible visualization experiences, we partnered with 10 BLV screen reader users (SRUs) in an iterative co-design study to design and develop accessible visualization experiences that afford SRUs the autonomy to interactively read and understand visualizations and their underlying data. During the five-month study, we explored accessible visualization prototypes with our design partners for three one-hour sessions. Our results provide feedback on the synthesized design concepts we explored, why (or why not) they aid comprehension and exploration for SRUs, and how differing design concepts can fit into cohesive accessible visualization experiences. We contribute both Chart Reader, a web-based accessibility engine resulting from our design iterations, and our distilled study findings—organized by design dimensions—in the creation of comprehensive accessible visualization experiences.

We present three studies that investigate the effectiveness of multiple glanceable part-to-whole proportion representations on smartwatch faces. Our goal was to understand how quickly and accurately people can make judgments about their progress toward multiple goals displayed in a small space. We designed our three studies with increasing external validity. The first study compared bar charts, radial bar charts, and text representations—shown with a digital time display. The second study added an analog time dial as a distractor to increase the complexity of the watch face. To emulate realistic viewing conditions, the third study investigated the effect of viewing angles. In Study 1 bar and radial bar charts outperformed text representations, in Study 2 adding an analog time dial as a distractor did not affect task performance, and in Study 3 only the most extreme angle led to some performance decrease. Supplementary material is available at https://osf.io/ad2z7/.

Data scientists require rich mental models of how AI systems behave to effectively train, debug, and work with them. Despite the prevalence of AI analysis tools, there is no general theory describing how people make sense of what their models have learned. We frame this process as a form of sensemaking and derive a framework describing how data scientists develop mental models of AI behavior. To evaluate the framework, we show how existing AI analysis tools fit into this sensemaking process and use it to design AIFinnity , a system for analyzing image-and-text models. Lastly, we explored how data scientists use a tool developed with the framework through a think-aloud study with 10 data scientists tasked with using AIFinnity to pick an image captioning model. We found that AIFinnity ’s sensemaking workflow reflected participants’ mental processes and enabled them to discover and validate diverse AI behaviors.

We present SizePairs, a new technique to create stable and balanced treemap layouts that visualize values changing over time in hierarchical data. To achieve an overall high-quality result across all time steps in terms of stability and aspect ratio, SizePairs employs a new hierarchical size-based pairing algorithm that recursively pairs two nodes that complement their size changes over time and have similar sizes. SizePairs maximizes the visual quality and stability by optimizing the splitting orientation of each internal node and flipping leaf nodes, if necessary. We also present a comprehensive comparison of SizePairs against the state-of-the-art treemaps developed for visualizing time-dependent data. SizePairs outperforms existing techniques in both visual quality and stability, while being faster than the local moves technique.

Current activity tracking technologies are largely trained on younger adults’ data, which can lead to solutions that are not well-suited for older adults. To build activity trackers for older adults, it is crucial to collect training data with them. To this end, we examine the feasibility and challenges with older adults in collecting activity labels by leveraging speech. Specifically, we built MyMove, a speech-based smartwatch app to facilitate the in-situ labeling with a low capture burden. We conducted a 7-day deployment study, where 13 older adults collected their activity labels and smartwatch sensor data, while wearing a thigh-worn activity monitor. Participants were highly engaged, capturing 1,224 verbal reports in total. We extracted 1,885 activities with corresponding effort level and timespan, and examined the usefulness of these reports as activity labels. We discuss the implications of our approach and the collected dataset in supporting older adults through personalized activity tracking technologies.

To better ground technical (systems) investigation and interaction design of cross-device experiences, we contribute an in-depth survey of existing multi-device practices, including fragmented workflows across devices and the way people physically organize and configure their workspaces to support such activity. Further, this survey documents a historically significant moment of transition to a new future of remote work, an existing trend dramatically accelerated by the abrupt switch to work-from-home (and having to contend with the demands of home-at-work) during the COVID-19 pandemic. We surveyed 97 participants, and collected photographs of home setups and open-ended answers to 50 questions categorized in 5 themes. We characterize the wide range of multi-device physical configurations and identify five usage patterns, including: partitioning tasks, integrating multi-device usage, cloning tasks to other devices, expanding tasks and inputs to multiple devices, and migrating between devices. Our analysis also sheds light on the benefits and challenges people face when their workflow is fragmented across multiple devices. These insights have implications for the design of multi-device experiences that support people’s fragmented workflows.

We present the first toolkit that equips blind and visually impaired (BVI) developers with the tools to create accessible data displays. Called PSST (Physical computing Streaming Sensor data Toolkit), it enables BVI developers to understand the data generated by sensors from a mouse to a micro:bit physical computing platform. By assuming visual abilities, earlier efforts to make physical computing accessible fail to address the need for BVI developers to access sensor data. PSST enables BVI developers to understand real-time, real-world sensor data by providing control over what should be displayed, as well as when to display and how to display sensor data. PSST supports filtering based on raw or calculated values, highlighting, and transformation of data. Output formats include tonal sonification, nonspeech audio files, speech, and SVGs for laser cutting. We validate PSST through a series of demonstrations and a user study with BVI developers.

Abstract Combining data visualizations, animations, and audio narrations, data videos can increase viewer engagement and effectively communicate data stories. Due to their increasing popularity, data videos have gained growing attention from the visualization research community. However, recent research on data videos has focused on animations, lacking an understanding of narrations. In this work, we study how data videos use narrations and animations to convey information effectively. We conduct a qualitative analysis on 426 clips with visualizations extracted from 60 data videos collected from a variety of media outlets, covering a diverse array of topics. We manually label 816 sentences with 1226 semantic labels and record the composition of 2553 animations through an open coding process. We also analyze how narrations and animations coordinate with each other by assigning links between semantic labels and animations. With 937 (76.4%) semantic labels and 2503 (98.0%) animations linked, we identify four types of narration‐animation relationships in the collected clips. Drawing from the findings, we discuss study implications and future research opportunities of data videos.

Speech as a natural and low-burden input modality has great potential to support personal data capture. However, little is known about how people use speech input, together with traditional touch input, to capture different types of data in self-tracking contexts. In this work, we designed and developed NoteWordy, a multimodal self-tracking application integrating touch and speech input, and deployed it in the context of productivity tracking for two weeks (N = 17). Our participants used the two input modalities differently, depending on the data type as well as personal preferences, error tolerance for speech recognition issues, and social surroundings. Additionally, we found speech input reduced participants' diary entry time and enhanced the data richness of the free-form text. Drawing from the findings, we discuss opportunities for supporting efficient personal data capture with multimodal input and implications for improving the user experience with natural language input to capture various self-tracking data.

This February 2022 issue of theIEEE Transactions on Visualization and Computer Graphics (TVCG)contains the proceedings of IEEE VIS 2021, held online on October 24-29, 2021, with General Chairs from Tulane University and Universidade de Sao Paulo. With IEEE VIS 2021, the conference series is in its 32nd year.

The factors influencing people’s food decisions, such as one’s mood and eating environment, are important information to foster self-reflection and to develop personalized healthy diet. But, it is difficult to consistently collect them due to the heavy data capture burden. In this work, we examine how speech input supports capturing everyday food practice through a week-long data collection study (N = 11). We deployed FoodScrap, a speech-based food journaling app that allows people to capture food components, preparation methods, and food decisions. Using speech input, participants detailed their meal ingredients and elaborated their food decisions by describing the eating moments, explaining their eating strategy, and assessing their food practice. Participants recognized that speech input facilitated self-reflection, but expressed concerns around re-recording, mental load, social constraints, and privacy. We discuss how speech input can support low-burden and reflective food journaling and opportunities for effectively processing and presenting large amounts of speech data.

Alternative (alt) text provides access to descriptions of digital images for people who use screen readers. While prior work studied screen reader users’ (SRUs’) preferences about alt text and automatic alt text (i.e., alt text generated by artificial intelligence), little work examined the alt text author’s experience composing or editing these descriptions. We built two types of prototype interfaces for two tasks: authoring alt text and providing feedback on automatic alt text. Through combined interview-usability testing sessions with alt text authors and interviews with SRUs, we tested the effectiveness of our prototypes in the context of Microsoft PowerPoint. Our results suggest that authoring interfaces that support authors in choosing what to include in their descriptions result in higher quality alt text. The feedback interfaces highlighted considerable differences in the perceptions of authors and SRUs regarding “high-quality” alt text. Finally, authors crafted significantly lower quality alt text when starting from the automatic alt text compared to starting from a blank box. We discuss the implications of these results on applications that support alt text.

We present CAST, an authoring tool that enables the interactive creation of chart animations. It introduces the visual specification of chart animations consisting of keyframes that can be played sequentially or simultaneously, and animation parameters (e.g., duration, delay). Building on Canis [19], a declarative chart animation grammar that leverages data-enriched SVG charts, CAST supports auto-completion for constructing both keyframes and keyframe sequences. It also enables users to refine the animation specification (e.g., aligning keyframes across tracks to play them together, adjusting delay) with direct manipulation and other parameters for animation effects (e.g., animation type, easing function) using a control panel. In addition to describing how CAST infers recommendations for auto-completion, we present a gallery of examples to demonstrate the expressiveness of CAST and a user study to verify its learnability and usability. Finally, we discuss the limitations and potentials of CAST as well as directions for future research.

Most mobile health apps employ data visualization to help people view their health and activity data, but these apps provide limited support for visual data exploration. Furthermore, despite its huge potential benefits, mobile visualization research in the personal data context is sparse. This work aims to empower people to easily navigate and compare their personal health data on smartphones by enabling flexible time manipulation with speech. We designed and developed [email protected], a mobile app that leverages the synergy of two complementary modalities: speech and touch. Through an exploratory study with 13 long-term Fitbit users, we examined how multimodal interaction helps participants explore their own health data. Participants successfully adopted multimodal interaction (i.e., speech and touch) for convenient and fluid data exploration. Based on the quantitative and qualitative findings, we discuss design implications and opportunities with multimodal interaction for better supporting visual data exploration on mobile devices.

Natural language interfaces (NLIs) for data visualization are becoming increasingly popular both in academic research and in commercial software. Yet, there is a lack of empirical understanding of how people specify visualizations through natural language. We conducted an online study (N = 102), showing participants a series of visualizations and asking them to provide utterances they would pose to generate the displayed charts. From the responses, we curated a dataset of 893 utterances and characterized the utterances according to (1) their phrasing (e.g., commands, queries, questions) and (2) the information they contained (e.g., chart types, data aggregations). To help guide future research and development, we contribute this utterance dataset and discuss its applications toward the creation and benchmarking of NLIs for visualization.

We contribute a method to automate parameter configurations for chart layouts by learning from human preferences. Existing charting tools usually determine the layout parameters using predefined heuristics, producing sub-optimal layouts. People can repeatedly adjust multiple parameters (e.g., chart size, gap) to achieve visually appealing layouts. However, this trial-and-error process is unsystematic and time-consuming, without a guarantee of improvement. To address this issue, we develop Layout Quality Quantifier (LQ2), a machine learning model that learns to score chart layouts from paired crowdsourcing data. Combined with optimization techniques, LQ2 recommends layout parameters that improve the charts’ layout quality. We apply LQ2 on bar charts and conduct user studies to evaluate its effectiveness by examining the quality of layouts it produces. Results show that LQ2 can generate more visually appealing layouts than both laypeople and baselines. This work demonstrates the feasibility and usages of quantifying human preferences and aesthetics for chart layouts.

Designing natural language interfaces for querying databases remains an important goal pursued by researchers in natural language processing, databases, and HCI. These systems receive natural language as input, translate it into a formal database query, and execute the query to compute a result. Because the responses from these systems are not always correct, it is important to provide people with mechanisms to assess the correctness of the generated query and computed result. However, this assessment can be challenging for people who lack expertise in query languages. We present Debug-It-Yourself (DIY), an interactive technique that enables users to assess the responses from a state-of-the-art natural language to SQL (NL2SQL) system for correctness and, if possible, fix errors. DIY provides users with a sandbox where they can interact with (1) the mappings between the question and the generated query, (2) a small-but-relevant subset of the underlying database, and (3) a multi-modal explanation of the generated query. End-users can then employ a back-of-the-envelope calculation debugging strategy to evaluate the system’s response. Through an exploratory study with 12 users, we investigate how DIY helps users assess the correctness of the system’s answers and detect & fix errors. Our observations reveal the benefits of DIY while providing insights about end-user debugging strategies and underscore opportunities for further improving the user experience.

AirConstellations supports a unique semi-fixed style of cross-device interactions via multiple self-spatially-aware armatures to which users can easily attach (or detach) tablets and other devices. In particular, AirConstellations affords highly flexible and dynamic device formations where the users can bring multiple devices together in-air — with 2–5 armatures poseable in 7DoF within the same workspace — to suit the demands of their current task, social situation, app scenario, or mobility needs. This affords an interaction metaphor where relative orientation, proximity, attaching (or detaching) devices, and continuous movement into and out of ad-hoc ensembles can drive context-sensitive interactions. Yet all devices remain self-stable in useful configurations even when released in mid-air.

We present a new visual exploration concept-Progressive Visual Analytics with Safeguards-that helps people manage the uncertainty arising from progressive data exploration. Despite its potential benefits, intermediate knowledge from progressive analytics can be incorrect due to various machine and human factors, such as a sampling bias or misinterpretation of uncertainty. To alleviate this problem, we introduce PVA-Guards, safeguards people can leave on uncertain intermediate knowledge that needs to be verified, and derive seven PVA-Guards based on previous visualization task taxonomies. PVA-Guards provide a means of ensuring the correctness of the conclusion and understanding the reason when intermediate knowledge becomes invalid. We also present ProReveal, a proof-of-concept system designed and developed to integrate the seven safeguards into progressive data exploration. Finally, we report a user study with 14 participants, which shows people voluntarily employed PVA-Guards to safeguard their findings and ProReveal's PVA-Guard view provides an overview of uncertain intermediate knowledge. We believe our new concept can also offer better consistency in progressive data exploration, alleviating people's heterogeneous interpretation of uncertainty.

A fundamental part of data visualization is transforming data to map abstract information onto visual attributes. While this abstraction is a powerful basis for data visualization, the connection between the representation and the original underlying data (i.e., what the quantities and measurements actually correspond with in reality) can be lost. On the other hand, virtual reality (VR) is being increasingly used to represent real and abstract models as natural experiences to users. In this work, we explore the potential of using VR to help restore the basic understanding of units and measures that are often abstracted away in data visualization in an approach we call data visceralization. By building VR prototypes as design probes, we identify key themes and factors for data visceralization. We do this first through a critical reflection by the authors, then by involving external participants. We find that data visceralization is an engaging way of understanding the qualitative aspects of physical measures and their real-life form, which complements analytical and quantitative understanding commonly gained from data visualization. However, data visceralization is most effective when there is a one-to-one mapping between data and representation, with transformations such as scaling affecting this understanding. We conclude with a discussion of future directions for data visceralization.

Multimodal interfaces that combine direct manipulation and natural language have shown great promise for data visualization. Such multimodal interfaces allow people to stay in the flow of their visual exploration by leveraging the strengths of one modality to complement the weaknesses of others. In this article, we introduce an approach that interweaves multimodal interaction combining direct manipulation and natural language with flexible unit visualizations. We employ the proposed approach in a proof-of-concept system, DataBreeze. Coupling pen, touch, and speech-based multimodal interaction with flexible unit visualizations, DataBreeze allows people to create and interact with both systematically bound (e.g., scatterplots, unit column charts) and manually customized views, enabling a novel visual data exploration experience. We describe our design process along with DataBreeze's interface and interactions, delineating specific aspects of the design that empower the synergistic use of multiple modalities. We also present a preliminary user study with DataBreeze, highlighting the data exploration patterns that participants employed. Finally, reflecting on our design process and preliminary user study, we discuss future research directions.

We contribute MobileVisFixer, a new method to make visualizations more mobile-friendly. Although mobile devices have become the primary means of accessing information on the web, many existing visualizations are not optimized for small screens and can lead to a frustrating user experience. Currently, practitioners and researchers have to engage in a tedious and time-consuming process to ensure that their designs scale to screens of different sizes, and existing toolkits and libraries provide little support in diagnosing and repairing issues. To address this challenge, MobileVisFixer automates a mobile-friendly visualization re-design process with a novel reinforcement learning framework. To inform the design of MobileVisFixer, we first collected and analyzed SVG-based visualizations on the web, and identified five common mobile-friendly issues. MobileVisFixer addresses four of these issues on single-view Cartesian visualizations with linear or discrete scales by a Markov Decision Process model that is both generalizable across various visualizations and fully explainable. MobileVisFixer deconstructs charts into declarative formats, and uses a greedy heuristic based on Policy Gradient methods to find solutions to this difficult, multi-criteria optimization problem in reasonable time. In addition, MobileVisFixer can be easily extended with the incorporation of optimization algorithms for data visualizations. Quantitative evaluation on two real-world datasets demonstrates the effectiveness and generalizability of our method.

Smart speakers such as Amazon Echo present promising opportunities for exploring voice interaction in the domain of in-home exercise tracking. In this work, we examine if and how voice interaction complements and augments a mobile app in promoting consistent exercise. We designed and developed TandemTrack, which combines a mobile app and an Alexa skill to support exercise regimen, data capture, feedback, and reminder. We then conducted a four-week between-subjects study deploying TandemTrack to 22 participants who were instructed to follow a short daily exercise regimen: one group used only the mobile app and the other group used both the app and the skill. We collected rich data on individuals' exercise adherence and performance, and their use of voice and visual interactions, while examining how TandemTrack as a whole influenced their exercise experience. Reflecting on these data, we discuss the benefits and challenges of incorporating voice interaction to assist daily exercise, and implications for designing effective multimodal systems to support self-tracking and promote consistent exercise.

While tablet devices are a promising platform for data visualization, supporting consistent interactions across different types of visualizations on tablets remains an open challenge. In this paper, we present multimodal interactions that function consistently across different visualizations, supporting common operations during visual data analysis. By considering standard interface elements (e.g., axes, marks) and grounding our design in a set of core concepts including operations, parameters, targets, and instruments, we systematically develop interactions applicable to different visualization types. To exemplify how the proposed interactions collectively facilitate data exploration, we employ them in a tablet-based system, InChorus that supports pen, touch, and speech input. Based on a study with 12 participants performing replication and factchecking tasks with InChorus, we discuss how participants adapted to using multimodal input and highlight considerations for future multimodal visualization systems.

Abstract People are becoming increasingly sophisticated in their ability to navigate information spaces using search, hyperlinks, and visualization. But, mobile phones preclude the use of multiple coordinated views that have proven effective in the desktop environment (e.g., for business intelligence or visual analytics). In this work, we propose to model information as multivariate heterogeneous networks to enable greater analytic expression for a range of sensemaking tasks while suggesting a new, list‐based paradigm with gestural navigation of structured information spaces on mobile phones. We also present a mobile application, called Orchard, which combines ideas from both faceted search and interactive network exploration in a visual query language to allow users to collect facets of interest during exploratory navigation. Our study showed that users could collect and combine these facets with Orchard, specifying network queries and projections that would only have been possible previously using complex data tools or custom data science.

We compare the efficacy of animated and small multiples variants of scatterplots on mobile phones for comparing trends in multivariate datasets. Visualization is increasingly prevalent in mobile applications and mobile-first websites, yet there is little prior visualization research dedicated to small displays. In this paper, we build upon previous experimental research carried out on larger displays that assessed animated and non-animated variants of scatterplots. Incorporating similar experimental stimuli and tasks, we conducted an experiment where 96 crowdworker participants performed nine trend comparison tasks using their mobile phones. We found that those using a small multiples design consistently completed tasks in less time, albeit with slightly less confidence than those using an animated design. The accuracy results were more task-dependent, and we further interpret our results according to the characteristics of the individual tasks, with a specific focus on the trajectories of target and distractor data items in each task. We identify cases that appear to favor either animation or small multiples, providing new questions for further experimental research and implications for visualization design on mobile devices. Lastly, we provide a reflection on our evaluation methodology.

An emerging generation of visualization authoring systems support expressive information visualization without textual programming. As they vary in their visualization models, system architectures, and user interfaces, it is challenging to directly compare these systems using traditional evaluative methods. Recognizing the value of contextualizing our decisions in the broader design space, we present critical reflections on three systems we developed -Lyra, Data Illustrator, and Charticulator. This paper surfaces knowledge that would have been daunting within the constituent papers of these three systems. We compare and contrast their (previously unmentioned) limitations and trade-offs between expressivity and learnability. We also reflect on common assumptions that we made during the development of our systems, thereby informing future research directions in visualization authoring systems.

We present a new technique to enable the creation of shape-bounded Wordles, we call ShapeWordle, in which we fit words to form a given shape. To guide word placement within a shape, we extend the traditional Archimedean spirals to be shape-aware by formulating the spirals in a differential form using the distance field of the shape. To handle non-convex shapes, we introduce a multi-centric Wordle layout method that segments the shape into parts for our shape-aware spirals to adaptively fill the space and generate word placements. In addition, we offer a set of editing interactions to facilitate the creation of semantically-meaningful Wordles. Lastly, we present three evaluations: a comprehensive comparison of our results against the state-of-the-art technique (WordArt), case studies with 14 users, and a gallery to showcase the coverage of our technique.

Productivity tracking tools often determine productivity based on the time interacting with work-related applications. To deconstruct productivity's diverse and nebulous nature, we investigate how knowledge workers conceptualize personal productivity and delimit productive tasks in both work and non-work contexts. We report a 2-week diary study followed by a semi-structured interview with 24 knowledge workers. Participants captured productive activities and provided the rationale for why the activities were assessed to be productive. They reported a wide range of productive activities beyond typical desk-bound work-ranging from having a personal conversation with dad to getting a haircut. We found six themes that characterize the productivity assessment-work product, time management, worker's state, attitude toward work, impact & benefit, and compound task and identified how participants interleaved multiple facets when assessing their productivity. We discuss how these findings could inform the design of a comprehensive productivity tracking system that covers a wide range of productive activities.

During sensemaking, people annotate insights: underlining sentences in a document or circling regions on a map. They jot down their hypotheses: drawing correlation lines on scatterplots or creating personal legends to track patterns. We present ActiveInk, a system enabling people to seamlessly transition between exploring data and externalizing their thoughts using pen and touch. ActiveInk enables the natural use of pen for active reading behaviors, while supporting analytic actions by activating any of these ink strokes. Through a qualitative study with eight participants, we contribute observations of active reading behaviors during data exploration and design principles to support sensemaking.

Externalizing one's thoughts can be helpful during data analysis, such as which one marks interesting data, notes hypotheses, and draws diagrams. In this paper, we present two exploratory studies conducted to investigate types and use of externalizations during the analysis process. We first studied how people take notes during different stages of data analysis using VoyagerNote, a visualization recommendation system augmented to support text annotations, and coupled with participants' favorite external note-taking tools (e.g., word processor, pen & paper). Externalizations manifested mostly as notes written on paper or in a word processor, with annotations atop views used almost exclusively in the initial phase of analysis. In the second study, we investigated two specific opportunities: (1) integrating digital pen input to facilitate the use of free-form externalizations and (2) providing a more explicit linking between visualizations and externalizations. We conducted the study with VoyagerInk, a visualization system that enabled free-form externalization with a digital pen as well as touch interactions to link externalizations to data. Participants created more graphical externalizations with VoyagerInk and revisited over half of their externalizations via the linking mechanism. Reflecting on the findings from these two studies, we discuss implications for the design of data analysis tools.

OBJECTIVE: Despite the potential values self-tracking data could offer, we have little understanding of how much access people have to "their" data. Our goal of this article is to unveil the current state of the data accessibility-the degree to which people can access their data-of personal health apps in the market. MATERIALS AND METHODS: We reviewed 240 personal health apps from the App Store and selected 45 apps that support semi-automated tracking. We characterized the data accessibility of these apps using two dimensions-data access methods and data types. RESULTS: More than 90% of our sample apps (n = 41) provide some types of data access support, which include synchronizing data with a health platform (ie, Apple Health), file download, and application program interfaces. However, the two approachable data access methods for laypeople-health platform and file download-typically put a significant limit on data format, granularity, and amount, which constrains people from easily repurposing the data. DISCUSSION: Personal data should be accessible to the people who collect them, but existing methods lack sufficient support for people in accessing the fine-grained data. Lack of standards in personal health data schema as well as frequent changes in market conditions are additional hurdles to data accessibility. CONCLUSIONS: Many stakeholders including patients, healthcare providers, researchers, third-party developers, and the general public rely on data accessibility to utilize personal data for various goals. As such, improving data accessibility should be considered as an important factor in designing personal health apps and health platforms.

We present the results of two perception studies to assess how quickly people can perform a simple data comparison task for small-scale visualizations on a smartwatch. The main goal of these studies is to extend our understanding of design constraints for smartwatch visualizations. Previous work has shown that a vast majority of smartwatch interactions last under 5 s. It is still unknown what people can actually perceive from visualizations during such short glances, in particular with such a limited display space of smartwatches. To shed light on this question, we conducted two perception studies that assessed the lower bounds of task time for a simple data comparison task. We tested three chart types common on smartwatches: bar charts, donut charts, and radial bar charts with three different data sizes: 7, 12, and 24 data values. In our first study, we controlled the differences of the two target bars to be compared, while the second study varied the difference randomly. For both studies, we found that participants performed the task on average in <300 ms for the bar chart, <220 ms for the donut chart, and in <1780 ms for the radial bar chart. Thresholds in the second study per chart type were on average 1.14-1.35× higher than in the first study. Our results show that bar and donut charts should be preferred on smartwatch displays when quick data comparisons are necessary.

In the first crowdsourced visualization experiment conducted exclusively on mobile phones, we compare approaches to visualizing ranges over time on small displays. People routinely consume such data via a mobile phone, from temperatures in weather forecasting apps to sleep and blood pressure readings in personal health apps. However, we lack guidance on how to effectively visualize ranges on small displays in the context of different value retrieval and comparison tasks, or with respect to different data characteristics such as periodicity, seasonality, or the cardinality of ranges. Central to our experiment is a comparison between two ways to lay out ranges: a more conventional linear layout strikes a balance between quantitative and chronological scale resolution, while a less conventional radial layout emphasizes the cyclicality of time and may prioritize discrimination between values at its periphery. With results from 87 crowd workers, we found that while participants completed tasks more quickly with linear layouts than with radial ones, there were few differences in terms of error rate between layout conditions. We also found that participants performed similarly with both layouts in tasks that involved comparing superimposed observed and average ranges.

We present Charticulator, an interactive authoring tool that enables the creation of bespoke and reusable chart layouts. Charticulator is our response to most existing chart construction interfaces that require authors to choose from predefined chart layouts, thereby precluding the construction of novel charts. In contrast, Charticulator transforms a chart specification into mathematical layout constraints and automatically computes a set of layout attributes using a constraint-solving algorithm to realize the chart. It allows for the articulation of compound marks or glyphs as well as links between these glyphs, all without requiring any coding or knowledge of constraint satisfaction. Furthermore, thanks to the constraint-based layout approach, Charticulator can export chart designs into reusable templates that can be imported into other visualization tools. In addition to describing Charticulator's conceptual framework and design, we present three forms of evaluation: a gallery to illustrate its expressiveness, a user study to verify its usability, and a click-count comparison between Charticulator and three existing tools. Finally, we discuss the limitations and potentials of Charticulator as well as directions for future research. Charticulator is available with its source code at https://charticulator.com.

Pictographic representations and animation techniques are commonly incorporated into narrative visualizations such as data videos. General belief is that these techniques may enhance the viewer experience, thus appealing to a broad audience and enticing the viewer to consume the entire video. However, no study has formally assessed the effect of these techniques on data insight communication and viewer engagement. In this paper, we first propose a scale-based questionnaire covering five factors of viewer engagement we identified from multiple application domains such as game design and marketing. We then validate this questionnaire through a crowdsourcing study on Amazon's Mechanical Turk to assess the effect of animation and pictographs in data videos. Our results reveal that each technique has an effect on viewer engagement, impacting different factors. In addition, insights from these studies lead to design considerations for authoring engaging data videos.

Multimodal interaction offers many potential benefits for data visualization. It can help people stay in the flow of their visual analysis and presentation, with the strengths of one interaction modality offsetting the weaknesses of others. Furthermore, multimodal interaction offers strong promise for leveraging data visualization on diverse display hardware including mobile, AR/VR, and large displays. However, prior research on visualization and interaction techniques has mostly explored a single input modality such as mouse, touch, pen, or more recently, natural language. The unique challenges and opportunities of synergistic multimodal interaction for data visualization have yet to be investigated. This workshop will bring together researchers with expertise in visualization, interaction design, and natural user interfaces. We aim to build a community of researchers focusing on multimodal interaction for data visualization, explore opportunities and challenges in our research, and establish an agenda for multimodal interaction research specifically for data visualization.

Play is an enjoyable and developmentally useful part of early childhood, and parent-child play is a highly productive mechanism by which children learn to participate in the world. We conducted an observational lab study to examine how 15 parent-child pairs (children age 4-6) respond to and play with tablet apps as compared to analog toys. We found that parents and children were less likely to engage with each other or to respond to each other's bids for attention during play sessions with tab-lets versus play sessions with toys. We also observed that specific design features of tablet devices and children's apps-such as one-sided interfaces, game paradigms that demand continual attention, and lack of support for parallel interaction-are the primary mechanism shaping these differences. We provide guidance suggesting how children's apps might be re-designed to preserve the ad-vantages of digital play experiences while also evolving to build in the advantages of traditional toys.

Extended periods of uninterrupted sedentary behavior are detrimental to long-term health. While prolonged sitting is prevalent among information workers, it is difficult for them to break prolonged sedentary behavior due to the nature of their work. This work aims to understand information workers' intentions & practices around standing or moving breaks. We developed Time for Break, a break prompting system that enables people to set their desired work duration and prompts them to stand up or move. We conducted an exploratory field study (N = 25) with Time for Break to collect participants' work & break intentions and behaviors for three weeks, followed by semi-structured interviews. We examined rich contexts affecting participants' receptiveness to standing or moving breaks, and identified how their habit strength and self-regulation are related to their break-taking intentions & practices. We discuss design implications for interventions to break up periods of prolonged sedentary behavior in workplaces.

An increasingly common approach to data analysis involves using information dashboards to visually compare changing data. However, layout constraints coupled with varying levels of visualization literacy among dashboard users make facilitating visual comparison in dashboards a challenging task. In this paper, we evaluate variants of bar charts, one of the most prevalent class of charts used in dashboards. We report an online experiment (N = 74) conducted to evaluate four alternative designs: 1) grouped bar chart, 2) grouped bar chart with difference overlays, 3) bar chart with difference overlays, and 4) difference bar chart. Results show that charts with difference overlays facilitate a wider range of comparison tasks while performing comparably to charts without them on individual tasks. Finally, we discuss the implications of our findings, with a focus on supporting visual comparison in dashboards.

Immersive data-driven storytelling, which uses interactive immersive visualizations to present insights from data, is a compelling use case for VR and AR environments. We present XRCreator, an authoring system to create immersive data-driven stories. The cross-platform nature of our React-inspired system architecture enables the collaboration among VR, AR, and web users, both in authoring and in experiencing immersive data-driven stories.

Abstract Visualization researchers have been increasingly leveraging crowdsourcing approaches to overcome a number of limitations of controlled laboratory experiments, including small participant sample sizes and narrow demographic backgrounds of study participants. However, as a community, we have little understanding on when, where, and how researchers use crowdsourcing approaches for visualization research. In this paper, we review the use of crowdsourcing for evaluation in visualization research. We analyzed 190 crowdsourcing experiments, reported in 82 papers that were published in major visualization conferences and journals between 2006 and 2017. We tagged each experiment along 36 dimensions that we identified for crowdsourcing experiments. We grouped our dimensions into six important aspects: study design & procedure, task type, participants, measures & metrics, quality assurance, and reproducibility. We report on the main findings of our review and discuss challenges and opportunities for improvements in conducting crowdsourcing studies for visualization research.

The four papers in this special section were presented at the 2018 IEEE Pacific Visualization Symposium (IEEE PacificVis’18), which was held at the Integrated Research Center of Kobe University, Kobe,Japan from April 10 to 13, 2018.

Parental controls allow parents to set limits on children's use of technology, but prior work suggests that controlling children alone is unlikely to foster the development of healthy media habits. We took elements from evidence-based preschool curricula that teach self-regulation and translated them to the digital space by creating a tool for preschoolers and parents to plan their device-based playtime. In an observational lab study with 11 parent-child dyads and follow-up interviews with 14 parents, we found that children demonstrated intentionality and made goal-directed choices as they planned, the mediating factor in developing self-regulation. We observed that parents prompted their child to be intentional and solicited children's input. When children played through their plan, they transitioned to the next activity without intervention 93% of the time. Our results suggest that evidence-based practices for teaching self-regulation in a non-digital context can be applied productively to children's use of technology. As parents supported children in trying the tool for the first time, a further contribution of this work is a hierarchical model of parents' approaches to scaffolding children's use of a novel technology.

Annotation plays an important role in conveying key points in visual data-driven storytelling; it helps presenters explain and emphasize core messages and specific data. However, the visualization research community has a limited understanding of annotation and its role in data-driven storytelling, and existing charting software provides limited support for creating annotations. In this paper, we characterize a design space of chart annotations, one informed by a survey of 106 annotated charts published by six prominent news graphics desks. Using this design space, we designed and developed ChartAccent, a tool that allows people to quickly and easily augment charts via a palette of annotation interactions that generate manual and data-driven annotations. We also report on a study in which participants reproduced a series of annotated charts using ChartAccent, beginning with unadorned versions of the same charts. Finally, we discuss the lessons learned during the process of designing and evaluating ChartAccent, and suggest directions for future research.

In this case study, we describe a design workshop with 7 children age 4-6 using existing co-design techniques known to elicit design insights in older individuals. We found that our 5- and 6-year-old participants successfully generated design ideas using these methods, while 4-year-olds were unable to use create solutions in a traditional format. How-ever, these younger children enthusiastically offered opportunities where, with methodological guidance, the research-er could have followed the child's lead and shifted the design question to one that was potentially more meaningful for the participant. We propose a future work to examine the effectiveness of giving these younger participants great-er authority in defining and scoping the problem space.

Recent advancements in tablet technology pose a great opportunity for information visualization to expand its horizons beyond desktops. In this paper, we present TouchPivot, a novel interface that assists visual data exploration on tablet devices. With novices in mind, TouchPivot supports data transformations, such as pivoting and filtering, with simple pen and touch interactions, and facilitates understanding of the transformations through tight coupling between a data table and visualization. We bring in WIMP interfaces to TouchPivot, leveraging their familiarity and accessibility to novices. We report on a user study conducted to compare TouchPivot with two commercial interfaces, Tableau and Microsoft Excel's PivotTable. Our results show that novices not only answered data-driven questions faster, but also created a larger number of meaningful charts during freeform exploration with TouchPivot than others. Finally, we discuss the main hurdles novices encountered during our study and possible remedies for them.

Charts are commonly used to present data in digital documents such as web pages, research papers, or presentation slides. When the underlying data is not available, it is necessary to extract the data from a chart image to utilize the data for further analysis or improve the chart for more accurate perception. In this paper, we present ChartSense, an interactive chart data extraction system. ChartSense first determines the chart type of a given chart image using a deep learning based classifier, and then extracts underlying data from the chart image using semi-automatic, interactive extraction algorithms optimized for each chart type. To evaluate chart type classification accuracy, we compared ChartSense with ReVision, a system with the state-of-the-art chart type classifier. We found that ChartSense was more accurate than ReVision. In addition, to evaluate data extraction performance, we conducted a user study, comparing ChartSense with WebPlotDigitizer, one of the most effective chart data extraction tools among publicly accessible ones. Our results showed that ChartSense was better than WebPlotDigitizer in terms of task completion time, error rate, and subjective preference.

Previous research suggests an important role for self-tracking in promoting mental wellness. Recent studies with college student populations have examined the feasibility of collecting everyday mood, activity, and social data. However, these studies do not account for students' experiences and challenges adopting self-tracking technologies to support mental wellness goals. We present two studies conducted to better understand self-tracking for stress management and mental wellness in student populations. First, focus groups and card sorting activities with 14 student health professionals reveal expert perspectives on the usefulness of tracking for three scenarios. Second, an online survey of 297 students examines personal experiences with self-tracking and attitudes toward sharing self-tracked data with others. We draw on findings from these studies to characterize students' motivations, challenges, and preferences in collecting and viewing self-tracked data related to mental wellness, and we compare findings between students with diagnosed mental illnesses and those without. We conclude with a discussion of challenges and opportunities in leveraging self-tracking for mental wellness, highlighting several design considerations.

Abstract Many factors can shape the flow of visual data‐driven stories, and thereby the way readers experience those stories. Through the analysis of 80 existing stories found on popular websites, we systematically investigate and identify seven characteristics of these stories, which we name “flow‐factors,” and we illustrate how they feed into the broader concept of “visual narrative flow.” These flow‐factors are navigation input, level of control, navigation progress, story layout, role of visualization, story progression, and navigation feedback. We also describe a series of studies we conducted, which shed initial light on how different visual narrative flows impact the reading experience. We report on two exploratory studies, in which we gathered reactions and preferences of readers for stepper‐ vs. scroller‐driven flows. We then report on a crowdsourced study with 240 participants, in which we explore the effect of the combination of different flow‐factors on readers’ engagement. Our results indicate that visuals and navigation feedback (e.g., static vs. animated transitions) have an impact on readers’ engagement, while level of control (e.g., discrete vs. continuous) may not.

Abstract Web‐based visualization libraries are in wide use, but performance bottlenecks occur when rendering, and especially animating, a large number of graphical marks. While GPU‐based rendering can drastically improve performance, that paradigm has a steep learning curve, usually requiring expertise in the computer graphics pipeline and shader programming. In addition, the recent growth of virtual and augmented reality poses a challenge for supporting multiple display environments beyond regular canvases, such as a Head Mounted Display (HMD) and Cave Automatic Virtual Environment (CAVE). In this paper, we introduce a new web‐based visualization library called Stardust, which provides a familiar API while leveraging GPU's processing power. Stardust also enables developers to create both 2D and 3D visualizations for diverse display environments using a uniform API. To demonstrate Stardust's expressiveness and portability, we present five example visualizations and a coding playground for four display environments. We also evaluate its performance by comparing it against the standard HTML5 Canvas, D3, and Vega.

Data videos, or short data-driven motion graphics, are an increasingly popular medium for storytelling. However, creating data videos is difficult as it involves pulling together a unique combination of skills. We introduce DataClips, an authoring tool aimed at lowering the barriers to crafting data videos. DataClips allows non-experts to assemble data-driven "clips" together to form longer sequences. We constructed the library of data clips by analyzing the composition of over 70 data videos produced by reputable sources such as The New York Times and The Guardian. We demonstrate that DataClips can reproduce over 90% of our data videos corpus. We also report on a qualitative study comparing the authoring process and outcome achieved by (1) non-experts using DataClips, and (2) experts using Adobe Illustrator and After Effects to create data-driven clips. Results indicated that non-experts are able to learn and use DataClips with a short training period. In the span of one hour, they were able to produce more videos than experts using a professional editing tool, and their clips were rated similarly by an independent audience.

The papers in this special issue were presented at IEEE VIS 2016, held during October 23-28, 2016 in Baltimore, MD. VIS contains three conferences, held concurrently: the IEEE Visual Analytics Science and Technology Conference (IEEE VAST 2016), the IEEE Information Visualization Conference (IEEE InfoVis 2016), and the IEEE Scientific Visualization Conference (IEEE SciVis2016).

There are many ways to visualize event sequences as timelines. In a storytelling context where the intent is to convey multiple narrative points, a richer set of timeline designs may be more appropriate than the narrow range that has been used for exploratory data analysis by the research community. Informed by a survey of 263 timelines, we present a design space for storytelling with timelines that balances expressiveness and effectiveness, identifying 14 design choices characterized by three dimensions: representation, scale, and layout. Twenty combinations of these choices are viable timeline designs that can be matched to different narrative points, while smooth animated transitions between narrative points allow for the presentation of a cohesive story, an important aspect of both interactive storytelling and data videos. We further validate this design space by realizing the full set of viable timeline designs and transitions in a proof-of-concept sandbox implementation that we used to produce seven example timeline stories. Ultimately, this work is intended to inform and inspire the design of future tools for storytelling with timelines.

Performance analysis is critical in applied machine learning because it influences the models practitioners produce. Current performance analysis tools suffer from issues including obscuring important characteristics of model behavior and dissociating performance from data. In this work, we present Squares, a performance visualization for multiclass classification problems. Squares supports estimating common performance metrics while displaying instance-level distribution information necessary for helping practitioners prioritize efforts and access data. Our controlled study shows that practitioners can assess performance significantly faster and more accurately with Squares than a confusion matrix, a common performance analysis tool in machine learning.

To help people enhance their personal productivity by providing effective feedback, we designed and developed TimeAware, a self-monitoring system for capturing and reflecting on personal computer usage behaviors. TimeAware employs an ambient widget to promote self-awareness and to lower the feedback access burden, and web-based information dashboard to visualize people's detailed computer usage. To examine the effect of framing on individual's productivity, we designed two versions of TimeAware, each with a different framing setting-one emphasizing productive activities (positive framing) and the other emphasizing distracting activities (negative framing), and conducted an eight-week deployment study (N = 24). We found a significant effect of framing on participants' productivity: only participants in the negative framing condition improved their productivity. The ambient widget seemed to help sustain engagement with data and enhance self-awareness. We discuss how to leverage framing effects to help people enhance their productivity, and how to design successful productivity monitoring tool.

As visualizations are increasingly used as a storytelling medium for the general public, it becomes important to help people learn how to understand visualizations. Prior studies indicate that interactive multimedia learning environments can increase the effectiveness of learning [11]. To investigate the efficacy of the multimedia learning environments for data visualization education, we compared four online learning approaches 1) baseline (i.e., no tutorial), 2) static tutorial, 3) video tutorial, and 4) interactive tutorial-through a crowdsourced user study. We measured participants' learning outcomes in using parallel coordinates with 18 tasks. Results show that participants with the interactive condition achieved higher scores than those with the static and baseline conditions, and reported that they had a more engaging experience than those with the static condition.

We propose to conduct a workshop on the topic of Immersive Analytics: a new multidisciplinary initiative to explore future interaction technologies for data analytics. Immersive Analytics aims to bring together researchers in Information Visualisation, Visual Analytics, Virtual and Augmented Reality and Natural User Interfaces. http://immersiveanalytics.net

The Slide and Canvas metaphors are two ways of helping people create visual aids for oral presentations. Although such physical metaphors help both authors and audiences make sense of material, they also constrain authoring in ways that can negatively impact presentation delivery. In this article, we derive heuristics for the design of presentation media that are independent of any underlying physical metaphors. We use these heuristics to craft a new kind of presentation medium called SlideSpace—one that combines hierarchical outlines, content collections, and design rules to automate the real-time, outline-driven synthesis of hybrid Slide-Canvas visuals. Through a qualitative study of SlideSpace use, we validate our heuristics and demonstrate that such a hybrid presentation medium can combine the advantages of existing systems while mitigating their drawbacks. Overall, we show how a heuristic design approach helped us challenge entrenched physical metaphors to create a fundamentally digital presentation medium with the potential to transform the activities of authoring, delivering, and viewing presentations.

In this work, we advance research efforts in combining the casual sketching approach of whiteboards with the machine's computing power. We present SketchInsight, a system that applies the familiar and collaborative features of a whiteboard interface to the accurate data exploration capabilities of interactive visualizations. SketchInsight enables data analysis with more fluid interaction, allowing people to visually explore their data by drawing simple charts and directly manipulating them. In addition, we report results from a qualitative study conducted to evaluate user experience in exploring data with SketchInsight, expanding our understanding on how people use a pen- and touch-enabled digital whiteboard for data exploration. We also discuss the challenges in building a working system that supports data analytic capabilities with pen and touch interaction and freeform annotation.

Model building in machine learning is an iterative process. The performance analysis and debugging step typically involves a disruptive cognitive switch from model building to error analysis, discouraging an informed approach to model building. We present ModelTracker, an interactive visualization that subsumes information contained in numerous traditional summary statistics and graphs while displaying example-level performance and enabling direct error examination and debugging. Usage analysis from machine learning practitioners building real models with ModelTracker over six months shows ModelTracker is used often and throughout model building. A controlled experiment focusing on ModelTracker's debugging capabilities shows participants prefer ModelTracker over traditional tools without a loss in model performance.

Data videos, motion graphics that incorporate visualizations about facts, are increasingly gaining popularity as a means of telling stories with data. However, very little is systematically recorded about (a) what elements are featured in data videos and (b) the processes used to create them. In this article, we provide initial insights to build this knowledge. We first report on a qualitative analysis of 50 professionally designed data videos, extracting and exposing their most salient constituents. Second, we report on a series of workshops with experienced storytellers from cinematography, graphics design and screenplay writing. We provided them with a set of data facts and visualizations and observed them create storyboards for data videos. From these exploratory studies, we derive broader implications for the design of an authoring tool to enable a wide audience to create data videos. Our findings highlight the importance of providing a flexible tool supporting a non-linear creation process and allowing users to iteratively go back to different phases of the process.

Manual tracking of health behaviors affords many benefits, including increased awareness and engagement. However, the capture burden makes long-term manual tracking challenging. In this study on sleep tracking, we examine ways to reduce the capture burden of manual tracking while leveraging its benefits. We report on the design and evaluation of SleepTight, a low-burden, self-monitoring tool that leverages the Android's widgets both to reduce the capture burden and to improve access to information. Through a four-week deployment study (N = 22), we found that participants who used SleepTight with the widgets enabled had a higher sleep diary compliance rate (92%) than participants who used SleepTight without the widgets (73%). In addition, the widgets improved information access and encouraged self-reflection. We discuss how to leverage widgets to help people collect more data and improve access to information, and more broadly, how to design successful manual self-monitoring tools that support self-reflection.

Showing a live software demonstration during a talk can be engaging, but it is often not easy: presenters may struggle with (or worry about) unexpected software crashes and encounter issues such as mismatched screen resolutions or faulty network connectivity. Furthermore, it can be difficult to recall the steps to show while talking and operating the system all at the same time. An alternative is to present with pre-recorded screencast videos. It is, however, challenging to precisely match the narration to the video when using existing video players. We introduce DemoWiz, a video presentation system that provides an increased awareness of upcoming actions through glanceable visualizations. DemoWiz supports better control of timing by overlaying visual cues and enabling lightweight editing. A user study shows that our design significantly improves the presenters' perceived ease of narration and timing compared to a system without visualizations that was similar to a standard playback control. Furthermore, nine (out of ten) participants preferred DemoWiz over the standard playback control with the last expressing no preference.

Researchers have studied how people use self-tracking technologies and discovered a long list of barriers including lack of time and motivation as well as difficulty in data integration and interpretation. Despite the barriers, an increasing number of Quantified-Selfers diligently track many kinds of data about themselves, and some of them share their best practices and mistakes through Meetup talks, blogging, and conferences. In this work, we aim to gain insights from these "extreme users," who have used existing technologies and built their own workarounds to overcome different barriers. We conducted a qualitative and quantitative analysis of 52 video recordings of Quantified Self Meetup talks to understand what they did, how they did it, and what they learned. We highlight several common pitfalls to self-tracking, including tracking too many things, not tracking triggers and context, and insufficient scientific rigor. We identify future research efforts that could help make progress toward addressing these pitfalls. We also discuss how our findings can have broad implications in designing and developing self-tracking technologies.

Conveying a narrative with visualizations often requires choosing an order in which to present visualizations. While evidence exists that narrative sequencing in traditional stories can affect comprehension and memory, little is known about how sequencing choices affect narrative visualization. We consider the forms and reactions to sequencing in narrative visualization presentations to provide a deeper understanding with a focus on linear, 'slideshow-style' presentations. We conduct a qualitative analysis of 42 professional narrative visualizations to gain empirical knowledge on the forms that structure and sequence take. Based on the results of this study we propose a graph-driven approach for automatically identifying effective sequences in a set of visualizations to be presented linearly. Our approach identifies possible transitions in a visualization set and prioritizes local (visualization-to-visualization) transitions based on an objective function that minimizes the cost of transitions from the audience perspective. We conduct two studies to validate this function. We also expand the approach with additional knowledge of user preferences for different types of local transitions and the effects of global sequencing strategies on memory, preference, and comprehension. Our results include a relative ranking of types of visualization transitions by the audience perspective and support for memory and subjective rating benefits of visualization sequences that use parallelism as a structural device. We discuss how these insights can guide the design of narrative visualization and systems that support optimization of visualization sequence.

Presenting and communicating insights to an audience-telling a story-is one of the main goals of data exploration. Even though visualization as a storytelling medium has recently begun to gain attention, storytelling is still underexplored in information visualization and little research has been done to help people tell their stories with data. To create a new, more engaging form of storytelling with data, we leverage and extend the narrative storytelling attributes of whiteboard animation with pen and touch interactions. We present SketchStory, a data-enabled digital whiteboard that facilitates the creation of personalized and expressive data charts quickly and easily. SketchStory recognizes a small set of sketch gestures for chart invocation, and automatically completes charts by synthesizing the visuals from the presenter-provided example icon and binding them to the underlying data. Furthermore, SketchStory allows the presenter to move and resize the completed data charts with touch, and filter the underlying data to facilitate interactive exploration. We conducted a controlled experiment for both audiences and presenters to compare SketchStory with a traditional presentation system, Microsoft PowerPoint. Results show that the audience is more engaged by presentations done with SketchStory than PowerPoint. Eighteen out of 24 audience participants preferred SketchStory to PowerPoint. Four out of five presenter participants also favored SketchStory despite the extra effort required for presentation.

We harness the ability of people to perceive and interact with visual patterns in order to enhance the performance of a machine learning method. We show how we can collect evidence about how people optimize the parameters of an ensemble classification system using a tool that provides a visualization of misclassification costs. Then, we use these observations about human attempts to minimize cost in order to extend the performance of a state-of-the-art ensemble classification system. The study highlights opportunities for learning from evidence collected about human problem solving to refine and extend automated learning and inference.

Problem-solving procedures have been typically aimed at achieving well-defined goals or satisfying straightforward preferences. However, learners and solvers may often generate rich multiattribute results with procedures guided by sets of controls that define different dimensions of quality. We explore methods that enable people to explore and express preferences about the operation of classification models in supervised multiclass learning. We leverage a leave-one-out confusion matrix that provides users with views and real-time controls of a model space. The approach allows people to consider in an interactive manner the global implications of local changes in decision boundaries. We focus on kernel classifiers and show the effectiveness of the methodology on a variety of tasks.

When exploiting the power of node-link diagrams to represent real-world data such as web structures, airline routes, electrical, telecommunication and social networks, link congestion frequently arises. Such areas in the diagram---with dense, overlapping links---are not readable connectivity, node shapes, labels, and contextual information are obscured. In response, graph-layout research has begun to consider the modification of link shapes with techniques such as link routing and bundling. In this paper, we delve into the interactive techniques afforded by variant use of link curvature, delineating a six-dimensional design space that is populated by four families of interactive techniques: bundling, fanning, magnets, and legends. Our taxonomy encompasses existing techniques and reveals several novel link interactions. We describe the implementation of these techniques and illustrate their potential for exploring dense graphs with multiple types of links.

Exploring large network datasets, such as scientific collaboration networks, is challenging because they often contain a large number of nodes and edges in several types and with multiple attributes. Analyses of such networks are often long and complex, and may require several sessions by multiple users. Therefore, it is often difficult for users to recall their own exploration history or share it with others. We introduce GraphTrail, an interactive visualization for analyzing networks through exploration of node and edge aggregates that captures users' interactions and integrates this history directly in the exploration workspace. To facilitate large network analysis, GraphTrail integrates aggregation with familiar charts, drag-and-drop interaction on a canvas, and a novel pivoting mechanism for transitioning between aggregates. Through a three-month field study with a team of archeologists and a qualitative lab study with ten users, we demonstrate the effectiveness of our design and the benefits of integrated exploration history, including analysis comprehension, insight discovery, and exploration recall.

Dictation using speech recognition could potentially serve as an efficient input method for touchscreen devices. However, dictation systems today follow a mentally disruptive speech interaction model: users must first formulate utterances and then produce them, as they would with a voice recorder. Because utterances do not get transcribed until users have finished speaking, the entire output appears and users must break their train of thought to verify and correct it. In this paper, we introduce Voice Typing, a new speech interaction model where users' utterances are transcribed as they produce them to enable real-time error identification. For fast correction, users leverage a marking menu using touch gestures. Voice Typing aspires to create an experience akin to having a secretary type for you, while you monitor and correct the text. In a user study where participants composed emails using both Voice Typing and traditional dictation, they not only reported lower cognitive demand for Voice Typing but also exhibited 29% relative reduction of user corrections. Overall, they also preferred Voice Typing.

Tools exist for people to create visualizations with their data; however, they are often designed for programmers or they restrict less technical people to pre-defined templates. This can make creating novel, custom visualizations difficult for the average person. For example, existing tools typically do not support syntax or interaction techniques that are natural to end users. To explore how to support a more natural production of data visualizations by end users, we conducted an exploratory study to illuminate the structure and content of the language employed by end users when describing data visualizations. We present our findings from the study and discuss their design implications for future visualization languages and toolkits.

The importance of interaction to Information Visualization (InfoVis) and, in particular, of the interplay between interactivity and cognition is widely recognized [12, 15, 32, 55, 70]. This interplay, combined with the demands from increasingly large and complex datasets, is driving the increased significance of interaction in InfoVis. In parallel, there have been rapid advances in many facets of interaction technologies. However, InfoVis interactions have yet to take full advantage of these new possibilities in interaction technologies, as they largely still employ the traditional desktop, mouse, and keyboard setup of WIMP (Windows, Icons, Menus, and a Pointer) interfaces. In this paper, we reflect more broadly about the role of more "natural" interactions for InfoVis and provide opportunities for future research. We discuss and relate general HCI interaction models to existing InfoVis interaction classifications by looking at interactions from a novel angle, taking into account the entire spectrum of interactions. Our discussion of InfoVis-specific interaction design considerations helps us identify a series of underexplored attributes of interaction that can lead to new, more "natural," interaction techniques for InfoVis.

Current interfaces for common information visualizations such as bar graphs, line graphs, and scatterplots usually make use of the WIMP (Windows, Icons, Menus and a Pointer) interface paradigm with its frequently discussed problems of multiple levels of indirection via cascading menus, dialog boxes, and control panels. Recent advances in interface capabilities such as the availability of pen and touch interaction challenge us to re-think this and investigate more direct access to both the visualizations and the data they portray. We conducted a Wizard of Oz study to explore applying pen and touch interaction to the creation of information visualization interfaces on interactive whiteboards without implementing a plethora of recognizers. Our wizard acted as a robust and flexible pen and touch recognizer, giving participants maximum freedom in how they interacted with the system. Based on our qualitative analysis of the interactions our participants used, we discuss our insights about pen and touch interactions in the context of learnability and the interplay between pen and touch gestures. We conclude with suggestions for designing pen and touch enabled interactive visualization interfaces.

Network alarm triage refers to grouping and prioritizing a stream of low-level device health information to help operators find and fix problems. Today, this process tends to be largely manual because existing tools cannot easily evolve with the network. We present CueT, a system that uses interactive machine learning to learn from the triaging decisions of operators. It then uses that learning in novel visualizations to help them quickly and accurately triage alarms. Unlike prior interactive machine learning systems, CueT handles a highly dynamic environment where the groups of interest are not known a-priori and evolve constantly. A user study with real operators and data from a large network shows that CueT significantly improves the speed and accuracy of alarm triage compared to the network's current practice.

Visions of smart homes have long caught the attention of researchers and considerable effort has been put toward enabling home automation. However, these technologies have not been widely adopted despite being available for over three decades. To gain insight into this state of affairs, we conducted semi-structured home visits to 14 households with home automation. The long term experience, both positive and negative, of the households we interviewed illustrates four barriers that need to be addressed before home automation becomes amenable to broader adoption. These barriers are high cost of ownership, inflexibility, poor manageability, and difficulty achieving security. Our findings also provide several directions for further research, which include eliminating the need for structural changes for installing home automation, providing users with simple security primitives that they can confidently configure, and enabling composition of home devices.

Computers today make it easy for people to scatter copies and versions of digital items across their file systems, but do little to help people manage the resulting mess. In this paper, we introduce the concept of a copy-aware computing ecosystem, inspired by a vision of computing when systems track and surface copy relationships between files. Based on two deployments of a copy-aware software prototype and in-depth interviews with individuals in collaborative relationships, we present our findings on the origins of copies and the barriers to eliminating them, but offer a promising solution based on the set of files that together represent a user's conceptual view of a document - the versionset. We show that the versionset is viable to infer, and we draw upon user activity logs and feedback on personalized views of versionsets to distill guidelines for the factors that define a versionset. We conclude by enumerating the many PIM user experiences that could be transformed as a result.

We propose a novel system to analyze gestural and nonverbal cues of participants in video conferencing. These cues have previously been referred to as “honest signals” and are usually associated with the underlying cognitive state of the participants. The presented system analyzes a set of audio-visual, non-linguistic features in real time from the audio and video streams of two participants in a video conference. We show how these features can be used to compute indicators of the overall quality and type of conversation being held. The system also provides visual feedback to the participants, who then have the choice of modifying their conversational style in order to achieve the desired outcome of the video conference. Experiments on real-life data show that the system can predict the type of conversation with high accuracy using the non-linguistic signals only. Qualitative user studies highlight the positive effects of increased awareness amongst the participants about their own gestural and non-verbal cues.

To increase its credibility and preserve the trust of its readers. Wikipedia needs to ensure a good quality of its articles. To that end, it is critical for Wikipedia administrators to be aware of contributors' editing activity to monitor vandalism, encourage reliable contributors to work on specific articles, or find mentors for new contributors. In this paper, we present iChase, a novel interactive visualization tool to provide administrators with better awareness of editing activities on Wikipedia. Unlike the currently used visualizations that provide only page-centric information. iChase visualizes the trend of activities for two entity types; articles and contributors. iChase is based on two heatmaps (one for each entity type) synchronized to one timeline. It allows users to interactively explore the history of changes by drilling down into specific articles and contributors, or time points to access the details of the changes. We also present a case study to illustrate how iChase can be used to monitor editing activities of Wikipedia authors, as well as a usability study. We conclude by discussing the strengths and weaknesses of iChase.

Interest has been growing within HCI on the use of machine learning and reasoning in applications to classify such hidden states as user intentions, based on observations. HCI researchers with these interests typically have little expertise in machine learning and often employ toolkits as relatively fixed "black boxes" for generating statistical classifiers. However, attempts to tailor the performance of classifiers to specific application requirements may require a more sophisticated understanding and custom-tailoring of methods. We present ManiMatrix, a system that provides controls and visualizations that enable system builders to refine the behavior of classification systems in an intuitive manner. With ManiMatrix, users directly refine parameters of a confusion matrix via an interactive cycle of re-classification and visualization. We present the core methods and evaluate the effectiveness of the approach in a user study. Results show that users are able to quickly and effectively modify decision boundaries of classifiers to tai-lor the behavior of classifiers to problems at hand.

Hierarchical structures with large fan-outs are hard to browse and understand. In the conventional node-link tree visualization, the screen quickly becomes overcrowded as users open nodes that have too many child nodes to fit in one screen. To address this problem, we propose two extensions to the conventional node-link tree visualization: a list view with a scrollbar and a multi-column interface. We compared them against the conventional tree visualization interface in a user study. Results show that users are able to browse and understand the tree structure faster with the multi-column interface than the other two interfaces. Overall, they also liked the multi-column better than others.

Current business conditions have given rise to distributed teams that are mostly collocated except for one remote member. These "hub-and-satellite" teams face the challenge of the satellite colleague being out-of-sight and out-of-mind. We developed a telepresence device, called an Embodied Social Proxy (ESP), which represents the satellite coworker 24x7. Beyond using ESPs in our own group, we deployed an ESP in four product teams within our company for six weeks. We studied how ESP was used through ethnographic observations, surveys, and usage log data. ESP not only increased the satellite worker's ability to fully participate in meetings, it also increased the hub's attention and affinity towards the satellite. The continuous physical presence of ESP in each team improved the interpersonal social connections between hub and satellite colleagues.

We argue that heterogeneity is hindering technological innovation in the home---homes differ in terms of their devices and how those devices are connected and used. To abstract these differences, we propose to develop a home-wide operating system. A HomeOS can simplify application development and let users easily add functionality by installing new devices or applications. The development of such an OS is an inherently inter-disciplinary exercise. Not only must the abstractions meet the usual goals of being efficient and easy to program, but the underlying primitives must also match how users want to manage and secure their home. We describe the preliminary design of HomeOS and our experience with developing applications for it.

Abstract Interactive information visualization systems rely on widgets to allow users to interact with the data and modify the representation. We define interactive legends as a class of controls combining the visual representation of static legends and interaction mechanisms of widgets. As interactive legends start to appear in popular websites, we categorize their designs for common data types and evaluate their effectiveness compare to standard widgets. Results suggest that 1) interactive legends can lead to faster perception of the mapping between data values and visual encodings and 2) interaction time is affected differently depending on the data type. Additionally, our study indicates superiority both in terms of perception and interaction of ordinal controls over numerical ones. Numerical techniques are mostly used in today's systems. By providing solutions to allowing users to modify ranges interactively, we believe that interactive legends make it possible to increase the use of ordinal techniques for visual exploration.

Among the multifarious tag-clouding techniques, Wordle stands out to the community by providing an aesthetic layout, eliciting the emergence of the participatory culture and usage of tag-clouding in the artistic creations. In this paper, we introduce ManiWordle, a Wordle-based visualization tool that revamps interactions with the layout by supporting custom manipulations. ManiWordle allows people to manipulate typography, color, and composition not only for the layout as a whole, but also for the individual words, enabling them to have better control over the layout result. We first describe our design rationale along with the interaction techniques for tweaking the layout. We then present the results both from the preliminary usability study and from the comparative study between ManiWordle and Wordle. The results suggest that ManiWordle provides higher user satisfaction and an efficient method of creating the desired "art work," harnessing the power behind the ever-increasing popularity of Wordle.

Tag clouds have proliferated over the web over the last decade. They provide a visual summary of a collection of texts by visually depicting the tag frequency by font size. In use, tag clouds can evolve as the associated data source changes over time. Interesting discussions around tag clouds often include a series of tag clouds and consider how they evolve over time. However, since tag clouds do not explicitly represent trends or support comparisons, the cognitive demands placed on the person for perceiving trends in multiple tag clouds are high. In this paper, we introduce SparkClouds, which integrate sparklines into a tag cloud to convey trends between multiple tag clouds. We present results from a controlled study that compares SparkClouds with two traditional trend visualizations—multiple line graphs and stacked bar charts—as well as Parallel Tag Clouds. Results show that SparkClouds ability to show trends compares favourably to the alternative visualizations.

Previous research has shown that faceted browsing is effective and enjoyable in searching and browsing large collections of data. In this work, we explore the efficacy of interactive visualization systems in supporting exploration and sensemaking within faceted datasets. To do this, we developed an interactive visualization system called FacetLens, which exposes trends and relationships within faceted datasets. FacetLens implements linear facets to enable users not only to identify trends but also to easily compare several trends simultaneously. Furthermore, it offers pivot operations to allow users to navigate the faceted dataset using relationships between items. We evaluate the utility of the system through a description of insights gained while experts used the system to explore the CHI publication repository as well as a database of funding grant data, and report a formative user study that identified usability issues.

Machine learning is an increasingly used computational tool within human-computer interaction research. While most researchers currently utilize an iterative approach to refining classifier models and performance, we propose that ensemble classification techniques may be a viable and even preferable alternative. In ensemble learning, algorithms combine multiple classifiers to build one that is superior to its components. In this paper, we present EnsembleMatrix, an interactive visualization system that presents a graphical view of confusion matrices to help users understand relative merits of various classifiers. EnsembleMatrix allows users to directly interact with the visualizations in order to explore and build combination models. We evaluate the efficacy of the system and the approach in a user study. Results show that users are able to quickly combine multiple classifiers operating on multiple feature sets to produce an ensemble classifier with accuracy that approaches best-reported performance classifying images in the CalTech-101 dataset.

As users enter web queries, real-time query expansion (RTQE) interfaces offer suggestions based on an index garnered from query logs. In selecting a suggestion, users can potentially reduce keystrokes, which can be very beneficial on mobile devices with deficient input means. Unfortunately, RTQE interfaces typically provide little assistance when only parts of an intended query appear among the suggestion choices. In this paper, we introduce Phrase Builder, an RTQE interface that reduces keystrokes by facilitating the selection of individual query words and by leveraging back-off query techniques to offer completions for out-of-index queries. We describe how we implemented a small memory footprint index and retrieval algorithm, and discuss lessons learned from three versions of the user interface, which was iteratively designed through user studies. Compared to standard auto-completion and typing, the last version of Phrase Builder reduced more keystrokes-per-character, was perceived to be faster, and was overall preferred by users.

UNLABELLED: Evolutionary biologists are often interested in finding correlations among biological traits across a number of species, as such correlations may lead to testable hypotheses about the underlying function. Because some species are more closely related than others, computing and visualizing these correlations must be done in the context of the evolutionary tree that relates species. In this note, we introduce PhyloDet (short for PhyloDetective), an evolutionary tree visualization tool that enables biologists to visualize multiple traits mapped to the tree. AVAILABILITY: http://research.microsoft.com/cue/phylodet/

The research presented in this paper compares user-generated and automatic graph layouts. Following the methods suggested by van Ham et al. (2008), a group of users generated graph layouts using both multi-touch interaction on a tabletop display and mouse interaction on a desktop computer. Users were asked to optimize their layout for aesthetics and analytical tasks with a social network. We discuss characteristics of the user-generated layouts and interaction methods employed by users in this process. We then report on a web-based study to compare these layouts with the output of popular automatic layout algorithms. Our results demonstrate that the best of the user-generated layouts performed as well as or better than the physics-based layout. Orthogonal and circular automatic layouts were found to be considerably less effective than either the physics-based layout or the best of the user-generated layouts. We highlight several attributes of the various layouts that led to high accuracy and improved task completion time, as well as aspects in which traditional automatic layout methods were unsuccessful for our tasks.

A widespread use of high-throughput gene expression analysis techniques enabled the biomedical research community to share a huge body of gene expression datasets in many public databases on the web. However, current gene expression data repositories provide static representations of the data and support limited interactions. This hinders biologists from effectively exploring shared gene expression datasets. Responding to the growing need for better interfaces to improve the utility of the public datasets, we have designed and developed a new web-based visual interface entitled GeneShelf (http://bioinformatics.cnmcresearch.org/GeneShelf). It builds upon a zoomable grid display to represent two categorical dimensions. It also incorporates an augmented timeline with expandable time points that better shows multiple data values for the focused time point by embedding bar charts. We applied GeneShelf to one of the largest microarray datasets generated to study the progression and recovery process of injuries at the spinal cord of mice and rats. We present a case study and a preliminary qualitative user study with biologists to show the utility and usability of GeneShelf.

The difference between using one mobile phone and another can feel like learning a new language based on our extensive experience designing mobile applications for spatial data navigation, faceted search, and glanceable information, we have developed design principles for unifying the various aspects of the internet connected mobile phone ("smartphone") user experience.

Uncertainty in data occurs in domains ranging from natural science to medicine to computer science. By developing ways to include uncertainty in our information visualizations we can provide more accurate visual depictions of critical datasets. One hindrance to visualizing uncertainty is that we must first understand what uncertainty is and how it is expressed by users. We reviewed existing work from several domains on uncertainty and conducted qualitative interviews with 18 people from diverse domains who self-identified as working with uncertainty. We created a classification of uncertainty representing commonalities in uncertainty across domains and that will be useful for developing appropriate visualizations of uncertainty.

Internet usage on mobile devices continues to grow as users seek anytime, anywhere access to information. Because users frequently search for businesses, directory assistance has been the focus of many voice search applications utilizing speech as the primary input modality. Unfortunately, mobile settings often contain noise which degrades performance. As such, we present Search Vox, a mobile search interface that not only facilitates touch and text refinement whenever speech fails, but also allows users to assist the recognizer via text hints. Search Vox can also take advantage of any partial knowledge users may have about the business listing by letting them express their uncertainty in an intuitive way using verbal wildcards. In simulation experiments conducted on real voice search data, leveraging multimodal refinement resulted in a 28% relative reduction in error rate. Providing text hints along with the spoken utterance resulted in even greater relative reduction, with dramatic gains in recovery for each additional character.

UNLABELLED: We developed an interactive gene ontology (GO) browser named GOTreePlus that superimposes annotation information over GO structures. It can facilitate the identification of important GO terms through interactive visualization of them in the GO structure. The interactive pie chart summarizing an annotation distribution for a selected GO term provides users with a succinct context-sensitive overview of their experimental results. We tested our GOTreePlus using a proteome profiling dataset obtained on differentiation of retinal pigment epithelial cells where 399 proteins were quantified. AVAILABILITY: http://bioinformatics.cnmcresearch.org/GOTreePlus/.

Animation has been used to show trends in multi-dimensional data. This technique has recently gained new prominence for presentations, most notably with Gapminder Trendalyzer. In Trendalyzer, animation together with interesting data and an engaging presenter helps the audience understand the results of an analysis of the data. It is less clear whether trend animation is effective for analysis. This paper proposes two alternative trend visualizations that use static depictions of trends: one which shows traces of all trends overlaid simultaneously in one display and a second that uses a small multiples display to show the trend traces side-by-side. The paper evaluates the three visualizations for both analysis and presentation. Results indicate that trend animation can be challenging to use even for presentations; while it is the fastest technique for presentation and participants find it enjoyable and exciting, it does lead to many participant errors. Animation is the least effective form for analysis; both static depictions of trends are significantly faster than animation, and the small multiples display is more accurate.

Scientific problem solving often involves concordance (or discordance) analysis among the result sets from different approaches. For example, different scientific analysis methods with the same samples often lead to different or even conflicting conclusions. To reach a more judicious conclusion, it is crucial to consider different perspectives by checking concordance among those result sets by different methods. In this paper, we present an interactive visualization tool called ConSet, where users can effectively examine relationships among multiple sets at once. ConSet provides an overview using an improved permutation matrix to enable users to easily identify relationships among sets with a large number of elements. Not only do we use a standard Venn diagram, we also introduce a new diagram called Fan diagram that allows users to compare two or three sets without any inconsistencies that may exist in Venn diagrams. A qualitative user study was conducted to evaluate how our tool works in comparison with a traditional set visualization tool based on a Venn diagram. We observed that ConSet enabled users to complete more tasks with fewer errors than the traditional interface did and most users preferred ConSet.

Online pick'em games, such as the recent NCAA college basketball March Madness tournament, form a large and rapidly growing industry. In these games, players make predictions on a tournament bracket that defines which competitors play each other and how they proceed toward a single champion. Throughout the course of the tournament, players monitor the brackets to track progress and to compare predictions made by multiple players. This is often a complex sensemaking task. The classic bracket visualization was designed for use on paper and utilizes an incrementally additive system in which the winner of each match-up is rewritten in the next round as the tournament progresses. Unfortunately, this representation requires a significant amount of space and makes it relatively difficult to get a quick overview of the tournament state since competitors take arbitrary paths through the static bracket. In this paper, we present AdaptiviTree, a novel visualization that adaptively deforms the representation of the tree and uses its shape to convey outcome information. AdaptiviTree not only provides a more compact and understandable representation, but also allows overlays that display predictions as well as other statistics. We describe results from a lab study we conducted to explore the efficacy of AdaptiviTree, as well as from a deployment of the system in a recent real-world sports tournament.

Despite extensive research, it is still difficult to produce effective interactive layouts for large graphs. Dense layout and occlusion make food webs, ontologies, and social networks difficult to understand and interact with. We propose a new interactive Visual Analytics component called TreePlus that is based on a tree-style layout. TreePlus reveals the missing graph structure with visualization and interaction while maintaining good readability. To support exploration of the local structure of the graph and gathering of information from the extensive reading of labels, we use a guiding metaphor of "Plant a seed and watch it grow." It allows users to start with a node and expand the graph as needed, which complements the classic overview techniques that can be effective at (but often limited to) revealing clusters. We describe our design goals, describe the interface, and report on a controlled user study with 28 participants comparing TreePlus with a traditional graph interface for six tasks. In general, the advantage of TreePlus over the traditional interface increased as the density of the displayed data increased. Participants also reported higher levels of confidence in their answers with TreePlus and most of them preferred TreePlus.

Fishnet is a web browser that always displays web pages in their entirety, independent of their size. Fishnet accomplishes this by using a fisheye view, i.e. by showing a focus region at readable scale while spatially compressing page content above and below that region. Fishnet offers search term highlighting, and assures that those terms are readable by using "popouts". This allows users to visually scan search results within the entire page without scrolling.The scope of this paper is twofold. First, we present fishnet as a novel way of viewing the results of highlighted search and we discuss the design space. Second, we present a user study that helps practitioners determine which visualization technique--- fisheye view, overview, or regular linear view---to pick for which type of visual search scenario.

Biodiversity databases have recently become widely available to the public and to other researchers. To retrieve information from these resources, users must understand the underlying data schemas even though they often are not content experts. Many other domains share this problem. We developed an interface, TaxonTree, to visualize the taxonomic hierarchy of animal names. We applied integrated searching and browsing so that users need not have complete knowledge either of appropriate keywords or the organization of the data. Our qualitative user study of TaxonTree in an undergraduate course is the first to describe usage patterns in the biodiversity domain. We found that tree-based interaction and visualization aided users ' understanding of the data. Most users approached biodiversity data by browsing, using common, general knowledge rather than the scientific keyword expertise necessary to search using traditional interfaces. Users with different levels of interest in the domain had different interaction preferences.

Abstract Motivation: Despite substantial efforts to develop and populate the back-ends of biological databases, front-ends to these systems often rely on taxonomic expertise. This research applies techniques from human–computer interaction research to the biodiversity domain. Results: We developed an interactive node-link tool, TaxonTree, illustrating the value of a carefully designed interaction model, animation, and integrated searching and browsing towards retrieval of biological names and other information. Users tested the tool using a new, large integrated dataset of animal names with phylogenetic-based and classification-based tree structures. These techniques also translated well for a tool, DoubleTree, to allow comparison of trees using coupled interaction. Our approaches will be useful not only for biological data but as general portal interfaces. Availability: Working versions of TaxonTree and DoubleTree and video demonstrations of DoubleTree are available at http://www.cs.umd.edu/hcil/biodiversity Supplementary information: All datasets and folk tree illustration are available at http://www.cs.umd.edu/hcil/biodiversity

With recent troubles in U.S. elections, there has been a nationwide push to update voting systems. Municipalities are investing heavily in electronic voting systems, many of which use a touch screen. These systems offer the promise of faster and more accurate voting, but the current reality is that they are fraught with usability and systemic problems. This paper surveys issues relating to usability of electronic voting systems and reports on a series of studies, including one with 415 voters using new systems that the State of Maryland purchased. Our analysis shows these systems work well, but have several problems, and many voters have concerns about them.