Abstract People craft choropleth maps to monitor, analyze, and understand spatially distributed data. Recent visualization work has addressed several known biases in choropleth maps by developing new model‐ and metrics‐ based approaches (e.g. Bayesian surprise). However, effective use of these techniques requires extensive parameter setting and tuning, making them difficult or impossible for users without substantial technical skills. In this paper we describe SurpriseExplora, which addresses this gap through direct manipulation techniques for re‐targeting a Bayesian surprise model's scope and parameters. We present three use cases to illustrate the capabilities of SurpriseExplora, showing for example how models calculated at a national level can obscure key findings that can be revealed through interaction sequences common to map visualizations (e.g. zooming), and how augmenting funnel‐plot visualizations with interactions that adjust underlying models can account for outliers or skews in spatial datasets. We evaluate SurpriseExplora through an expert review with visualization researchers and practitioners. We conclude by discussing how SurpriseExplora uncovers new opportunities for sense‐making within the broader ecosystem of map visualizations, as well as potential empirical studies with non‐expert populations. Code and demo video available at https://osf.io/7m89w/

Interactive visualizations are powerful tools for Exploratory Data Analysis (EDA), but how do they affect the observations analysts make about their data? We conducted a qualitative experiment with 13 professional data scientists analyzing two datasets with Jupyter notebooks, collecting a rich dataset of interaction traces and think-aloud utterances. By qualitatively coding participant utterances, we introduce a formalism that describes EDA as a sequence of analysis states, where each state is comprised of either a representation an analyst constructs (e.g., the output of a data frame, an interactive visualization, etc.) or an observation the analyst makes (e.g., about missing data, the relationship between variables, etc.). By applying our formalism to our dataset, we identify that interactive visualizations, on average, lead to earlier and more complex insights about relationships between dataset attributes compared to static visualizations. Moreover, by calculating metrics such as revisit count and representational diversity, we uncover that some representations serve more as "planning aids" during EDA rather than tools strictly for hypothesis-answering. We show how these measures help identify other patterns of analysis behavior, such as the "80-20 rule", where a small subset of representations drove the majority of observations. Based on these findings, we offer design guidelines for interactive exploratory analysis tooling and reflect on future directions for studying the role that visualizations play in EDA.

Leaving the context of visualizations invisible can have negative impacts on understanding and transparency. While common wisdom suggests that recontextualizing visualizations with metadata (e.g., disclosing the data source or instructions for decoding the visualizations' encoding) may counter these effects, the impact remains largely unknown. To fill this gap, we conducted two experiments. In Experiment 1, we explored how chart type, topic, and user goal impacted which categories of metadata participants deemed most relevant. We presented 64 participants with four real-world visualizations. For each visualization, participants were given four goals and selected the type of metadata they most wanted from a set of 18 types. Our results indicated that participants were most interested in metadata which explained the visualization's encoding for goals related to understanding and metadata about the source of the data for assessing trustworthiness. In Experiment 2, we explored how these two types of metadata impact transparency, trustworthiness and persuasiveness, information relevance, and understanding. We asked 144 participants to explain the main message of two pairs of visualizations (one with metadata and one without); rate them on scales of transparency and relevance; and then predict the likelihood that they were selected for a presentation to policymakers. Our results suggested that visualizations with metadata were perceived as more thorough than those without metadata, but similarly relevant, accurate, clear, and complete. Additionally, we found that metadata did not impact the accuracy of the information extracted from visualizations, but may have influenced which information participants remembered as important or interesting.

As more people rely on visualization to inform their personal and collective decisions, researchers have focused on a broader range of audiences, including “novices.” But successfully applying, interrogating, or advancing visualization research for novices demands a clear understanding of what “novice” means in theory and practice. Misinterpreting who a “novice” is could lead to misapplying guidelines and overgeneralizing results. In this paper, we investigated how visualization researchers define novices and how they evaluate visualizations intended for novices. We analyzed 79 visualization papers that used “novice,” “non-expert,” “laypeople,” or “general public” in their titles or abstracts. We found ambiguity within papers and disagreement between papers regarding what defines a novice. Furthermore, we found a mismatch between the broad language describing novices and the narrow population representing them in evaluations (i.e., young people, students, and US residents). We suggest directions for inclusively supporting novices in both theory and practice.

With the 2018 revision of the ACM Code of Ethics and Professional Conduct, there is a growing interest in how computer science faculty can integrate these principles into the education of future practitioners. This special session illustrates one approach by highlighting assignments that blend ethics and technology. These assignments can be used in a variety of courses, including CS1, CS2, and later courses. Presenters will provide an overview of each assignment and gather feedback from the audience. All materials, including descriptions, starter files, and guidelines for instructors, will be published at https://ethics.acm.org/SIGCSE2020.

The Nifty Assignments special session is about promoting and sharing the ideas and ready-to-use materials of successful assignments. Each presenter will introduce their assignment, give a quick demo, and describe its niche in the curriculum and its strengths and weaknesses. The presentations (and the descriptions below) merely introduce the assignment. A key part of Nifty Assignments is the mundane but vital role of distributing the materials - handouts, data files, starter code, rubrics, autograders - that make each assignment ready to adopt. Each assignment presented has complete materials freely available on the Nifty Assignments home page nifty.stanford.edu. If you have an assignment that works well and would be of interest to the CSE community, please consider applying to present at Nifty Assignments.

Many of the guidelines that inform how designers create data visualizations originate in studies that unintentionally exclude populations that are most likely to be among the 'data poor'. In this paper, we explore which factors may drive attention and trust in rural populations with diverse economic and educational backgrounds - a segment that is largely underrepresented in the data visualization literature. In 42 semi-structured interviews in rural Pennsylvania (USA), we find that a complex set of factors intermix to inform attitudes and perceptions about data visualization - including educational background, political affiliation, and personal experience. The data and materials for this research can be found at https://osf.io/uxwts/

This Workshop is intended to be a generative space for participants to create a module on ethics in a Computer Science course of their choice. Participants will choose an existing course in the typical CS curriculum; and coalesce around common themes to create a module that integrates critical reflection on ethical choices and societal impact of computing with the practice of computing. The workshop will offer a space to collaboratively work on designing an ethics module for a CS course chosen by the participants. The ethics module would integrate teaching specific concepts and practices in computing alongside a critical reflection of these practices. Ideally, these modules would be flexible enough to fit into curricula at various institutions.

The diverse and vibrant ecosystem of interactive visualizations on the web presents an opportunity for researchers and practitioners to observe and analyze how everyday people interact with data visualizations. However, existing metrics of visualization interaction behavior used in research do not fully reveal the breadth of peoples' open-ended explorations with visualizations. One possible way to address this challenge is to determine high-level goals for visualization interaction metrics, and infer corresponding features from user interaction data that characterize different aspects of peoples' explorations of visualizations. In this paper, we identify needs for visualization behavior measurement, and develop corresponding candidate features that can be inferred from users' interaction data. We then propose metrics that capture novel aspects of peoples' open-ended explorations, including exploration uniqueness and exploration pacing. We evaluate these metrics along with four other metrics recently proposed in visualization literature by applying them to interaction data from prior visualization studies. The results of these evaluations suggest that these new metrics 1) reveal new characteristics of peoples' use of visualizations, 2) can be used to evaluate statistical differences between visualization designs, and 3) are statistically independent of prior metrics used in visualization research. We discuss implications of these results for future studies, including the potential for applying these metrics in visualization interaction analysis, as well as emerging challenges in developing and selecting metrics depicting visualization explorations.

The widespread use of text-based search in user interfaces has led designers in visualization to occasionally add search functionality to their creations. Yet it remains unclear how search may impact a person's behavior. Given the unstructured context of the web, users may not have explicit information-seeking goals and designers cannot make assumptions about user attention. To bridge this gap, we observed the impact of integrating search with five visualizations across 830 online participants. In an unguided task, we find that (1) the presence of text-based search influences people's information-seeking goals, (2) search can alter the data that people explore and how they engage with it, and (3) the effects of search are amplified in visualizations where people are familiar with the underlying dataset. These results suggest that text-search in web visualizations drives users towards more diverse information seeking goals, and may be valuable in a range of existing visualization designs.

There has been a dramatic increase in computer science undergraduate research activity at colleges and universities in recent years. However, developing a research culture that is explicitly designed to empower undergraduates (student-centered research) requires different models and objectives than those traditionally employed at more research-oriented universities. The goal of this BOF is to explore what effective techniques are employed by other primarily undergraduate institutions to build a culture of research that benefits undergraduate students. Some of the key issues covered in this BOF will be: defining student-centered research and its impact (How does student-centered research differ from traditional research? What secondary effects in the classroom and community might undergraduate research have?), redefining success metrics in student-centered research (How can we capture impact beyond publications and grants? How can we define measures that align with student impact?), exploring issues of accessibility and participation (How might student-centered research change models of student selection? How might it change faculty's scope and focus of research?), and structural mechanisms to empower student-centered research (Given constraints on time and/or resources, how can faculty enable undergraduate research?). Through this BOF, we also plan to build a sustainable community of interested academics leaders (using private Google+ or Facebook group) interested to share and collaborate on future undergraduate research efforts.

As a consequence of living in a data ecosystem, we often relinquish personal information to be used in contexts in which we have no control. In this paper, we begin to examine the usability of differential privacy, a mechanism that proposes to promise privacy with a mathematical "proof" to the data donor. Do people trust this promise and adjust their privacy decisions if the interfaces through which they interact make differential privacy less opaque? In a study with 228 participants, we measured comfort, understanding, and trust using a variant of differential privacy known as Randomized Response Technique (RRT). We found that allowing individuals to see the amount of obfuscation applied to their responses increased their trust in the privacy-protecting mechanism. However, participants who associated obfuscating privacy mechanisms with deception did not make the "safest" privacy decisions, even as they demonstrated an understanding of RRT. We demonstrate that prudent privacy-related decisions can be cultivated with simple explanations of usable privacy.

Physical and digital objects often leave markers of our use. Website links turn purple after we visit them, for example, showing us information we have yet to explore. These "footprints" of interaction offer substantial benefits in information saturated environments - they enable us to easily revisit old information, systematically explore new information, and quickly resume tasks after interruption. While applying these design principles have been successful in HCI contexts, direct encodings of personal interaction history have received scarce attention in data visualization. One reason is that there is little guidance for integrating history into visualizations where many visual channels are already occupied by data. More importantly, there is not firm evidence that making users aware of their interaction history results in benefits with regards to exploration or insights. Following these observations, we propose HindSight - an umbrella term for the design space of representing interaction history directly in existing data visualizations. In this paper, we examine the value of HindSight principles by augmenting existing visualizations with visual indicators of user interaction history (e.g. How the Recession Shaped the Economy in 255 Charts, NYTimes). In controlled experiments of over 400 participants, we found that HindSight designs generally encouraged people to visit more data and recall different insights after interaction. The results of our experiments suggest that simple additions to visualizations can make users aware of their interaction history, and that these additions significantly impact users' exploration and insights.

We present Brain Automated Chorales (BACh), an adaptive brain-computer system that dynamically increases the levels of difficulty in a musical learning task based on pianists' cognitive workload measured by functional near-infrared spectroscopy. As users' cognitive workload fell below a certain threshold, suggesting that they had mastered the material and could handle more cognitive information, BACh automatically increased the difficulty of the learning task. We found that learners played with significantly increased accuracy and speed in the brain-based adaptive task compared to our control condition. Participant feedback indicated that they felt they learned better with BACh and they liked the timings of the level changes. The underlying premise of BACh can be applied to learning situations where a task can be broken down into increasing levels of difficulty.

Decades of research have repeatedly shown that people perform poorly at estimating and understanding conditional probabilities that are inherent in Bayesian reasoning problems. Yet in the medical domain, both physicians and patients make daily, life-critical judgments based on conditional probability. Although there have been a number of attempts to develop more effective ways to facilitate Bayesian reasoning, reports of these findings tend to be inconsistent and sometimes even contradictory. For instance, the reported accuracies for individuals being able to correctly estimate conditional probability range from 6% to 62%. In this work, we show that problem representation can significantly affect accuracies. By controlling the amount of information presented to the user, we demonstrate how text and visualization designs can increase overall accuracies to as high as 77%. Additionally, we found that for users with high spatial ability, our designs can further improve their accuracies to as high as 100%. By and large, our findings provide explanations for the inconsistent reports on accuracy in Bayesian reasoning tasks and show a significant improvement over existing methods. We believe that these findings can have immediate impact on risk communication in health-related fields.

A growing body of recent work has shown the feasibility of brain and body sensors as input to interactive systems. However, the interaction techniques and design decisions for their effective use are not well defined. We present a conceptual framework for considering implicit input from the brain, along with design principles and patterns we have developed from our work. We also describe a series of controlled, offline studies that lay the foundation for our work with functional near-infrared spectroscopy (fNIRS) neuroimaging, as well as our real-time platform that serves as a testbed for exploring brain-based adaptive interaction techniques. Finally, we present case studies illustrating the principles and patterns for effective use of brain data in human--computer interaction. We focus on signals coming from the brain, but these principles apply broadly to other sensor data and in domains such as aviation, education, medicine, driving, and anything involving multitasking or varying cognitive workload.

Dynamic difficulty adjustments can be used in human-computer systems in order to improve user engagement and performance. In this paper, we use functional near-infrared spectroscopy (fNIRS) to obtain passive brain sensing data and detect extended periods of boredom or overload. From these physiological signals, we can adapt a simulation in order to optimize workload in real-time, which allows the system to better fit the task to the user from moment to moment. To demonstrate this idea, we ran a laboratory study in which participants performed path planning for multiple unmanned aerial vehicles (UAVs) in a simulation. Based on their state, we varied the difficulty of the task by adding or removing UAVs and found that we were able to decrease error by 35% over a baseline condition. Our results show that we can use fNIRS brain sensing to detect task difficulty in real-time and construct an interface that improves user performance through dynamic difficulty adjustment.

The bubble cursor is a promising cursor expansion technique, improving a user's movement time and accuracy in pointing tasks. We introduce a brain-based target expansion system, which improves the efficacy of bubble cursor by increasing the expansion of high importance targets at the optimal time based on brain measurements correlated to a particular type of multitasking. We demonstrate through controlled experiments that brain-based target expansion can deliver a graded and continuous level of assistance to a user according to their cognitive state, thereby improving task and speed-accuracy metrics, even without explicit visual changes to the system. Such an adaptation is ideal for use in complex systems to steer users toward higher priority goals during times of increased demand.

We show how brain sensing can lend insight to the evaluation of visual interfaces and establish a role for fNIRS in visualization. Research suggests that the evaluation of visual design benefits by going beyond performance measures or questionnaires to measurements of the user's cognitive state. Unfortunately, objectively and unobtrusively monitoring the brain is difficult. While functional near-infrared spectroscopy (fNIRS) has emerged as a practical brain sensing technology in HCI, visual tasks often rely on the brain's quick, massively parallel visual system, which may be inaccessible to this measurement. It is unknown whether fNIRS can distinguish differences in cognitive state that derive from visual design alone. In this paper, we use the classic comparison of bar graphs and pie charts to test the viability of fNIRS for measuring the impact of a visual design on the brain. Our results demonstrate that we can indeed measure this impact, and furthermore measurements indicate that there are not universal differences in bar graphs and pie charts.