There have been extraordinary advances in our ability to collect, analyze, and interpret vast amounts of data which have transformed the fundamental nature of artificial intelligence (AI). The human aspects of AI, including how to support creativity and human insight without violating individual rights, how to address ethical concerns, and the consideration of societal impacts, have received less attention. Yet these human issues are becoming increasingly vital to the future of AI. Dr. Aragon will reflect on a 30-year career in data science and AI in industry, government, and academia, discuss what it means for AI to be both rigorous and human-centered, and speculate upon future directions for data science and AI.

The Latinx diaspora in the United States is a rapidly growing and complex demographic who face intersectional harms and marginalizations in sociotechnical systems and are currently underserved in CSCW research. While the field understands that algorithms and digital content are experienced differently by marginalized populations, more investigation is needed about how Latinx people experience social media and, in particular, visual media. In this paper, we focus on how Latinx people experience the algorithmic system of the video-sharing platform TikTok. Through a bilingual interview and visual elicitation study of 19 Latinx TikTok users and 59 survey participants, we explore how Latinx individuals experience TikTok and its Latinx content. We find Latinx TikTok users actively use platform affordances to create positive and affirming identity content feeds, but these feeds are interrupted by negative content (i.e. violence, stereotypes, linguistic assumptions) due to platform affordances that have unique consequences for Latinx diaspora users. We discuss these implications on Latinx identity and representation, introduce the concept of algorithmic identity compression, where sociotechncial systems simplify, flatten, and conflate intersection identities, resulting in compression via the loss of critical cultural data deemed unnecessary by these systems and designers of them. This study explores how Latinx individuals are particularly vulnerable to this in sociotechnical systems, such as, but not limited to, TikTok.

Raiding is a format in digital gaming that requires groups of people to collaborate and/or compete for a common goal. In 2017, the raiding format was introduced in the location-based mobile game Pokémon GO, which offers a mixed reality experience to friends and strangers coordinating for in-person raids. To understand this technology-mediated social phenomenon, we conducted over a year of participant observations, surveys with 510 players, and interviews with 25 players who raid in Pokémon GO. Using the analytical lens of Arrow, McGrath, and Berdahl's theory of small groups as complex systems, we identify global, local, and contextual dynamics in location-based raiding that support and challenge ad-hoc group formation in real life. Based on this empirical and theoretical understanding, we discuss implications to design for transparency, social affordances, and bridging gaps between global and contextual dynamics for increased positive and inclusive community interactions.

Collaborative qualitative coding often involves coders assign- ing different labels to the same instance, leading to ambiguity. We refer to such an instance of ambiguity as disagreement in coding. Analyzing reasons for such a disagreement is essential-- both for purposes of bolstering user understanding gained from coding and reinterpreting the data collaboratively, and for negotiating user-assigned labels for building effective machine learning models. We propose a conceptual definition of collective disagreement using diversity and divergence within the coding distributions. This perspective of disagreement translates to diverse coding contexts and groups of coders irrespective of discipline. We introduce two tree-based ranking metrics as standardized ways of comparing disagreements in how data instances have been coded. We empirically validate that, of the two tree-based metrics, coders' perceptions of dis- agreement match more closely with the n-ary tree metric than with the post-traversal tree metric.

Machine learning (ML) has become increasingly influential to human society, yet the primary advancements and applications of ML are driven by research in only a few computational disciplines. Even applications that affect or analyze human behaviors and social structures are often developed with limited input from experts outside of computational fields. Social scientists—experts trained to examine and explain the complexity of human behavior and interactions in the world—have considerable expertise to contribute to the development of ML applications for human-generated data, and their analytic practices could benefit from more human-centered ML methods. Although a few researchers have highlighted some gaps between ML and social sciences [51, 57, 70], most discussions only focus on quantitative methods. Yet many social science disciplines rely heavily on qualitative methods to distill patterns that are challenging to discover through quantitative data. One common analysis method for qualitative data is qualitative coding . In this article, we highlight three challenges of applying ML to qualitative coding. Additionally, we utilize our experience of designing a visual analytics tool for collaborative qualitative coding to demonstrate the potential in using ML to support qualitative coding by shifting the focus to identifying ambiguity. We illustrate dimensions of ambiguity and discuss the relationship between disagreement and ambiguity. Finally, we propose three research directions to ground ML applications for social science as part of the progression toward human-centered machine learning.

Qualitative coding offers the potential to obtain deep insights into social media, but the technique can be inconsistent and hard to scale. Researchers using qualitative coding impose structure on unstructured data through “codes” that represent categories for analysis. Our visual analytics interface, Aeonium, supports human insight in collaborative coding through visual overviews of codes assigned by multiple researchers and distributions of important keywords and codes. The underlying machine learning model highlights ambiguity and inconsistency. Our goal was not to reduce qualitative coding to a machine-solvable problem, but rather to bolster human understanding gained from coding and reinterpreting the data collaboratively. We conducted an experimental study with 39 participants who coded tweets using our interface. In addition to increased understanding of the topic, participants reported that Aeonium's collaborative coding functionality helped them reflect on their own interpretations. Feedback from participants demonstrates that visual analytics can help facilitate rich qualitative analysis and suggests design implications for future exploration.

A distance cartogram (DC) is a technique that alters distances between a user-specified origin and the other locations in a map with respect to travel time. With DC, users can weigh the relative travel time costs between the origin and potential destinations at a glance because travel times are projected in a linearly interpolated time space from the origin. Such glance-ability is known to be useful for travelers who are mindful of travel time when finding their travel destinations. When constructing DC, however, uneven urban traffic conditions introduce excessive distortion and challenge user intuition. In addition, there has been little research focusing on DC's user interaction design. To tackle these challenges and realize the potential of DC as an interactive decision-making support tool, we derive a set of useful interactions through two formative studies and devise two novel techniques called Geo-contextual Anchoring Projection and Scalable Road-network Construction. We develop an interactive map system using these techniques and evaluate this system by comparing it against an equidistant map (EM), a widely used conventional layout that preserves the geographical reality. Based on the analysis of user behavior and qualitative feedback, we identify several benefits of using DC itself and of the interaction techniques we derived. We also analyze the specific reasons behind these identified benefits.

With its roots dating to popular television shows of the 1960s such as Star Trek, fanfiction has blossomed into an extremely widespread form of creative expression. The transition from printed zines to online fanfiction repositories has facilitated this growth in popularity, with millions of fans writing stories and adding daily to sites such as Archive Of Our Own, Fanfiction.net, FIMfiction.net, and many others. Enthusiasts are sharing their writing, reading stories written by others, and helping each other to grow as writers. Yet, this domain is often undervalued by society and understudied by researchers. To facilitate the study of this large but often marginalized community, we present a fully anonymized data release (via differential privacy) of the metadata from a large fanfiction site (to protect author privacy, story, profile, and review text is excluded, and only metadata is provided). We use visual analytics techniques to draw several intriguing insights from the data and show the potential for future research. We hope other researchers can use this data to explore further questions related to online fanfiction communities.

From Harry Potter to American Horror Story, fanfiction is extremely popular among young people. Sites such as Fanfiction.net host millions of stories, with thousands more posted each day. Enthusiasts are sharing their writing and reading stories written by others. Exactly how does a generation known more for videogame expertise than long-form writing become so engaged in reading and writing in these communities? Via a nine-month ethnographic investigation of fanfiction communities that included participant observation, interviews, a thematic analysis of 4,500 reader reviews and an in-depth case study of a discussion group, we found that members of fanfiction communities spontaneously mentor each other in open forums, and that this mentoring builds upon previous interactions in a way that is distinct from traditional forms of mentoring and made possible by the affordances of networked publics. This work extends and develops the theory of distributed mentoring. Our findings illustrate how distributed mentoring supports fanfiction authors as they work to develop their writing skills. We believe distributed mentoring holds potential for supporting learning in a variety of formal and informal learning environments.

Many successful digital interfaces employ visual metaphors to convey features or data properties to users, but the characteristics that make a visual metaphor effective are not well understood. We used a theoretical conception of metaphor from cognitive linguistics to design an interactive system for viewing the citation network of the corpora of literature in the JSTOR database, a highly connected compound graph of 2 million papers linked by 8 million citations. We created 4 variants of this system, manipulating 2 distinct properties of metaphor. We conducted a between‐subjects experimental study with 80 participants to compare understanding and engagement when working with each version. We found that building on known image schemas improved response time on look‐up tasks, while contextual detail predicted increases in persistence and the number of inferences drawn from the data. Schema‐congruency combined with contextual detail produced the highest gains in comprehension. These findings provide concrete mechanisms by which designers presenting large data sets through metaphorical interfaces may improve their effectiveness and appeal with users.

Young people worldwide are participating in ever-increasing numbers in online fan communities. Far from mere shallow repositories of pop culture, these sites are accumulating significant evidence that sophisticated informal learning is taking place online in novel and unexpected ways. In order to understand and analyze in more detail how learning might be occurring, we conducted an in-depth nine-month ethnographic investigation of online fanfiction communities, including participant observation and fanfiction author interviews. Our observations led to the development of a theory we term distributed mentoring, which we present in detail in this paper. Distributed mentoring exemplifies one instance of how networked technology affords new extensions of behaviors that were previously bounded by time and space. Distributed mentoring holds potential for application beyond the spontaneous mentoring observed in this investigation and may help students receive diverse, thoughtful feedback in formal learning environments as well.

High performance computing (HPC) has driven collaborative science discovery for decades. Exascale computing platforms, currently in the design stage, will be deployed around 2022. The next generation of supercomputers is expected to utilize radically different computational paradigms, necessitating fundamental changes in how the community of scientific users will make the most efficient use of these powerful machines. However, there have been few studies of how scientists work with exascale or close-to-exascale HPC systems. Time as a metaphor is so pervasive in the discussions and valuation of computing within the HPC community that it is worthy of close study. We utilize time as a lens to conduct an ethnographic study of scientists interacting with HPC systems. We build upon recent CSCW work to consider temporal rhythms and collective time within the HPC sociotechnical ecosystem and provide considerations for future system design.

Affect has been identified as an important component of the communication practices of distributed teams. Our emerging theory of distributed affect moves beyond the individual as the primary unit of analysis, focusing instead on affect as a dynamic group process. Drawing upon a data set of over four years of chat logs from a distributed scientific collaboration relying on text-based communication to coordinate their work, we expand upon the framework of distributed affect and characterize the concept through five features: transference, resonance, pervasiveness, persistence, and representation. These features provide a set of descriptive components for interactions between people and their environment, their tools, and their present and historical references as part of a dynamical system of affect. We examine specific events in the group's history which highlight the dynamic way affect is operating in this context, and how it influences factors such as creative problem solving. The framework we describe offers a unique analytic lens for the study of computer-supported group work, and a useful tool for framing questions about the continued study of affect in collaborative teams.

Most geographic maps visually represent physical distance; however, travel time can in some cases be more important than distance because it directly indicates availability. The technique of creating maps from temporal data is known as isochronal cartography, and is a form of distortion for clarification. In an isochronal map, congestion expands areas, while ideal travel conditions make the map shrink in comparison to the actual distance scale of a traditional map. Although there have been many applications of this technique, detailed user studies of its efficacy remain scarce, and there are conflicting views on its practical value. To attempt to settle this issue, we utilized a user-centered design process to determine which features of isochronal cartography might be most usable in practice. We developed an interactive cartographic visualization system, Traffigram, that features a novel combination of efficient isochronal map algorithms and an interface designed to give map users a quick and seamless experience while preserving geospatial integrity and aesthetics. We validated our design choices with multiple usability studies. We present our results and discuss implications for design.

SUMMARY As a growing number of serious games have been developed for biology and computer science learning, few address the communication and technical challenges that arise in cyberinfrastructure (CI) intensive projects, where multiple‐domain scientists collaborate. This paper describes empirical data collected during a year‐long human‐centered game design process, in which design ideas generated by high‐school students were bridged with bioinformatics and parallel computing learning concepts. Our research shows that ‘fun’ and engaging game elements are actually well suited for addressing the sociotechnical aspects of CI projects. We provide a human‐centered game design methodology, as well as a case study, in which this methodology is applied to the design of parallel computing‐focused mini‐games. This research has implications for integrating large‐scale computing concepts such as shared resources and services into gaming experiences. It also has implications for supporting learning through enjoyable and fun experiences as part of a larger CI collaborative environment. Copyright © 2014 John Wiley & Sons, Ltd.

Geographically distributed collaborative teams often rely on synchronous text-based online communication for accomplishing tasks and maintaining social contact. This technology leaves a trace that can help researchers understand affect expression and dynamics in distributed groups. Although manual labeling of affect in chat logs has shed light on complex group communication phenomena, scaling this process to larger data sets through automation is difficult. We present a pipeline of natural language processing and machine learning techniques that can be used to build automated classifiers of affect in chat logs. Interpreting affect as a dynamic, contextualized process, we explain our development and application of this method to four years of chat logs from a longitudinal study of a multi-cultural distributed scientific collaboration. With ground truth generated through manual labeling of affect over a subset of the chat logs, our approach can successfully identify many commonly occurring types of affect.

We present VizDeck, a web-based tool for exploratory visual analytics of unorganized relational data. Motivated by collaborations with domain scientists who search for complex patterns in hundreds of data sources simultaneously, VizDeck automatically recommends appropriate visualizations based on the statistical properties of the data and adopts a card game metaphor to help organize the recommended visualizations into interactive visual dashboard applications in seconds with zero programming. The demonstration allows users to derive, share, and permanently store their own dashboard from hundreds of real science datasets using a production system deployed at the University of Washington.

The study of creativity has received significant attention over the past century, with a recent increase in interest in collaborative, distributed creativity. We posit that creativity in distributed groups is fostered by software interfaces that specifically enable socio-emotional or affective communication. However, previous work on creativity and affect has primarily focused on the individual, while group creativity research has concentrated more on cognition rather than affect. In this paper we propose a new model for creativity in distributed groups, based on the theory of groups as complex systems, that includes affect as well as cognition and that explicitly calls out the interface between individuals as a key parameter of the model. We describe the model, the four stages of collaborative creativity and the causal dynamics in each stage, and demonstrate how affect and interface can facilitate the generation, selection, and amplification of ideas in the various stages of collaborative creativity. We then validate our model with data from three field sites. The data was collected from longitudinal studies of two distributed groups involved in producing creative products--astrophysicists studying supernovae and the expansion rate of the universe and children creating multimedia programming projects online-"-and interviews with staff in a multinational engineering company.

There has been increased interest in reliable, non-intrusive methods of biometric identification due to the growing emphasis on security and increasing prevalence of identity theft. This paper presents a new biometric approach that involves an estimation of the unique oculomotor plant (OP) or eye globe muscle parameters from an eye movement trace. These parameters model individual properties of the human eye, including neuronal control signal, series elasticity, length tension, force velocity, and active tension. These properties can be estimated for each extraocular muscle, and have been shown to differ between individuals. We describe the algorithms used in our approach and the results of an experiment with 41 human subjects tracking a jumping dot on a screen. Our results show improvement over existing eye movement biometric identification methods. The technique of using Oculomotor Plant Mathematical Model (OPMM) parameters to model the individual eye provides a number of advantages for biometric identification: it includes both behavioral and physiological human attributes, is difficult to counterfeit, non-intrusive, and could easily be incorporated into existing biometric systems to provide an extra layer of security.

Verification and validation (V&V) of geospatial image analysis algorithms is a difficult task and is becoming increasingly important. While there are many types of image analysis algorithms, we focus on developing V&V methodologies for algorithms designed to provide textual descriptions of geospatial imagery. In this paper, we present a novel methodological basis for V&V that employs a domain-specific ontology, which provides a naming convention for a domain-bounded set of objects and a set of named relationships between these objects. We describe a validation process that proceeds through objectively comparing benchmark imagery, produced using the ontology, with algorithm results. As an example, we describe how the proposed V&V methodology would be applied to algorithms designed to provide textual descriptions of facilities.

There has been much recent interest in the development of tools to foster remote collaboration and shared creative work. An open question is: what are the guidelines for this process? What are the key socio-technical preconditions required for a geographically distributed group to collaborate effectively on creative work, and are they different from the conditions of a decade or two ago? In an attempt to answer these questions, we conducted empirical studies of two seemingly very different online communities, both requiring effective collaboration and creative work: an international collaboration of astrophysicists studying supernovae to learn more about the expansion rate of the universe, and a group of children, ages 8-15, from different parts of the world, creating and sharing animated stories and video games on the Scratch online community developed at MIT. Both groups produced creative technical work jointly and were considered successful in their communities. Data included the analysis of thousands of lines from chat and comment logs over a period of several months, and interviews with community members. We discovered some surprising commonalities and some intriguing possibilities, and suggest guidelines for successful creative collaborations. Specifically, systems that support social creativity must facilitate sharing and play, and their design must consider the effects of repurposing, augmentation and behavior adaptation.

There is a growing need for distributed teams to analyze complex and dynamic data streams and make critical decisions under time pressure. Via a case study, we discuss potential guidelines for the design of software tools to facilitate such collaborative decision-making. We introduce the term context-linked to characterize systems where both task and context information are included in a shared space. We describe a novel, lightweight, context-linked event notification/virtual assistant system developed to aid a cross-cultural, geographically distributed team of astrophysicists to remotely maneuver a custom-built instrument under challenging operational conditions, where critical decisions must be made in as little as 45 seconds. The system has been in use since 2005 by a major international astrophysics collaboration. We describe the design and implementation of the event notification system and then present a case study, based on event log analysis and user interviews, of its effectiveness in substantially improving user performance during time-critical science tasks. Finally, we discuss the implications of context linking for supporting common ground in distributed teams.

Laser wakefield particle accelerators have shown the potential to generate electric fields thousands of times higher than those of conventional accelerators. The resulting extremely short particle acceleration distance could yield a potential new compact source of energetic electrons and radiation, with wide applications from medicine to physics. Physicists investigate laser-plasma internal dynamics by running particle-in-cell simulations; however, this generates a large dataset that requires time-consuming, manual inspection by experts in order to detect key features such as beam formation. This paper describes a framework to automate the data analysis and classification of simulation data. First, we propose a new method to identify locations with high density of particles in the space-time domain, based on maximum extremum point detection on the particle distribution. We analyze high density electron regions using a lifetime diagram by organizing and pruning the maximum extrema as nodes in a minimum spanning tree. Second, we partition the multivariate data using fuzzy clustering to detect time steps in a experiment that may contain a high quality electron beam. Finally, we combine results from fuzzy clustering and bunch lifetime analysis to estimate spatially confined beams. We demonstrate our algorithms successfully on four different simulation datasets.

We introduce a novel application of support vector machines (SVMs) to the problem of identifying potential supernovae using photometric and geometric features computed from astronomical imagery. The challenges of this supervised learning application are significant: 1) noisy and corrupt imagery resulting in high levels of feature uncertainty, 2) features with heavy-tailed, peaked distributions, 3) extremely imbalanced and overlapping positive and negative data sets, and 4) the need to reach high positive classification rates, i.e. to find all potential supernovae, while reducing the burdensome workload of manually examining false positives. High accuracy is achieved via a sign-preserving, shifted log transform applied to features with peaked, heavy-tailed distributions. The imbalanced data problem is handled by oversampling positive examples, selectively sampling misclassified negative examples, and iteratively training multiple SVMs for improved supernova recognition on unseen test data. We present cross-validation results and demonstrate the impact on a large-scale supernova survey that currently uses the SVM decision value to rank-order 600,000 potential supernovae each night

Many aircraft accidents each year are caused by encounters with invisible airflow hazards. Recent advances in aviation sensor technology offer the potential for aircraft-based sensors that can gather large amounts of airflow velocity data in real-time. With this influx of data comes the need to study how best to present it to the pilot - a cognitively overloaded user focused on a primary task other than that of information visualization.We focus on one particular aviation application, but the results may be relevant to user interfaces in other operationally stressful environments.

In order to safely operate their aircraft, pilots must make rapid decisions based on integrating and processing large amounts of heterogeneous information. Visual displays are often the most efficient method of presenting safety-critical data to pilots in real time. However, care must be taken to ensure the pilot is provided with the appropriate amount of information to make effective decisions and not become cognitively overloaded. The results of two usability studies of a prototype airflow hazard visualization cockpit decision support system are summarized. The studies demonstrate that such a system significantly improves the performance of helicopter pilots landing under turbulent conditions. Based on these results, design principles and implications for cockpit decision support systems using visualization are presented.

A randomized strategy for maintaining balance in dynamically changing search trees that has optimal expected behavior is presented. In particular, in the expected case an update takes logarithmic time and requires fewer than two rotations. Moreover, the update time remains logarithmic, even if the cost of a rotation is taken to be proportional to the size of the rotated subtree. The approach generalizes naturally to weighted trees, where the expected time bounds for accesses and updates again match the worst case time bounds of the best deterministic methods. The balancing strategy and algorithms are exceedingly simple and should be fast in practice.>