Artificial intelligence (AI) is increasingly deployed in high-stakes domains such as search-and-rescue (SAR), where detections or classifications can shape how teams share information, build trust, and make time-critical decisions. This paper investigates how teams of SAR professionals incorporate AI into their teamwork, highlighting both benefits and challenges. To support this study, we developed the Council of Wizards, a multi-agent Wizard-of-Oz technique that simulates distributed AI systems, enabling scalable and controlled evaluation of collaborative dynamics. Using this novel method, we conducted an experiment with 24 subject-matter experts (SMEs) who reviewed SAR video footage as small teams and made group decisions, with or without AI support. Quantitative results showed that AI-assisted teams reached consensus faster than controls. Qualitative feedback revealed how participants interpreted trust cues, adapted strategies, and sometimes struggled with overload or conflicting detections. Findings illustrate how AI shapes teamwork in SAR and provide design implications for trustworthy distributed human-AI interactions.

We are again thrilled to be able to present the Computer-Supported Cooperative Work and Social Computing (CSCW) community with an issue of the Proceedings of the ACM on Human-Computer Interaction, containing very interesting and relevant scholarship from its members. This issue includes 42 papers from the May 2025 cycle, selected from a total of 637 submissions and following two rounds of reviews and one revision. 209 submissions from this round will be further revised, reviewed again, and may appear in another issue of the journal later this year. Our external reviewers and track editorial board have together conducted a rigorous review process to select contributions of the highest quality advancing the CSCW field. As Track Chairs, we are grateful for the community’s collective efforts to continue shaping and sharing CSCW’s tradition of high-quality scholarship across the years.

Small businesses need vulnerability assessments to identify and mitigate cyber risks. Cybersecurity clinics provide a solution by offering students hands-on experience while delivering free vulnerability assessments to local organizations. To scale this model, we propose an Open Source Intelligence (OSINT) clinic where students conduct assessments using only publicly available data.We enhance the quality of investigations in the OSINT clinic by addressing the technical and collaborative challenges. Over the duration of the 2023-24 academic year, we conducted a three-phase co-design study with six students. Our study identified key challenges in the OSINT investigations and explored how generative AI could address these performance gaps. We developed design ideas for effective AI integration based on the use of AI probes and collaboration platform features. A pilot with three small businesses highlighted both the practical benefits of AI in streamlining investigations, and limitations, including privacy concerns and difficulty in monitoring progress.

AI models are constantly evolving, with new versions released frequently. Human-AI interaction guidelines encourage notifying users about changes in model capabilities, ideally supported by thorough benchmarking. However, as AI systems integrate into domain-specific workflows, exhaustive benchmarking can become impractical, often resulting in silent or minimally communicated updates. This raises critical questions: Can users notice these updates? What cues do they rely on to distinguish between models? How do such changes affect their behavior and task performance? We address these questions through two studies in the context of facial recognition for historical photo identification: an online experiment examining users’ ability to detect model updates, followed by a diary study exploring perceptions in a real-world deployment. Our findings highlight challenges in noticing AI model updates, their impact on downstream user behavior and performance, and how they lead users to develop divergent folk theories. Drawing on these insights, we discuss strategies for effectively communicating model updates in AI-infused systems.

In urban search and rescue (USAR) operations, communication between handlers and specially trained canines is crucial but often complicated by challenging environments and the specific behaviors canines are trained to exhibit when detecting a person. Since a USAR canine often works out of sight of the handler, the handler lacks awareness of the canine's location and situation, known as the 'sensemaking gap.' In this paper, we propose KHAIT, a novel approach to close the sensemaking gap and enhance USAR effectiveness by integrating object detection-based Artificial Intelligence (AI) and Augmented Reality (AR). Equipped with AI-powered cameras, edge computing, and AR headsets, KHAIT enables precise and rapid object detection from a canine's perspective, improving survivor localization. We evaluate this approach in a real-world USAR environment, demonstrating an average survival allocation time decrease of 22%, enhancing the speed and accuracy of operations.

Genealogy, the study of family history and lineage, has seen tremendous growth over the past decade, fueled by technological advances such as home DNA testing and mass digitization of historical records. However, HCI research on genealogy practices is nascent, with the most recent major studies predating this transformation. In this paper, we present a qualitative study of the current state of technological support for genealogy research, collaboration, and education. Through semi-structured interviews with 20 genealogists with diverse expertise, we report on current practices, challenges, and success stories around how genealogists conduct research, collaborate, and learn skills. We contrast the experiences of amateurs and experts, describe the emerging importance of standardization and professionalization of the field, and stress the critical role of computer systems in genealogy education. We bridge studies of sensemaking and information literacy through this empirical study on genealogy research practices, and conclude by discussing how genealogy presents a unique perspective through which to study collective sensemaking and education in online communities.

This article introduces the design and prototype of Ajna, a wearable shared perception system for supporting extreme sensemaking in emergency scenarios. Ajna addresses technical challenges in Augmented Reality (AR) devices, specifically the limitations of depth sensors and cameras. These limitations confine object detection to close proximity and hinder perception beyond immediate surroundings, through obstructions, or across different structural levels, impacting collaborative use. It harnesses the Inertial Measurement Unit (IMU) in AR devices to measure users’ relative distances from a set physical point, enabling object detection sharing among multiple users across obstacles like walls and over distances. We tested Ajna’s effectiveness in a controlled study with 15 participants simulating emergency situations in a multi-story building. We found that Ajna improved object detection, location awareness, and situational awareness and reduced search times by 15%. Ajna’s performance in simulated environments highlights the potential of artificial intelligence (AI) to enhance sensemaking in critical situations, offering insights for law enforcement, search and rescue, and infrastructure management.

Open Source Intelligence (OSINT) investigations, which rely entirely on publicly available data such as social media, play an increasingly important role in solving crimes and holding governments accountable. The growing volume of data and complex nature of tasks, however, means there is a pressing need to scale and speed up OSINT investigations. Expert-led crowdsourcing approaches show promise, but tend to either focus on narrow tasks or domains, or require resource-intense, long-term relationships between expert investigators and crowds. We address this gap by providing a flexible framework that enables investigators across domains to enlist crowdsourced support for discovery and verification of OSINT. We use a design-based research (DBR) approach to develop OSINT Research Studios (ORS), a sociotechnical system in which novice crowds are trained to support professional investigators with complex OSINT investigations. Through our qualitative evaluation, we found that ORS facilitates ethical and effective OSINT investigations across multiple domains. We also discuss broader implications of expert-crowd collaboration and opportunities for future work.

Public deliberations are often a staple ingredient in community decision-making. However, traditional, time-constrained, in-person debates can become highly polarized, eroding trust in authorities, and leaving the community divided. This is the case in redistricting deliberations for public school zoning. Seeking alternative ways of support, we evaluated the potential introduction of an online platform that combines multiple streams of data, visualizes school attendance boundaries, and enables the manipulation of representations of land parcels. To capture multiple stakeholders' values about the potential to enhance public engagement in school rezoning decision-making through an online platform, we conducted interviews with 12 participants with previous experiences in traditional, in-person deliberations. Insights from the interviews highlight the several roles an online platform could take, especially as it provides alternative means of participation (online, synchronous, and asynchronous). Additionally, we discuss the potential for technology to increase the visibility and participation of multiple community actors in public deliberations and present implications for the design of future tools to support public decision-making.

GIFs afford a high degree of personalization, as they are often created from popular movie and video clips with diverse and realistic characters, each expressing a nuanced emotional state through a combination of characters' own unique bodily gestures and distinctive visual backgrounds. These properties of high personalization and embodiment provide a unique window for exploring how individuals represent and express themselves on social media through the lens of the GIFs they use. In this study, we explore how Twitter users express their gender and racial identities through characters in GIFs. We conducted a behavioral study (n=398) to simulate a series of tweeting and GIF-picking scenarios. We annotated the gender and race identities of GIF characters, and we found that gender and race identities have significant impacts on users' GIF choices: men chose more gender-matching GIFs than women, and White participants chose more race-matching GIFs than Black participants. We also found that users' prior familiarity with the source of a GIF and perceptions about the composition of the audience (viz., having a matching identity) have significant effects on whether a user will choose race- and gender-matching GIFs. This work has implications for practitioners supporting personalized social identity construction and impression management mechanisms online.

Crowdsourced investigations shore up democratic institutions by debunking misinformation and uncovering human rights abuses. However, current crowdsourcing approaches rely on simplistic collaborative or competitive models and lack technological support, limiting their collective impact. Prior research has shown that blending elements of competition and collaboration can lead to greater performance and creativity, but crowdsourced investigations pose unique analytical and ethical challenges. In this paper, we employed a four-month-long Research through Design process to design and evaluate a novel interaction style called collaborative capture the flag competitions (CoCTFs). We instantiated this interaction style through CoSINT, a platform that enables a trained crowd to work with professional investigators to identify and investigate social media misinformation. Our mixed-methods evaluation showed that CoSINT leverages the complementary strengths of competition and collaboration, allowing a crowd to quickly identify and debunk misinformation. We also highlight tensions between competition versus collaboration and discuss implications for the design of crowdsourced investigations.

Social media platforms have enabled extremists to organize violent events, such as the 2021 U.S. Capitol Attack. Simultaneously, these platforms enable professional investigators and amateur sleuths to collaboratively collect and identify imagery of suspects with the goal of holding them accountable for their actions. Through a case study of Sedition Hunters, a Twitter community whose goal is to identify individuals who participated in the 2021 U.S. Capitol Attack, we explore what are the main topics or targets of the community, who participates in the community, and how. Using topic modeling, we find that information sharing is the main focus of the community. We also note an increase in awareness of privacy concerns. Furthermore, using social network analysis, we show how some participants played important roles in the community. Finally, we discuss implications for the content and structure of online crowdsourced investigations.

Facial recognition systems pose numerous ethical challenges around privacy, racial and gender bias, and accuracy, yet little guidance is available for designers and developers. We explore solutions to these challenges in a three-phase design process to create Civil War Twin (CWT), an educational web-based application where users can discover their lookalikes from the American Civil War era (1861--65) while learning more about facial recognition and history. Through this design process, we operationalize a framework for AI literacy, consult with scholars of history, gender, and race, and evaluate CWT in feedback sessions with diverse prospective users. We iteratively formulate design goals to incorporate transparency, inclusivity, speculative design, and empathy into our application. We found that users' perceived learning about the strengths and limitations of facial recognition and Civil War history improved after using CWT, and that our design successfully met users' ethical standards. We also discuss how our ethical design process can be applied to future facial recognition applications.

Online investigations are increasingly conducted by individuals with diverse skill levels and experiences, with mixed results. Novice investigations often result in vigilantism or doxxing, while expert investigations have greater success rates and fewer mishaps. Many of these experts are involved in a community of practice known as Open Source Intelligence (OSINT), with an ethos and set of techniques for conducting investigations using only publicly available data. Through semi-structured interviews with 14 expert OSINT investigators from nine different organizations, we examine the social dynamics of this community, including the collaboration and competition patterns that underlie their investigations. We also describe investigators’ use of and challenges with existing OSINT tools, and implications for the design of social computing systems to better support crowdsourced investigations.

Modern experiments in many disciplines generate large quantities of network (graph) data. Researchers require aesthetic layouts of these networks that clearly convey the domain knowledge and meaning. However, the problem remains challenging due to multiple conflicting aesthetic criteria and complex domain-specific constraints. In this article, we present a strategy for generating visualizations that can help network biologists understand the protein interactions that underlie processes that take place in the cell. Specifically, we have developed Flud, a crowd-powered system that allows humans with no expertise to design biologically meaningful graph layouts with the help of algorithmically generated suggestions. Furthermore, we propose a novel hybrid approach for graph layout wherein crowd workers and a simulated annealing algorithm build on each other’s progress. A study of about 2,000 crowd workers on Amazon Mechanical Turk showed that the hybrid crowd–algorithm approach outperforms the crowd-only approach and state-of-the-art techniques when workers were asked to lay out complex networks that represent signaling pathways. Another study of seven participants with biological training showed that Flud layouts are more effective compared to those created by state-of-the-art techniques. We also found that the algorithmically generated suggestions guided the workers when they are stuck and helped them improve their score. Finally, we discuss broader implications for mixed-initiative interactions in layout design tasks beyond biology.

Investigators in fields such as journalism and law enforcement have long sought the public's help with investigations. New technologies have also allowed amateur sleuths to lead their own crowdsourced investigations - that have traditionally only been the purview of expert investigators - with mixed results. Through an ethnographic study of a four-day, co-located event with over 250 attendees, we examine the human infrastructure responsible for enabling the success of an expert-led crowdsourced investigation. We find that the experts enabled attendees to generate useful leads; the attendees formed a community around the event; and the victims' families felt supported. Additionally, the co-located setting, legal structures, and emergent social norms impacted collaborative work practice. We also surface three important tensions to consider in future investigations and provide design recommendations to manage these tensions.

Identifying people in historical photographs is important for interpreting material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this paper, we focus on identifying portraits of soldiers who participated in the American Civil War (1861-65). Millions of these portraits survive, but only 10-20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluation of Photo Sleuth one month after its public launch showed that it helped users successfully identify unknown portraits.

Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this article, we focus on identifying portraits of soldiers who participated in the American Civil War (1861--65), the first widely photographed conflict. Many thousands of these portraits survive, but only 10%--20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluations of Photo Sleuth one month and 11 months after its public launch showed that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution. We also discuss implications for crowd-AI interaction and person identification pipelines.

Checklists and guidelines have played an increasingly important role in complex tasks ranging from the cockpit to the operating theater. Their role in creative tasks like design is less explored. In a needfinding study with expert web designers, we identified designers' challenges in adhering to a checklist of design guidelines. We built Critter, which addressed these challenges with three components: Dynamic Checklists that progressively disclose guideline complexity with a self-pruning hierarchical view, AutoQA to automate common quality assurance checks, and guideline-specific feedback provided by a reviewer to highlight mistakes as they appear. In an observational study, we found that the more engaged a designer was with Critter, the fewer mistakes they made in following design guidelines. Designers rated the AutoQA and contextual feedback experience highly, and provided feedback on the tradeoffs of the hierarchical Dynamic Checklists. We additionally found that a majority of designers rated the AutoQA experience as excellent and felt that it increased the quality of their work. Finally, we discuss broader implications for supporting complex creative tasks.

As AI-based face recognition technologies are increasingly adopted for high-stakes applications like locating suspected criminals, public concerns about the accuracy of these technologies have grown as well. These technologies often present a human expert with a shortlist of high-confidence candidate faces from which the expert must select correct match(es) while avoiding false positives, which we term the “last-mile problem.” We propose Second Opinion, a web-based software tool that employs a novel crowdsourcing workflow inspired by cognitive psychology, seed-gather-analyze, to assist experts in solving the last-mile problem. We evaluated Second Opinion with a mixed-methods lab study involving 10 experts and 300 crowd workers who collaborate to identify people in historical photos. We found that crowds can eliminate 75% of false positives from the highest-confidence candidates suggested by face recognition, and that experts were enthusiastic about using Second Opinion in their work. We also discuss broader implications for crowd–AI interaction and crowdsourced person identification.

Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this paper, we focus on identifying portraits of soldiers who participated in the American Civil War (1861-65), the first widely-photographed conflict. Many thousands of these portraits survive, but only 10--20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluation of Photo Sleuth one month after its public launch showed that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution. We also discuss implications for crowd-AI interaction and person identification pipelines.

Crowdsourced sensemaking has shown great potential for enabling scalable analysis of complex data sets, from planning trips, to designing products, to solving crimes. Yet, most crowd sensemaking approaches still require expert intervention because of worker errors and bottlenecks that would otherwise harm the output quality. Mitigating these errors and bottlenecks would significantly reduce the burden on experts, yet little is known about the types of mistakes crowds make with sensemaking micro-tasks and how they propagate in the sensemaking loop. In this paper, we conduct a series of studies with 325 crowd workers using a crowd sensemaking pipeline to solve a fictional terrorist plot, focusing on understanding why errors and bottlenecks happen and how they propagate. We classify types of crowd errors and show how the amount and quality of input data influence worker performance. We conclude by suggesting design recommendations for integrated crowdsourcing systems and speculating how a complementary top-down path of the pipeline could refine crowd analyses.

Expert investigators bring advanced skills and deep experience to analyze visual evidence, but they face limits on their time and attention. In contrast, crowds of novices can be highly scalable and parallelizable, but lack expertise. In this paper, we introduce the concept of shared representations for crowd--augmented expert work, focusing on the complex sensemaking task of image geolocation performed by professional journalists and human rights investigators. We built GroundTruth, an online system that uses three shared representations-a diagram, grid, and heatmap-to allow experts to work with crowds in real time to geolocate images. Our mixed-methods evaluation with 11 experts and 567 crowd workers found that GroundTruth helped experts geolocate images, and revealed challenges and success strategies for expert-crowd interaction. We also discuss designing shared representations for visual search, sensemaking, and beyond.

Exploring coordinated relationships (e.g., shared relationships between two sets of entities) is an important analytics task in a variety of real-world applications, such as discovering similarly behaved genes in bioinformatics, detecting malware collusions in cyber security, and identifying products bundles in marketing analysis. Coordinated relationships can be formalized as biclusters. In order to support visual exploration of biclusters, bipartite graphs based visualizations have been proposed, and edge bundling is used to show biclusters. However, it suffers from edge crossings due to possible overlaps of biclusters, and lacks in-depth understanding of its impact on user exploring biclusters in bipartite graphs. To address these, we propose a novel bicluster-based seriation technique that can reduce edge crossings in bipartite graphs drawing and conducted a user experiment to study the effect of edge bundling and this proposed technique on visualizing biclusters in bipartite graphs. We found that they both had impact on reducing entity visits for users exploring biclusters, and edge bundles helped them find more justified answers. Moreover, we identified four key trade-offs that inform the design of future bicluster visualizations. The study results suggest that edge bundling is critical for exploring biclusters in bipartite graphs, which helps to reduce low-level perceptual problems and support high-level inferences.

Biologists often perform experiments whose results generate large quantities of data, such as interactions between molecules in a cell, that are best represented as networks (graphs). To visualize these networks and communicate them in publications, biologists must manually position the nodes and edges of each network to reflect their real-world physical structure. This process does not scale well, and graph layout algorithms lack the biological underpinnings to offer a viable alternative. In this paper, we present CrowdLayout, a crowdsourcing system that leverages human intelligence and creativity to design layouts of biological network visualizations. CrowdLayout provides design guidelines, abstractions, and editing tools to help novice workers perform like experts. We evaluated CrowdLayout in two experiments with paid crowd workers and real biological network data, finding that crowds could both create and evaluate meaningful, high-quality layouts. We also discuss implications for crowdsourced design and network visualizations in other domains.

Many types of investigative work involve verifying the legitimacy of visual evidence by identifying the precise geographic location where a photo or video was taken. Professional geolocation is often a manual, time-consuming process that can involve searching large areas of satellite imagery for potential matches. In this paper, we explore how crowdsourcing can be used to support expert image geolocation. We adapt an expert diagramming technique to overcome spatial reasoning limitations of novice crowds so that they can support an expert's search. In an experiment (n=540), we found that diagrams work significantly better than ground-level photos and allow crowds to reduce a search area by half before any expert intervention. We also discuss hybrid approaches to complex image analysis combining crowds, experts, and computer vision.

The increasing volume of text data is challenging the cognitive capabilities of expert analysts. Machine learning and crowdsourcing present new opportunities for large-scale sensemaking, but we must overcome the challenge of modeling the overall process so that many distributed agents can contribute to suitable components asynchronously and meaningfully. In this paper, we explore how to crowdsource the sensemaking process via a pipeline of modularized steps connected by clearly defined inputs and outputs. Our pipeline restructures and partitions information into "context slices" for individual workers. We implemented CrowdIA, a software platform to enable unsupervised crowd sensemaking using our pipeline. With CrowdIA, crowds successfully solved two mysteries, and were one step away from solving the third. The crowd's intermediate results revealed their reasoning process and provided evidence that justifies their conclusions. We suggest broader possibilities to optimize each component, as well as to evaluate and refine previous intermediate analyses to improve the final result.

Crowdsourcing more complex and creative tasks is seen as a desirable goal for both employers and workers, but these tasks traditionally require domain expertise. Employers can recruit only expert workers, but this approach does not scale well. Alternatively, employers can decompose complex tasks into simpler micro-tasks, but some domains, such as historical analysis, cannot be easily modularized in this way. A third approach is to train workers to learn the domain expertise. This approach offers clear benefits to workers, but is perceived as costly or infeasible for employers. In this paper, we explore the trade-offs between learning and productivity in training crowd workers to analyze historical documents. We compare CrowdSCIM, a novel approach that teaches historical thinking skills to crowd workers, with two crowd learning techniques from prior work and a baseline. Our evaluation (n=360) shows that CrowdSCIM allows workers to learn domain expertise while producing work of equal or higher quality versus other conditions, but efficiency is slightly lower.

Geolocation, the process of identifying the precise location in the world where a photo or video was taken, is central to many types of investigative work, from debunking fake news posted on social media to locating terrorist training camps. Professional geolocation is often a manual, time-consuming process that involves searching large areas of satellite imagery for potential matches. In this paper, we explore how crowdsourcing can be used to support expert image geolocation. We adapt an expert diagramming technique to overcome spatial reasoning limitations of novice crowds, allowing them to support an expert’s search. In two experiments (n=1080), we found that diagrams work significantly better than ground-level photos and allow crowds to reduce a search area by half before any expert intervention. We also discuss hybrid approaches to complex image analysis combining crowds, experts, and computer vision.

SUMMARY: Networks have become ubiquitous in systems biology. Visualization is a crucial component in their analysis. However, collaborations within research teams in network biology are hampered by software systems that are either specific to a computational algorithm, create visualizations that are not biologically meaningful, or have limited features for sharing networks and visualizations. We present GraphSpace, a web-based platform that fosters team science by allowing collaborating research groups to easily store, interact with, layout and share networks. AVAILABILITY AND IMPLEMENTATION: Anyone can upload and share networks at http://graphspace.org. In addition, the GraphSpace code is available at http://github.com/Murali-group/graphspace if a user wants to run his or her own server. CONTACT: [email protected]. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Expert feedback is valuable but hard to obtain for many designers. Online crowds can provide fast and affordable feedback, but workers may lack relevant domain knowledge and experience. Can expert rubrics address this issue and help novices provide expert-level feedback? To evaluate this, we conducted an experiment with a 2x2 factorial design. Student designers received feedback on a visual design from both experts and novices, who produced feedback using either an expert rubric or no rubric. We found that rubrics helped novice workers provide feedback that was rated nearly as valuable as expert feedback. A follow-up analysis on writing style showed that student designers found feedback most helpful when it was emotionally positive and specific, and that a rubric increased the occurrence of these characteristics in feedback. The analysis also found that expertise correlated with longer critiques, but not the other favorable characteristics. An informal evaluation indicates that experts may instead have produced value by providing clearer justifications.

Feedback is an important component of the design process, but gaining access to high-quality critique outside a classroom or firm is challenging. We present CrowdCrit, a web-based system that allows designers to receive design critiques from non-expert crowd workers. We evaluated CrowdCrit in three studies focusing on the designer's experience and benefits of the critiques. In the first study, we compared crowd and expert critiques and found evidence that aggregated crowd critique approaches expert critique. In a second study, we found that designers who got crowd feedback perceived that it improved their design process. The third study showed that designers were enthusiastic about crowd critiques and used them to change their designs. We conclude with implications for the design of crowd feedback services.

Learning about a new area of knowledge is challenging for novices partly because they are not yet aware of which topics are most important. The Internet contains a wealth of information for learning the underlying structure of a domain, but relevant sources often have diverse structures and emphases, making it hard to discern what is widely considered essential knowledge vs. what is idiosyncratic. Crowdsourcing offers a potential solution because humans are skilled at evaluating high-level structure, but most crowd micro-tasks provide limited context and time. To address these challenges, we present Crowdlines, a system that uses crowdsourcing to help people synthesize diverse online information. Crowdworkers make connections across sources to produce a rich outline that surfaces diverse perspectives within important topics. We evaluate Crowdlines with two experiments. The first experiment shows that a high context, low structure interface helps crowdworkers perform faster, higher quality synthesis, while the second experiment shows that a tournament-style (parallelized) crowd workflow produces faster, higher quality, more diverse outlines than a linear (serial/iterative) workflow.

We report on our design of Curated City, a website that lets people build their own personal guide to the city's neighborhoods by chronicling their favorite experiences. Although users make their own personal guides, they are immersed in a social curatorial experience where they are influenced directly and indirectly by the guides of others. We use a 2-week field trial involving 20 residents of Pittsburgh as a technological probe to explore the initial design decisions, and we further refine the design landscape through subject interviews. Based on this study, we identify a set of design recommendations for building scalable social platforms for curating the experiences of the city.

In this paper, we integrate theories of distributed leadership and distributed cognition to account for the roles of people and technology in online leadership. When leadership is distributed effectively, the result can be success stories like Wikipedia and Linux. However, finding a successful distribution is challenging. In the online community Newgrounds, hundreds of collaborative animation projects called "collabs" are started each year, but less than 20% are completed. We suggest that many collabs fail because leaders are overburdened and lack adequate technological support. We introduce Pipeline, a collaboration tool designed to support and transform leadership, with the goal of easing the burden on leaders of online creative projects. Through a case study of a six-week, 30-artist collaboration called Holiday Flood, we show how Pipeline supported redistributed leadership. We conclude with implications for theory and the design of social computing systems.

While microblog readers have a wide variety of reactions to the content they see, studies have tended to focus on extremes such as retweeting and unfollowing. To understand the broad continuum of reactions in-between, which are typically not shared publicly, we designed a website that collected the first large corpus of follower ratings on Twitter updates. Using our dataset of over 43,000 voluntary ratings, we find that nearly 36% of the rated tweets are worth reading, 25% are not, and 39% are middling. These results suggest that users tolerate a large amount of less-desired content in their feeds. We find that users value information sharing and random thoughts above me-oriented or presence updates. We also offer insight into evolving social norms, such as lack of context and misuse of @mentions and hashtags. We discuss implications for emerging practice and tool design.

Online creative collaboration (peer production) has enabled the creation of Wikipedia and open source software (OSS), and is rapidly expanding to encompass new domains, such as video, music, and animation. But what are the underlying principles allowing online creative collaboration to succeed, and how well do they transfer from one domain to another? In this paper, we address these questions by comparing and contrasting online, collaborative animated movies, called collabs, with OSS projects. First, we use qualitative methods to solicit potential success factors from collab participants. Then, we test these predictions by quantitatively analyzing a data set of nearly 900 collabs. Finally, we compare and contrast our results with the literature on OSS development and propose broader theoretical implications. Our findings offer a starting point for a systematic research agenda seeking to unlock the potential of online creative collaboration.

Online creative collaboration projects are started every day, but many fail to produce new artifacts of value. In this poster, we address the question of why some of these projects succeed and others fail. Our quantitative analysis of 892 online collaborative animation projects, or "collabs," indicates that the early presence of organizational and structural elements, particularly those of a technical nature, can predict successful completion.

This paper presents preliminary findings from our empirical study of the cognition employed by performers in improvisational theatre. Our study has been conducted in a laboratory setting with local improvisers. Participants performed predesigned improv "games", which were videotaped and shown to each individual participant for a retrospective protocol collection. The participants were then shown the video again as a group to elicit data on group dynamics, misunderstandings, etc. This paper presents our initial findings that we have built based on our initial analysis of the data and highlights details of interest.

This workshop is aimed at describing the elusive creative process: addressing models and of creative practice, from art to craft, from dance to education. In particular, we wish to discuss creative models that are conversational: connect the creator and the consumer via the creative act or artifact. The goal is to foster creative collaboration across domains and address the practice and design of the creative act to bring new ideas for researchers and practitioners alike.

For over a century, citizen scientists have volunteered to collect huge quantities of data for professional scientists to analyze. We designed Pathfinder, an online environment that challenges this traditional division of labor by providing tools for citizen scientists to collaboratively discuss and analyze the data they collect. We evaluated Pathfinder in a sustainability and commuting context using a mixed methods approach in both naturalistic and experimental settings. Our results showed that citizen scientists preferred Pathfinder to a standard wiki and were able to go beyond data collection and engage in deeper discussion and analyses. We also found that citizen scientists require special types of technological support because they generate original research. This paper offers an early example of the mutually beneficial relationship between HCI and citizen science.

Behind every successful online creative collaboration, from Wikipedia to Linux, is at least one effective project leader. Yet, we know little about what such leaders do and how technology supports or inhibits their work. My thesis investigates leadership in online creative collaboration, focusing on the novel context of animated movie-making. I first conducted an empirical study of existing leadership practices in this context. I am now designing a Web-based collaborative system, Sandbox, to understand the impact of technological support for centralized versus decentralized leadership in this context. My expected contributions include a comparative investigation of the effects of different types of leadership on online creative collaboration, and a set of empirically validated design principles for supporting leadership in online creative collaboration.

Distributed teams are increasingly common in today's workplace. For these teams, face-to-face meetings where members can most easily build trust are rare and often costprohibitive. 3D virtual worlds and games may provide an alternate means for encouraging team development due to their affordances for facile communication, emotional engagement, and social interaction among participants. Using principles derived from social psychological theory, we have designed and built a collection of team-building games within the popular virtual world Second Life. We detail here the design decisions made in the creation of these games and discuss how they evolved based on early participant observations.

Leadership plays a central role in the success of many forms of online creative collaboration, yet little is known about the challenges leaders must manage. In this paper, we report on a qualitative study of leadership in three online communities whose members collaborate over the Internet to create computer-animated movies called "collabs." Our results indicate that most collabs fail. Collab leaders face two major challenges. First, leaders must design collabora-tive projects. Second, leaders must manage artists during the collab production process. We contrast these challenges with the available empirical research on leadership in open-source software and Wikipedia, identifying four themes: originality, completion, subjectivity, and ownership. We conclude with broader implications for online creative col-laboration in its many forms.

We have developed an audio-based casual puzzle game which produces a time-stamped transcription of spoken audio as a by-product of play. Our evaluation of the game indicates that it is both fun and challenging. The transcripts generated using the game are more accurate than those produced using a standard automatic transcription system and the time-stamps of words are within several hundred milliseconds of ground truth.