AI development for healthcare aims to enhance medical decision-making through risk evaluation. Scholars have focused on improving the accuracy of Breast Cancer AI Risk Assessment Tools (BC-AIRAT), yet these tools remain underutilized in clinical practices. This provides an opportunity to explore how these tools are used and how they may support risk communications. We conducted a three-phase study, with clinicians and patients, in the context of the United States healthcare system, including formative interviews that surface the challenges of BC-AIRAT practices, design probe re-purposing BC-AIRAT as supporting risk communications, and design probe-driven interviews with diverse stakeholders. Our findings surface the gap and opportunity for designing AI-mediated risk communication tool, highlighting the participants’ reflections on AI for managing risk assessment workflows, mediating fragmented breast health guidelines, and delivering information to patients for proactive decision making. We conclude with design implications for using AI as a mediator in breast cancer risk communication.

Modern cities across the globe increasingly rely on ridehail services for on-demand transportation and mobility. But for drivers, such marketed affordances give rise to hidden driver burdens and vulnerabilities that evade the oversight of consumers and regulators. To effectively advance worker protections and motivate more socially responsible practices, consumers must understand the realistic labor, logistics and costs involved with ridehail driving. Through think-aloud nine workshops with 19 drivers and 15 passengers, we explore the potential for gamified in-ride interactions to facilitate engagement with real (and lived) driver experiences, surfacing passenger knowledge gaps around latent working conditions, prompting reflection and shifts in perception of their relative power and consumption behaviors, highlighting drivers’ preferences for creating more immersive and contextualized service experiences, and identifying design opportunities for safe and appropriate passenger-driver interactions that motivate solidarity. In sum, we advance conceptual understandings of social and managerial relations within a ride, uncover future potential for citizen-led labor advocacy, and offer design guidelines for more human-centered workplace technologies.

AI agents, or bots, serve important roles in online communities. However, they are often designed by outsiders or a few tech-savvy members, leading to bots that may not align with the broader community’s needs. How might communities collectively shape the behavior of community bots? We present Botender, a system that enables communities to collaboratively design LLM-powered bots without coding. With Botender, community members can directly propose, iterate on, and deploy custom bot behaviors tailored to community needs. Botender facilitates testing and iteration on bot behavior through case-based provocations: interaction scenarios generated to spark user reflection and discussion around desirable bot behavior. A validation study found these provocations more useful than standard test cases for revealing improvement opportunities and surfacing disagreements. During a five-day deployment across six Discord servers, Botender supported communities in tailoring bot behavior to their specific needs, showcasing the usefulness of case-based provocations in facilitating collaborative bot design.

Peer Reviewed

The wide adoption of platformized work has generated remarkable advancements in the labor patterns and mobility of modern society. Underpinning such progress, gig workers are exposed to unprecedented challenges and accountabilities: lack of data transparency, social and physical isolation, as well as insufficient infrastructural safeguards. Gig2Gether presents a space designed for workers to engage in an initial experience of voluntarily contributing anecdotal and statistical data to affect policy and build solidarity across platforms by exchanging unifying and diverse experiences. Our 7-day field study with 16 active workers from three distinct platforms and work domains showed existing affordances of data-sharing: facilitating mutual support across platforms, as well as enabling financial reflection and planning. Additionally, workers envisioned future use cases of data-sharing for collectivism (e.g., collaborative examinations of algorithmic speculations) and informing policy (e.g., around safety and pay), which motivated (latent) worker desiderata of additional capabilities and data metrics. Based on these findings, we discuss remaining challenges to address and how data-sharing tools can complement existing structures to maximize worker empowerment and policy impact.

Equity is crucial to the ethical implications in technology development. However, implementing equity in practice comes with complexities and nuances. In response, the research community, especially the human-computer interaction (HCI) and Fairness community, has endeavored to integrate equity into technology design, addressing issues of societal inequities. With such increasing efforts, it is yet unclear why and how researchers discuss equity and its integration into technology, what research has been conducted, and what gaps need to be addressed. We conducted a systematic literature review on equity and technology, collecting and analyzing 202 papers published in HCI and Fairness-focused venues. Amidst the substantial growth of relevant publications within the past four years, we deliver three main contributions: (1) we elaborate a comprehensive understanding researchers' motivations for studying equity and technology, (2) we illustrate the different equity definitions and frameworks utilized to discuss equity, (3) we characterize the key themes addressing interventions as well as tensions and trade-offs when advancing and integrating equity to technology. Based on our findings, we elaborate an equity framework for researchers who seek to address existing gaps and advance equity in technology.

A growing body of literature has focused on understanding and addressing workplace AI design failures. However, past work has largely overlooked the role of occupational devaluation in shaping the dynamics of AI development and deployment. In this paper, we examine the case of feminized labor: a class of devalued occupations historically misnomered as ``women's work,'' such as social work, K-12 teaching, and home healthcare. Drawing on literature on AI deployments in feminized labor contexts, we conceptualize AI Failure Loops: a set of interwoven, socio-technical failures that help explain how the systemic devaluation of workers' expertise negatively impacts, and is impacted by, AI design, evaluation, and governance practices. These failures demonstrate how misjudgments on the automatability of workers' skills can lead to AI deployments that fail to bring value and, instead, further diminish the visibility of workers' expertise. We discuss research and design implications for workplace AI, especially for devalued occupations.

Public sector agencies are rapidly deploying AI systems to augment or automate critical decisions in real-world contexts like child welfare, criminal justice, and public health. A growing body of work documents how these AI systems often fail to improve services in practice. These failures can often be traced to decisions made during the early stages of AI ideation and design, such as problem formulation. However, today, we lack systematic processes to support effective, early-stage decision-making about whether and under what conditions to move forward with a proposed AI project. To understand how to scaffold such processes in real-world settings, we worked with public sector agency leaders, AI developers, frontline workers, and community advocates across four public sector agencies and three community advocacy groups in the United States. Through an iterative co-design process, we created the Situate AI Guidebook: a structured process centered around a set of deliberation questions to scaffold conversations around (1) goals and intended use for a proposed AI system, (2) societal and legal considerations, (3) data and modeling constraints, and (4) organizational governance factors. We discuss how the guidebook’s design is informed by participants’ challenges, needs, and desires for improved deliberation processes. We further elaborate on implications for designing responsible AI toolkits in collaboration with public sector agency stakeholders and opportunities for future work to expand upon the guidebook. This design approach can be more broadly adopted to support the co-creation of responsible AI toolkits that scaffold key decision-making processes surrounding the use of AI in the public sector and beyond.

AI tools are increasingly deployed in community contexts. However, datasets used to evaluate AI are typically created by developers and annotators outside a given community, which can yield misleading conclusions about AI performance. How might we empower communities to drive the intentional design and curation of evaluation datasets for AI that impacts them? We investigate this question on Wikipedia, an online community with multiple AI-based content moderation tools deployed. We introduce Wikibench, a system that enables communities to collaboratively curate AI evaluation datasets, while navigating ambiguities and differences in perspective through discussion. A field study on Wikipedia shows that datasets curated using Wikibench can effectively capture community consensus, disagreement, and uncertainty. Furthermore, study participants used Wikibench to shape the overall data curation process, including refining label definitions, determining data inclusion criteria, and authoring data statements. Based on our findings, we propose future directions for systems that support community-driven data curation.

Online platforms provide support for many kinds of distress, including suicidal thoughts and behaviors. However, because many platforms restrict suicidal talk, volunteers on these platforms struggle with how to help suicidal people who come for support. We interviewed 11 volunteer counselors in a large online support platform, including after they role-played conversations with varying severities of suicidality, to explore practices and challenges when identifying and responding to suicidality. We then presented Speed Dating design concepts around emotional preparation and support, real-time guidance, training, and suicide detection. Participants wanted more support and preparation for conversations with suicidal people, but were conflicted about AI-based technologies, including trade-offs between potential benefits of conversational agents for training and limitations of prediction or real-time response suggestions, due to the sensitive, context-dependent decisions that volunteers must make. Our work has important implications for nuanced considerations and design choices around developing digital mental health technologies.

LGBTQ+ people have received increased attention in HCI research, paralleling a greater emphasis on social justice in recent years. However, there has not been a systematic review of how LGBTQ+ people are researched or discussed in HCI. In this work, we review all research mentioning LGBTQ+ people across the HCI venues of CHI, CSCW, DIS, and TOCHI. Since 2014, we find a linear growth in the number of papers substantially about LGBTQ+ people and an exponential increase in the number of mentions. Research about LGBTQ+ people tends to center experiences of being politicized, outside the norm, stigmatized, or highly vulnerable. LGBTQ+ people are typically mentioned as a marginalized group or an area of future research. We identify gaps and opportunities for (1) research about and (2) the discussion of LGBTQ+ in HCI and provide a dataset to facilitate future Queer HCI research.

As public sector agencies rapidly introduce new AI tools in high-stakes domains like social services, it becomes critical to understand how decisions to adopt these tools are made in practice. We borrow from the anthropological practice to "study up" those in positions of power, and reorient our study of public sector AI around those who have the power and responsibility to make decisions about the role that AI tools will play in their agency. Through semi-structured interviews and design activities with 16 agency decision-makers, we examine how decisions about AI design and adoption are influenced by their interactions with and assumptions about other actors within these agencies (e.g., frontline workers and agency leaders), as well as those above (legal systems and contracted companies), and below (impacted communities). By centering these networks of power relations, our findings shed light on how infrastructural, legal, and social factors create barriers and disincentives to the involvement of a broader range of stakeholders in decisions about AI design and adoption. Agency decision-makers desired more practical support for stakeholder involvement around public sector AI to help overcome the knowledge and power differentials they perceived between them and other stakeholders (e.g., frontline workers and impacted community members). Building on these findings, we discuss implications for future research and policy around actualizing participatory AI approaches in public sector contexts.

Public sector agencies aim to innovate not just for efficiency but also to enhance equity. Despite the growing adoption of data-driven decision-making systems in the public sector, efforts to integrate equity as a primary goal often fall short. This typically arises from inadequate early-stage involvement of underserved stakeholders and prevalent misunderstandings concerning the authentic meaning of equity from these stakeholders' perspectives. Our research seeks to address this gap by actively involving undersevered stakeholders in the process of envisioning the integration of equity within public sector data-driven decisions, particularly in the context of a building department in a Northeastern mid-sized U.S. city. Applying a speed dating method with storyboards, we explore diverse equity-centric futures within the realm of local business development, a domain where small businesses, particularly women-and minority-owned businesses, historically confront inequitable distribution of public services. We explored three essential aspects of equity: monitoring equity, resource allocation prioritization, as well as information and equity. Our findings illuminate the complexities of integrating equity into data-driven decisions, offering nuanced insights about the needs of stakeholders. We found that attempts to monitor and incorporate equity goals into public sector decision-making can unexpectedly backfire, inadvertently sparking community apprehension and potentially exacerbating existing inequities. Small business owners, including those identifying as women-and minority-owned, advocated against the use of demographic-based data in equity-focused data-driven decision-making in the public sector, instead emphasizing factors such as community needs, application complexity, and uncertainties inherent in small businesses. Drawing from these insights, we propose design implications to assist designers of public sector data-driven decision-making systems to better accommodate equity considerations.

Online mental health support communities, in which volunteer counselors provide accessible mental and emotional health support, have grown in recent years. Despite millions of people using these platforms, the clinical effectiveness of these communities on mental health symptoms remains unknown. Although volunteers receive some training on the therapeutic skills proven effective in face-to-face environments, such as active listening and motivational interviewing, it is unclear how the usage of these skills in an online context affects people's mental health. In our work, we collaborate with one of the largest online peer support platforms and use both natural language processing and machine learning techniques to examine how one-on-one support chats on the platform affect clients' depression and anxiety symptoms. We measure how characteristics of support-providers, such as their experience on the platform and use of therapeutic skills (e.g. affirmation, showing empathy), affect support-seekers' mental health changes. Based on a propensity-score matching analysis to approximate a random-assignment experiment, results shows that online peer support chats improve both depression and anxiety symptoms with a statistically significant but relatively small effect size. Additionally, support providers' techniques such as emphasizing the autonomy of the client lead to better mental health outcomes. However, we also found that the use of some behaviors, such as persuading and providing information, are associated with worsening of mental health symptoms. Our work provides key understanding for mental health care in the online setting and designing training systems for online support providers.

HCI scholars are increasingly engaging in research about “marginalized groups,” such as LGBTQ+ people. While normative habitual readings of marginalized people in HCI often highlight real problems, this work has been criticized for flattening heterogeneous experiences and overemphasizing harms. Some have advocated for expanding how we approach research on marginalized people (e.g., assets-based design, the everyday, and joy). Sensitized by unmaking literature, we explore this tension between conditions, experiences, and representations of marginality in HCI scholarship. To do so, we perform a diffractive analysis of posts in a gay online community by bringing two readings of the same data together: a normative habitual reading of marginalization and an expanded reading. By examining the relationship between empirical material and its representations by HCI researchers, we explore how to carefully unmake HCI research, thus maintaining and repairing our research community. We discuss the political and designerly implications of different readings of marginalized people and offer considerations for attending to the processes and afterlives of HCI research.

Gig workers, and the products and services they provide, play an increasingly ubiquitous role in our daily lives. But despite growing evidence suggesting that worker well-being in gig economy platforms have become significant societal problems, few studies have investigated possible solutions. We take a stride in this direction by engaging workers, platform employees, and local regulators in a series of speed dating workshops using storyboards based on real-life situations to rapidly elicit stakeholder preferences for addressing financial, physical, and social issues related to worker well-being. Our results reveal that existing public and platformic infrastructures fall short in providing workers with resources needed to perform gigs, surfacing a need for multi-platform collaborations, technological innovations, as well as changes in regulations, labor laws, and the public’s perception of gig workers, among others. Drawing from multi-stakeholder findings, we discuss these implications for technology, policy, and service as well as avenues for collaboration.

On-demand work platforms are attractive alternatives to traditional employment arrangements. However, several questions around employment classification, compensation, data privacy, and equitable outcomes remain open. The abilities of algorithmic management to structure different forms of platform-worker relationships compounds fraught regulatory debates. Understanding the conditions of algorithmic management that result in these variations could point us towards better worker futures. In this work, we studied the platform-worker relationships in Instacart work through the accounts of its workers. From a qualitative analysis of 400 Reddit posts by Instacart’s workers, we identified sources and types of ambiguity that gave rise to open-ended experiences for workers. Ambiguities supplemented gamification mechanisms to regulate worker behaviors. Yet, they also generated affective experiences for workers that enabled their playful participation in the Reddit community. We propose the frame of ludification to explain these seemingly contradicting findings and conclude with implications for accountability in on-demand work platforms.

Media coverage has historically played an influential and often stigmatizing role in the public’s understanding of mental illness through harmful language and inaccurate portrayals of those with mental health issues. However, it is unknown how and to what extent media events may affect stigma in online discourse regarding mental health. In this study, we examine a highly publicized event – the celebrity defamation trial between Johnny Depp and Amber Heard – to uncover how stigmatizing and destigmatizing language on Twitter changed during and after the course of the trial. Using causal impact and language analysis methods, we provided a first look at how external events can lead to significantly greater levels of stigmatization and lower levels of destigmatization on Twitter towards not only particular disorders targeted in the coverage of external events but also general mental health discourse.

Recent years have seen growing adoption of AI-based decision-support systems (ADS) in homeless services, yet we know little about stakeholder desires and concerns surrounding their use. In this work, we aim to understand impacted stakeholders’ perspectives on a deployed ADS that prioritizes scarce housing resources. We employed AI lifecycle comicboarding, an adapted version of the comicboarding method, to elicit stakeholder feedback and design ideas across various components of an AI system’s design. We elicited feedback from county workers who operate the ADS daily, service providers whose work is directly impacted by the ADS, and unhoused individuals in the region. Our participants shared concerns and design suggestions around the AI system’s overall objective, specific model design choices, dataset selection, and use in deployment. Our findings demonstrate that stakeholders, even without AI knowledge, can provide specific and critical feedback on an AI system’s design and deployment, if empowered to do so.

Much of our modern digital infrastructure relies critically upon open sourced software. The communities responsible for building this cyberinfrastructure require maintenance and moderation, which is often supported by volunteer efforts. Moderation, as a non-technical form of labor, is a necessary but often overlooked task that maintainers undertake to sustain the community around an OSS project. This study examines the various structures and norms that support community moderation, describes the strategies moderators use to mitigate conflicts, and assesses how bots can play a role in assisting these processes. We interviewed 14 practitioners to uncover existing moderation practices and ways that automation can provide assistance. Our main contributions include a characterization of moderated content in OSS projects, moderation techniques, as well as perceptions of and recommendations for improving the automation of moderation tasks. We hope that these findings will inform the implementation of more effective moderation practices in open source communities.

Data-driven AI systems are increasingly used to augment human decision-making in complex, social contexts, such as social work or legal practice. Yet, most existing design knowledge regarding how to best support AI-augmented decision-making comes from studies in comparatively well-defined settings. In this paper, we present findings from design interviews with 12 social workers who use an algorithmic decision support tool (ADS) to assist their day-to-day child maltreatment screening decisions. We generated a range of design concepts, each envisioning different ways of redesigning or augmenting the ADS interface. Overall, workers desired ways to understand the risk score and incorporate contextual knowledge, which move beyond existing notions of AI interpretability. Conversations around our design concepts also surfaced more fundamental concerns around the assumptions underlying statistical prediction, such as inference based on similar historical cases and statistical notions of uncertainty. Based on our findings, we discuss how ADS may be better designed to support the roles of human decision-makers in social decision-making contexts.

Machine learning tools have been deployed in various contexts to support human decision-making, in the hope that human-algorithm collaboration can improve decision quality. However, the question of whether such collaborations reduce or exacerbate biases in decision-making remains underexplored. In this work, we conducted a mixed-methods study, analyzing child welfare call screen workers’ decision-making over a span of four years, and interviewing them on how they incorporate algorithmic predictions into their decision-making process. Our data analysis shows that, compared to the algorithm alone, workers reduced the disparity in screen-in rate between Black and white children from 20% to 9%. Our qualitative data show that workers achieved this by making holistic risk assessments and adjusting for the algorithm’s limitations. Our analyses also show more nuanced results about how human-algorithm collaboration affects prediction accuracy, and how to measure these effects. These results shed light on potential mechanisms for improving human-algorithm collaboration in high-risk decision-making contexts.

As more individuals consider permanently working from home, the online labor market continues to grow as an alternative working environment. While the flexibility and autonomy of these online gigs attracts many workers, success depends critically upon self-management and workers’ efficient allocation of scarce resources. To achieve this, freelancers may develop alternative work strategies, employing highly standardized schedules and communication patterns while taking on large work volumes, or engaging in smaller numbers of jobs whilst tailoring their activities to build relationships with individual employers. In this study, we consider this contrast in relation to worker communication patterns. We demonstrate the heterogeneous effects of standardization versus personalization across different stages of a project and examine the relative impact on job acquisition, project completion, and earnings. Our findings can inform the design of platforms and various worker support tools for the gig economy.

AI-based decision support tools (ADS) are increasingly used to augment human decision-making in high-stakes, social contexts. As public sector agencies begin to adopt ADS, it is critical that we understand workers’ experiences with these systems in practice. In this paper, we present findings from a series of interviews and contextual inquiries at a child welfare agency, to understand how they currently make AI-assisted child maltreatment screening decisions. Overall, we observe how workers’ reliance upon the ADS is guided by (1) their knowledge of rich, contextual information beyond what the AI model captures, (2) their beliefs about the ADS’s capabilities and limitations relative to their own, (3) organizational pressures and incentives around the use of the ADS, and (4) awareness of misalignments between algorithmic predictions and their own decision-making objectives. Drawing upon these findings, we discuss design implications towards supporting more effective human-AI decision-making.

Many AI system designers grapple with how best to collect human input for different types of training data. Online crowds provide a cheap on-demand source of intelligence, but they often lack the expertise required in many domains. Experts offer tacit knowledge and more nuanced input, but they are harder to recruit. To explore this trade off, we compared novices and experts in terms of performance and perceptions on human intelligence tasks in the context of designing a text-based conversational agent. We developed a preliminary chatbot that simulates conversations with someone seeking mental health advice to help educate volunteer listeners at 7cups.com. We then recruited experienced listeners (domain experts) and MTurk novice workers (crowd workers) to conduct tasks to improve the chatbot with different levels of complexity. Novice crowds perform comparably to experts on tasks that only require natural language understanding, such as correcting how the system classifies a user statement. For more generative tasks, like creating new lines of chatbot dialogue, the experts demonstrated higher quality, novelty, and emotion. We also uncovered a motivational gap: crowd workers enjoyed the interactive tasks, while experts found the work to be tedious and repetitive. We offer design considerations for allocating crowd workers and experts on input tasks for AI systems, and for better motivating experts to participate in low-level data work for AI.

Online mental health communities (OMHCs) have emerged in recent years as an effective and accessible way to obtain peer support, filling crucial gaps of traditional mental health resources. However, the mechanisms for users to find relationships that fulfill their needs and capabilities in these communities are highly underdeveloped. Using a mixed-methods approach of user interviews and behavioral log analysis on 7Cups.com, we explore central challenges in finding adequate peer relationships in online support platforms and how algorithmic matching can alleviate many of these issues. We measure the impact of using qualities like gender and age in purposeful matching to improve member experiences, with especially salient results for users belonging to vulnerable populations. Lastly, we note key considerations for designing matching systems in the online mental health context, such as the necessity for better moderation to avoid potential harassment behaviors exacerbated by algorithmic matching. Our findings yield key insights into current user experiences in OMHCs as well as design implications for building matching systems in the future for OMHCs.

Online peer-to-peer therapy sessions can be effective in improving people's mental well-being. However, online volunteer counselors may lack the expertise and necessary training to provide high-quality sessions, and these low-quality sessions may negatively impact volunteers' motivations as well as clients' well-being. This paper uses interviews with 20 senior online volunteer counselors to examine how they addressed challenges and acquired skills when volunteering in a large, mental-health support community - 7Cups.com. Although volunteers in this community received some training based on principles of active listening and motivational interviewing, results indicate that the training was insufficient and that volunteer counselors had to independently develop strategies to deal with specific challenges that they encountered in their volunteer work. Their strategies, however, might deviate from standard practice since they generally lacked systematic feedback from mentors or clients and, instead, relied on their personal experiences. Additionally, volunteer counselors reported having difficulty maintaining their professional boundaries with the clients. Even though training and support resources were available, they were underutilized. The results of this study have uncovered new design spaces for HCI practitioners and researchers, including social computing and artificial intelligence approaches that may provide better support to volunteer counselors in online mental health communities.

With the growing industry applications of Artificial Intelligence (AI) systems, pre-trained models and APIs have emerged and greatly lowered the barrier of building AI-powered products. However, novice AI application designers often struggle to recognize the inherent algorithmic trade-offs and evaluate model fairness before making informed design decisions. In this study, we examined the Objective Revision Evaluation System (ORES), a machine learning (ML) API in Wikipedia used by the community to build anti-vandalism tools. We designed an interactive visualization system to communicate model threshold trade-offs and fairness in ORES. We evaluated our system by conducting 10 in-depth interviews with potential ORES application designers. We found that our system helped application designers who have limited ML backgrounds learn about in-context ML knowledge, recognize inherent value trade-offs, and make design decisions that aligned with their goals. By demonstrating our system in a real-world domain, this paper presents a novel visualization approach to facilitate greater accessibility and human agency in AI application design.

Recent work in fair machine learning has proposed dozens of technical definitions of algorithmic fairness and methods for enforcing these definitions. However, we still lack an understanding of how to develop machine learning systems with fairness criteria that reflect relevant stakeholders’ nuanced viewpoints in real-world contexts. To address this gap, we propose a framework for eliciting stakeholders’ subjective fairness notions. Combining a user interface that allows stakeholders to examine the data and the algorithm’s predictions with an interview protocol to probe stakeholders’ thoughts while they are interacting with the interface, we can identify stakeholders’ fairness beliefs and principles. We conduct a user study to evaluate our framework in the setting of a child maltreatment predictive system. Our evaluations show that the framework allows stakeholders to comprehensively convey their fairness viewpoints. We also discuss how our results can inform the design of predictive systems.

Artificial intelligence (AI) technology has been increasingly used in the implementation of advanced Clinical Decision Support Systems (CDSS). Research demonstrated the potential usefulness of AI-powered CDSS (AI-CDSS) in clinical decision making scenarios. However, post-adoption user perception and experience remain understudied, especially in developing countries. Through observations and interviews with 22 clinicians from 6 rural clinics in China, this paper reports the various tensions between the design of an AI-CDSS system (“Brilliant Doctor”) and the rural clinical context, such as the misalignment with local context and workflow, the technical limitations and usability barriers, as well as issues related to transparency and trustworthiness of AI-CDSS. Despite these tensions, all participants expressed positive attitudes toward the future of AI-CDSS, especially acting as “a doctor’s AI assistant” to realize a Human-AI Collaboration future in clinical settings. Finally we draw on our findings to discuss implications for designing AI-CDSS interventions for rural clinical contexts in developing countries.

Bulk email is a primary communication channel within organizations, with all-company emails and regular newsletters serving as a mechanism for making employees aware of policies and events. Ineffective communication could result in wasted employee time and a lack of compliance or awareness. Previous studies on organizational emails focused mostly on recipients. However, organizational bulk email system is a multi-stakeholder problem including recipients, communicators, and the organization itself. We studied the effectiveness, practice, and assessments of the organizational bulk email system of a large university from multi-stakeholders' perspectives. We conducted a qualitative study with the university's communicators, recipients, and managers. We delved into the organizational bulk email's distributing mechanisms of the communicators, the reading behaviors of recipients, and the perspectives on emails' values of communicators, managers, and recipients. We found that the organizational bulk email system as a whole was strained, and communicators are caught in the middle of this multi-stakeholder problem. First, though the communicators had an interest in preserving the effectiveness of channels in reaching employees, they had high-level clients whose interests might outweigh judgment about whether a message deserves widespread circulation. Second, though communicators thought they were sending important information, recipients viewed most of the organizational bulk emails as not relevant to them. Third, this disagreement was amplified by the success metric used by communicators. They viewed their bulk emails as successful if they had a high open rate. But recipients often opened and then rapidly discarded emails without reading the details. Last, while the communicators in general understood the challenge, they had a limited set of targeting and feedback tools to support their task.

The individualistic nature of gig work allows workers to have high levels of flexibility, but it also leads to atomization, leaving them isolated from peer workers. In this paper, we employed a qualitative approach to understand how online social media groups provide informational and emotional support to physical gig workers during the COVID-19 pandemic. We found that social media groups alleviate the atomization effect, as workers use these groups to obtain experiential knowledge from their peers, build connections, and organize collective action. However, we noted a reluctance among workers to share strategic information where there was a perceived risk of being competitively disadvantaged. In addition, we found that the diversity among gig workers has also led to limited empathy for one another, which further impedes the provision of emotional support. While social media groups could potentially become places where workers organize collective efforts, several factors, including the uncertainty of other workers' activities and the understanding of the independent contractor status, have diminished the effectiveness of efforts at collective action.

Online health communities (OHCs) have become important resources from which members can obtain social support. Since most benefits of OHCs are provided by its members, it is crucial that OHCs maintain a critical mass of active members. This paper examines temporal changes in members' participation in a cancer-orientedOHC, focusing on the changes in members' motivations and behavior as they transition from newcomers to other roles or when they ultimately leave the community. Our work used mixed methods, combining behavioral log analysis, automated content analysis, surveys and interviews. We found that shifts in members' motivations seemed to be driven by two sources: the internal dynamics common to becoming a member of most online communities and the external needs associated with their cancer journey. When members' disease-driven needs for support decreased, most members quit the site. The motivations of those who stayed shifted from receiving support to providing it to others in the community. As in many online communities, old-timers contributed the vast majority of content. However, they encountered challenges that threatened their commitment, including negative emotions related to other members' deaths, which led them to take leaves of absence from the community or to drop out permanently. Implications for the motivation changes ofOHC members are discussed.

Artificial intelligence algorithms have been used to enhance a wide variety of products and services, including assisting human decision making in high-stake contexts. However, these algorithms are complex and have trade-offs, notably between prediction accuracy and fairness to population subgroups. This makes it hard for designers to understand algorithms and design products or services in a way that respects users' goals, values, and needs. We proposed a method to help designers and users explore algorithms, visualize their trade-offs, and select algorithms with trade-offs consistent with their goals and needs. We evaluated our method on the problem of predicting criminal defendants' likelihood to re-offend through (i) a large-scale Amazon Mechanical Turk experiment, and (ii) in-depth interviews with domain experts. Our evaluations show that our method can help designers and users of these systems better understand and navigate algorithmic trade-offs. This paper contributes a new way of providing designers the ability to understand and control the outcomes of algorithmic systems they are creating.

Algorithmic decision-making systems are increasingly used throughout the public and private sectors to make important decisions or assist humans in making these decisions with real social consequences. While there has been substantial research in recent years to build fair decision-making algorithms, there has been less research seeking to understand the factors that affect people's perceptions of fairness in these systems, which we argue is also important for their broader acceptance. In this research, we conduct an online experiment to better understand perceptions of fairness, focusing on three sets of factors: algorithm outcomes, algorithm development and deployment procedures, and individual differences. We find that people rate the algorithm as more fair when the algorithm predicts in their favor, even surpassing the negative effects of describing algorithms that are very biased against particular demographic groups. We find that this effect is moderated by several variables, including participants' education level, gender, and several aspects of the development procedure. Our findings suggest that systems that evaluate algorithmic fairness through users' feedback must consider the possibility of "outcome favorability" bias.

We conduct a study of hiring bias on a simulation platform where we ask Amazon MTurk participants to make hiring decisions for a mathematically intensive task. Our findings suggest hiring biases against Black workers and less attractive workers, and preferences towards Asian workers, female workers and more attractive workers. We also show that certain UI designs, including provision of candidates' information at the individual level and reducing the number of choices, can significantly reduce discrimination. However, provision of candidate's information at the subgroup level can increase discrimination. The results have practical implications for designing better online freelance marketplaces.

Mass media afford researchers critical opportunities to disseminate research findings and trends to the general public. Yet researchers also perceive that their work can be miscommunicated in mass media, thus generating unintended understandings of HCI research by the general public. We conduct a Grounded Theory analysis of interviews with 12 HCI researchers and find that miscommunication can occur at four origins along the socio-technical infrastructure known as the Media Production Pipeline (MPP) for science news. Results yield researchers' perceived hazards of disseminating their work through mass media, as well as strategies for fostering effective communication of research. We conclude with implications for augmenting or innovating new MPP technologies.

On Wikipedia, sophisticated algorithmic tools are used to assess the quality of edits and take corrective actions. However, algorithms can fail to solve the problems they were designed for if they conflict with the values of communities who use them. In this study, we take a Value-Sensitive Algorithm Design approach to understanding a community-created and -maintained machine learning-based algorithm called the Objective Revision Evaluation System (ORES)---a quality prediction system used in numerous Wikipedia applications and contexts. Five major values converged across stakeholder groups that ORES (and its dependent applications) should: (1) reduce the effort of community maintenance, (2) maintain human judgement as the final authority, (3) support differing peoples' differing workflows, (4) encourage positive engagement with diverse editor groups, and (5) establish trustworthiness of people and algorithms within the community. We reveal tensions between these values and discuss implications for future research to improve algorithms like ORES.

McCarthy (1998) defined Artificial Intelligence (AI) as both “the science and engineering of in- telligent machines, especially computer programs” and the “computational part of the ability to achi...

Ensuring effective public understanding of algorithmic decisions that are powered by machine learning techniques has become an urgent task with the increasing deployment of AI systems into our society. In this work, we present a concrete step toward this goal by redesigning confusion matrices for binary classification to support non-experts in understanding the performance of machine learning models. Through interviews (n=7) and a survey (n=102), we mapped out two major sets of challenges lay people have in understanding standard confusion matrices: the general terminologies and the matrix design. We further identified three sub-challenges regarding the matrix design, namely, confusion about the direction of reading the data, layered relations and quantities involved. We then conducted an online experiment with 483 participants to evaluate how effective a series of alternative representations target each of those challenges in the context of an algorithm for making recidivism predictions. We developed three levels of questions to evaluate users' objective understanding. We assessed the effectiveness of our alternatives for accuracy in answering those questions, completion time, and subjective understanding. Our results suggest that (1) only by contextualizing terminologies can we significantly improve users' understanding and (2) flow charts, which help point out the direction of reading the data, were most useful in improving objective understanding. Our findings set the stage for developing more intuitive and generally understandable representations of the performance of machine learning models.

Toxicity in online discussions has been an intriguing phenomenon and an important problem. In this paper, we seek to better understand toxicity dynamics in online discussions via a case study on Reddit that explores the antecedents and consequences of toxicity in text. We inspected two dimensions of toxicity: language toxicity, i.e. how toxic the text itself is; and toxicity elicitation, i.e. how much toxicity it elicits in its response. Through regression analyses on Reddit comments, we found that both author propensity and toxicity in discussion context were strong positive antecedents of language toxicity; meanwhile, language toxicity significantly increased the volume and user evaluation of the discussion in some sub-communities, while toxicity elicitation showed mixed effects. We then discuss how our results help understand and regulate toxicity in online discussions by interpreting the complicated triggers and outcomes of toxicity.

Increasingly, algorithms are used to make important decisions across society. However, these algorithms are usually poorly understood, which can reduce transparency and evoke negative emotions. In this research, we seek to learn design principles for explanation interfaces that communicate how decision-making algorithms work, in order to help organizations explain their decisions to stakeholders, or to support users' "right to explanation". We conducted an online experiment where 199 participants used different explanation interfaces to understand an algorithm for making university admissions decisions. We measured users' objective and self-reported understanding of the algorithm. Our results show that both interactive explanations and "white-box" explanations (i.e. that show the inner workings of an algorithm) can improve users' comprehension. Although the interactive approach is more effective at improving comprehension, it comes with a trade-off of taking more time. Surprisingly, we also find that users' trust in algorithmic decisions is not affected by the explanation interface or their level of comprehension of the algorithm.

To support ethical considerations and system integrity in learning analytics, this paper introduces two cases of applying the Value Sensitive Design methodology to learning analytics design. The first study applied two methods of Value Sensitive Design, namely stakeholder analysis and value analysis, to a conceptual investigation of an existing learning analytics tool. This investigation uncovered a number of values and value tensions, leading to design trade-offs to be considered in future tool refinements. The second study holistically applied Value Sensitive Design to the design of a recommendation system for the Wikipedia WikiProjects. To proactively consider values among stakeholders, we derived a multi-stage design process that included literature analysis, empirical investigations, prototype development, community engagement, iterative testing and refinement, and continuous evaluation. By reporting on these two cases, this paper responds to a need of practical means to support ethical considerations and human values in learning analytics systems. These two cases demonstrate that Value Sensitive Design could be a viable approach for balancing a wide range of human values, which tend to encompass and surpass ethical issues, in learning analytics design.

Group projects are an essential component of teaching user interface (UI) design. We identified six challenges in transferring traditional group projects into the context of Massive Open Online Courses: managing dropout, avoiding free-riding, appropriate scaffolding, cultural and time zone differences, and establishing common ground. We present a case study of the design of a group project for a UI Design MOOC, in which we implemented technical tools and social structures to cope with the above challenges. Based on survey analysis, interviews, and team chat data from the students over a six-month period, we found that our socio-technical design addressed many of the obstacles that MOOC learners encountered during remote collaboration. We translate our findings into design implications for better group learning experiences at scale.

Online platforms provide new channels for people in need to seek help from friends and strangers. However, individuals often encounter psychological barriers that deter them from asking for help. For example, people might have different concerns about asking for help, including acknowledging incompetence, bothering others, and accruing social debt. These perceived social costs limit the potential benefits of help solicitations. In this study, we attempt to investigate whether anonymity (posting a question anonymously), ephemerality (allowing questions to be visible for only a short period), and system routing (having the system handle the question routing) could reduce social costs in a typical online help-seeking behavior-question asking. We built a platform to support these three features and conducted a controlled within-subjects experiment to test their effects on the social costs of posting questions. Results suggest that the presence of anonymity, ephemerality, and system routing reduce social costs. Further, we find that employing anonymity and system routing features did not lower the quality and quantity of answers to the questions in our system.

Understanding team communication and collaboration patterns is critical for improving work efficiency in organizations. This paper presents an interactive visualization system, T-Cal, that supports the analysis of conversation data from modern team messaging platforms (e.g., Slack). T-Cal employs a user-familiar visual interface, a calendar, to enable seamless multi-scale browsing of data from different perspectives. T-Cal also incorporates a number of analytical techniques for disentangling interleaving conversations, extracting keywords, and estimating sentiment. The design of T-Cal is based on an iterative user-centered design process including interview studies, requirements gathering, initial prototypes demonstration, and evaluation with domain users. The resulting two case studies indicate the effectiveness and usefulness of T-Cal in real-world applications, including daily conversations within an industry research lab and student group chats in a MOOC.

Organizers of online groups often struggle to recruit members who can most effectively carry out the group's activities and remain part of the group over time. In a study of a sample of 30,000 new editors belonging to 1,054 English WikiProjects, we empirically examine the effects of generalized prior work-productivity experience (measured by overall prior article edits), prior leadership experience (measured by overall prior project edits), and localized prior work-productivity experience (measured by pre-joining article edits on a project) on early retention and productivity. We find that (1)generalized prior work-productivity experience is positively associated with retention, but negatively associated with productivity (2) prior leadership experience is negatively associated with both retention and productivity, and (3) localized prior work-productivity experience is positively associated with both retention and productivity within that focal project. We then discuss implications to inform the designs of early interventions aimed at group success.

Contemporary scientific media production requires a complex socio-technical infrastructure we call the "Media Production Pipeline" (MPP). Media professionals engage with researchers along the MPP to disseminate science news to the lay public. However, differing incentive structures and professional contexts frequently set researchers' values and needs at odds with those of media professionals, resulting in problematic or failed interactions. We ask the research question: what pain points in scientific media production afford opportunities for future HCI innovation? We then present a grounded theory analysis of 24 interviews with researchers and media professionals, yielding several key contributions. First, we describe two collaborative domains in scientific media production between research advocates and media outlets. Second, we characterize discrete technological gaps and pain points in both domains. Finally, we discuss implications for design and propose solutions from HCI areas like peer production, online communities, recommender systems, and online collaboration.

Most commonly used approaches to developing automated or artificially intelligent algorithmic systems are Big Data-driven and machine learning-based. However, these approaches can fail, for two notable reasons: (1) they may lack critical engagement with users and other stakeholders; (2) they rely largely on historical human judgments, which do not capture and incorporate human insights into how the world can be improved in the future. We propose and describe a novel method for the design of such algorithms, which we call Value Sensitive Algorithm Design. Value Sensitive Algorithm Design incorporates stakeholders' tacit knowledge and explicit feedback in the early stages of algorithm creation. This increases the chance to avoid biases in design choices or to compromise key stakeholder values. Generally, we believe that algorithms should be designed to balance multiple stakeholders' needs, motivations, and interests, and to help achieve important collective goals. We also describe a specific project "Designing Intelligent Socialization Algorithms for WikiProjects in Wikipedia" to illustrate our method. We intend this paper to contribute to the rich ongoing conversation concerning the use of algorithms in supporting critical decision-making in society.

Productive and dedicated members are critical to the success of online production communities like Wikipedia. Many communities organize in subgroups where members voluntarily work on projects of shared interest. In this paper, we investigate how members' pre-joining connections with the subgroup predict their productivity and withdrawal after joining. Drawing insights from attachment theories in social psychology, we examine two types of pre-joining connections: textit{identity-based} attachment (how much members' interests were aligned with the subgroup's topics) and textit{bonds-based} attachment (how much members had interacted with other members of the subgroup). Analyses of 79,704 editors in 1,341 WikiProjects show that 1) both identity-based and bonds-based attachment increased editors' post-joining productivity and reduced their likelihood of withdrawal; 2) identity-based attachment had a stronger effect on boosting direct contributions to articles while bonds-based attachment had a stronger effect on increasing article and project coordination, and reducing member withdrawal.

Team project, which emphasizes collaborative learning in a project-based context, is one of the most commonly-used teaching and learning methods in higher education classrooms, but is not well-supported on existing Massive Open Online Course (MOOC) platforms. In this paper, we present ProjectLens, a MOOC supplement tool that supports team projects building and collaborative learning on MOOC platforms like Coursera and edX. In addition, ProjectLens is a research tool that provides opportunities to conduct large-scale field experiments to study how different factors influence the effectiveness of collaborative learning. We illustrate how ProjectLens can achieve these two goals in a case example.

The sharing economy has quickly become a very prominent subject of research in the broader computing literature and the in human--computer interaction (HCI) literature more specifically. When other computing research areas have experienced similarly rapid growth (e.g. human computation, eco-feedback technology), early stage literature reviews have proved useful and influential by identifying trends and gaps in the literature of interest and by providing key directions for short- and long-term future work. In this paper, we seek to provide the same benefits with respect to computing research on the sharing economy. Specifically, following the suggested approach of prior computing literature reviews, we conducted a systematic review of sharing economy articles published in the Association for Computing Machinery Digital Library to investigate the state of sharing economy research in computing. We performed this review with two simultaneous foci: a broad focus toward the computing literature more generally and a narrow focus specifically on HCI literature. We collected a total of 112 sharing economy articles published between 2008 and 2017 and through our analysis of these papers, we make two core contributions: (1) an understanding of the computing community's contributions to our knowledge about the sharing economy, and specifically the role of the HCI community in these contributions (i.e. what has been done ) and (2) a discussion of under-explored and unexplored aspects of the sharing economy that can serve as a partial research agenda moving forward (i.e. what is next to do ).

Although Couchsurfing and Airbnb are both online communities that help users host strangers in their homes, they differ in an important sense: Couchsurfing prohibits monetary payment while Airbnb is built around it.We conducted interviews with users experienced on both Couchsurfing and Airbnb ("dual-users") to better understand systemic differences between the platforms. Based on these interviews we propose that, compared to Couchsurfing, Airbnb: (1) appears to require higher quality services, (2) places more emphasis on places over people, and (3) shifts social power from hosts to guests. Using public profiles from both platforms, we present analyses exploring each theme. Finally, we present evidence showing that Airbnb's growth has coincided with a decline in Couchsurfing. Taken together, our findings paint a complex picture of the changing character of network hospitality.

Exploiting evidence that sporting results affect fans' mood, we analyze whether National Football League game outcomes can affect the contributions of Wikipedia editors who identify as fans of a specific team. We find that the day after a team loses, their fans decrease their contributions towards football-related pages (relative to after a win). Relative decreases are bigger if losses are unexpected, or if losing margins are big. In contrast, unexpected wins do not cause more contributions relative to wins that were not unexpected. Neither do big wins result in more contributions relative to small wins. Additionally, contributions to non-football-related pages are not affected by NFL game results. Our findings add to the literatures on (i) the determinants of individual contributions to peer production communities, (ii) how community dynamics affect user contributions, (iii) the importance of emotions, (iv) the effect of offline events on online behavior, and (v) the applicability of behavioral economics concepts to the HCI literature.

CoCoRaHS is a multinational citizen science project for observing precipitation. Like many citizen science projects, volunteer retention is a key measure of engagement and data quality. Through survival analysis, we found that participant age (self-reported at account creation) is a significant predictor of retention. Compared to all other age groups, participants aged 60-70 are much more likely to sign up for CoCoRaHS, and to remain active for several years. We also measured the influence of task difficulty and the relative frequency of rain, finding small but statistically significant and counterintuitive effects. Finally, we confirmed previous work showing that participation levels within the first month are highly predictive of eventual retention. We conclude with implications for observational citizen science projects and crowdsourcing research in general.

Nearly any group is subject to turnover : some people leave, while others join. Turnover can be especially high in online groups, since participation typically is strictly voluntary. We investigated the effects of member turnover in online groups, specifically in Wikipedia's WikiProjects. We based our studies on theories from organizational science, which suggest that it is not just the amount of turnover, but the characteristics of those leaving and those joining that matter. We characterized leavers and newcomers by their prior productivity, tenure (in the group or community), and participation in other groups within the larger community. Furthermore, we considered the moderating effect of group size on turnover. We analyzed data from 88,427 editors who participated in 1,054 WikiProjects, finding that (1) the positive effects of newcomers to a group were larger than the negative effects of leavers, (2) prior productivity, tenure, and participation in other groups all played significant roles, and (3) the effects of leavers and newcomers were amplified in larger groups.

Nowadays people have many overlapping relationships across different social network sites. Will the communication frequency of a relationship on a focal SNS increase or decrease after the corresponding parties begin interacting with each other on a new SNS? What kinds of relationships and parties on the focal SNS are more robust to relationship overlapping across different sites? We conducted a case study on two Chinese popular social network sites (Weibo and Douban). Our results indicate that relationships' interactions on a new SNS have negative effect on the parties' communication frequency on the focal SNS. The communication frequency of older relationships on the focal SNS is more susceptible to the influence of the interacting on the new SNS, while relationships with more common groups and more extensive posts are less likely to be influenced. Our findings imply opportunities for SNS designers to strengthen their existing fragile ties and help users to build much more robust relationships.

Friendsourcing consists of broadcasting questions and help requests to friends on social networking sites. Despite its potential value, friendsourcing requests often fall on deaf ears. One way to improve response rates and motivate friends to undertake more effortful tasks may be to offer extrinsic rewards, such as money or a gift, for responding to friendsourcing requests. However, past research suggests that these extrinsic rewards can have unintended consequences, including undermining intrinsic motivations and undercutting the relationship between people. To explore the effects of extrinsic reward on friends' response rate and perceived relationship, we conducted an experiment on a new friendsourcing platform - Mobilyzr. Results indicate that large extrinsic rewards increase friends' response rates without reducing the relationship strength between friends. Additionally, the extrinsic rewards allow requesters to explain away the failure of friendsourcing requests and thus preserve their perceptions of relationship ties with friends.

In online collaboration projects, conflicts often arise in the peer review process, due to the disagreement over whether one's contribution should be accepted. These conflicts generally have detrimental effects on contributors' continuing participation in the community. Few studies have investigated how to manage these conflicts effectively. This paper aims to examine the effectiveness of three strategies - rational explanation, constructive suggestion, and social encouragement - in managing conflicts. In an analysis of 170 online software development projects, we investigated how different conflict management strategies aimed at handling contributors' arguments during the peer review process influenced their subsequent participation in the projects. The results show that (i) conflicts significantly increase contributors' likelihood of leaving the communities; (ii) neither rational explanations nor social encouragement could reduce the negative consequences of conflicts; (iii) only constructive suggestions have a positive effect in retaining the contributors.

Online communities, much like companies in the business world, often need to transfer 'best practices' internally from one unit to another to improve their performance. Organizational scholars disagree about how much a recipient unit should modify a best practice when incorporating it. Some evidence indicates that modifying a practice that has been successful in one environment will introduce problems, undercut its effectiveness and harm the performance of the recipient unit. Other evidence, though, suggests that recipients need to adapt the practice to fit their local environment. The current research introduces a contingency perspective on practice transfer, holding that the value of modifications depends on when they are introduced and who introduces them. Empirical research on the transfer of a quality-improvement practice between projects within Wikipedia shows that modifications are more helpful if they are introduced after the receiving project has had experience with the imported practice. Furthermore, modifications are more effective if they are introduced by members who have experience in a variety of other projects.

The gender gap in Wikipedia's content, specifically in the representation of women in biographies, is well-known but has been difficult to measure. Furthermore the impacts of efforts to address this gender gap have received little attention. To investigate we utilise Wikidata, the database that feeds Wikipedia, and introduce the "Wikidata Human Gender Indicators" (WHGI), a free and open source, longitudinal, biographical dataset monitoring gender disparities across time, space, culture, occupation and language. Through these lenses we show how the representation of women is changing along 11 dimensions. Validations of WHGI are presented against three exogenous datasets: the world's historical population, "traditional" gender-disparity indices (GDI, GEI, GGGI and SIGI), and occupational gender according to the US Bureau of Labor Statistics. Furthermore, to demonstrate its general use in research, we revisit previously published findings on Wikipedia's gender bias that can be strengthened by WHGI.

Online question and answer (Q&A) sites, which are platforms for users to post and answer questions on a wide range of topics, are becoming large repositories of valuable knowledge and important to societies. In order to sustain success, Q&A sites face the challenges of ensuring content quality and encouraging user contributions. This paper examines a particular design decision in Q&A sites-allowing Wikipedia-like collaborative editing on questions and answers, and explores its beneficial effects on content quality and potential detrimental effects on users' contributions. By examining five years' archival data of Stack Overflow, we found that the benefits of collaborative editing outweigh its risks. For example, each substantive edit from other users can increase the number of positive votes by 181% for the questions and 119% for the answers. On the other hand, each edit only decreases askers and answerers' subsequent contributions by no more than 5%. This work has implications for understanding and designing large-scale social computing systems.

Online communities are successful only if they achieve their goals, but there has been little direct study of goals. We analyze novel data characterizing the goals of enterprise online communities, assessing the importance of goals for leaders, how goals influence member perceptions of community value, and how goals relate to success measures proposed in the literature. We find that most communities have multiple goals and common goals are learning, reuse of resources, collaboration, networking, influencing change, and innovation. Leaders and members agree that all of these goals are important, but their perceptions of success on goals do not align with each other, or with commonly used behavioral success measures. We conclude that simple behavioral measures and leader perceptions are not good success metrics, and propose alternatives based on specific goals members and leaders judge most important.

Online communities serve various important functions, but many fail to thrive. Research on community success has traditionally focused on internal factors. In contrast, we take an ecological view to understand how the success of a community is influenced by other communities. We measured a community's relationship with other communities - its "niche" - through four dimensions: topic overlap, shared members, content linking, and shared offline organizational affiliation. We used a mixed-method approach, combining the quantitative analysis of 9495 online enterprise communities and interviews with community members. Our results show that too little or too much overlap in topic with other communities causes a community's activity to suffer. We also show that this main result is moderated in predictable ways by whether the community shares members with, links to content in, or shares an organizational affiliation with other communities. These findings provide new insight on community success, guiding online community designers on how to effectively position their community in relation to others.

If the people belong to multiple online communities, their joint membership can influence the survival of each of the communities to which they belong. Communities with many joint memberships may struggle to get enough of their members' time and attention, but find it easy to import best practices from other communities. In this paper, we study the effects of membership overlap on the survival of online communities. By analyzing the historical data of 5673 Wikia communities, we find that higher levels of membership overlap are positively associated with higher survival rates of online communities. Furthermore, we find that it is beneficial for young communities to have shared members who play a central role in other mature communities. Our contributions are two-fold. Theoretically, by examining the impact of membership overlap on the survival of online communities we identified an important mechanism underlying the success of online communities. Practically, our findings may guide community creators on how to effectively manage their members, and tool designers on how to support this task.

In modern crowdsourcing markets, requesters face the challenge of training and managing large transient workforces. Requesters can hire peer workers to review others' work, but the value may be marginal, especially if the reviewers lack requisite knowledge. Our research explores if and how workers learn and improve their performance in a task domain by serving as peer reviewers. Further, we investigate whether peer reviewing may be more effective in teams where the reviewers can reach consensus through discussion. An online between-subjects experiment compares the trade-offs of reviewing versus producing work using three different organization strategies: working individually, working as an interactive team, and aggregating individuals into nominal groups. The results show that workers who review others' work perform better on subsequent tasks than workers who just produce. We also find that interactive reviewer teams outperform individual reviewers on all quality measures. However, aggregating individual reviewers into nominal groups produces better quality assessments than interactive teams, except in task domains where discussion helps overcome individual misconceptions.

One of the most significant challenges for many online communities is increasing members' contributions over time. Prior studies on peer feedback in online communities have suggested its impact on contribution, but have been limited by their correlational nature. In this paper, we conducted a field experiment on Wikipedia to test the effects of different feedback types (positive feedback, negative feedback, directive feedback, and social feedback) on members' contribution. Our results characterize the effects of different feedback types, and suggest trade-offs in the effects of feedback between the focal task and general motivation, as well as differences in how newcomers and experienced editors respond to peer feedback. This research provides insights into the mechanisms underlying peer feedback in online communities and practical guidance to design more effective peer feedback systems.

We designed and ran an experiment to measure social influence in online recommender systems, specifically how often people's choices are changed by others' recommendations when facing different levels of confirmation and conformity pressures. In our experiment participants were first asked to provide their preferences between pairs of items. They were then asked to make second choices about the same pairs with knowledge of others' preferences. Our results show that others people's opinions significantly sway people's own choices. The influence is stronger when people are required to make their second decision sometime later (22.4%) than immediately (14.1%). Moreover, people seem to be most likely to reverse their choices when facing a moderate, as opposed to large, number of opposing opinions. Finally, the time people spend making the first decision significantly predicts whether they will reverse their decisions later on, while demographics such as age and gender do not. These results have implications for consumer behavior research as well as online marketing strategies.

We report on a study of the English edition of Wikipedia in which we used a mixed methods approach to understand how nested organizational structures called WikiProjects support collaboration. We first conducted two rounds of interviews with a total of 20 Wikipedians to understand how WikiProjects function and what it's like to participate in them from the perspective of Wikipedia editors. We then used a quantitative approach to further explore interpretations that arose from the qualitative data. Our analysis of these data together demonstrates how WikiProjects not only help Wikipedians coordinate tasks and produce articles, but also support community members and small groups of editors in important ways such as: providing a place to find collaborators, socialize and network; protecting editors' work; and structuring opportunities to contribute.

Traditional research on leadership in online communities has consistently focused on the small set of people occupying leadership roles. In this paper, we use a model of shared leadership, which posits that leadership behaviors come from members at all levels, not simply from people in high-level leadership positions. Although every member can exhibit some leadership behavior, different types of leadership behavior performed by different types of leaders may not be equally effective. This paper investigates how distinct types of leadership behaviors (transactional, aversive, directive and person-focused) and the legitimacy of the people who deliver them (people in formal leadership positions or not) influence the contributions that other participants make in the context of Wikipedia. After using propensity score matching to control for potential pre-existing differences among those who were and were not targets of leadership behaviors, we found that 1) leadership behaviors performed by members at all levels significantly influenced other members' motivation; 2) transactional leadership and person-focused leadership were effective in motivating others to contribute more, whereas aversive leadership decreased other contributors' motivations; and 3) legitimate leaders were in general more influential than regular peer leaders. We discuss the theoretical and practical implication of our work.

A challenge for many online production communities is to direct their members to accomplish tasks that are important to the group, even when these tasks may not match individual members' interests. Here we investigate how combining group identification and direction setting can motivate volunteers in online communities to accomplish tasks important to the success of the group as a whole. We hypothesize that group identity, the perception of belonging to a group, triggers in-group favoritism; and direction setting (including explicit direction from group goals and implicit direction from role models) focuses people's group-oriented motivation towards the group's important tasks. We tested our hypotheses in the context of Wikipedia's Collaborations of the Week (COTW), a group goal setting mechanism and a social event within Wikiprojects. Results demonstrate that 1) publicizing important group goals via COTW can have a strong motivating influence on editors who have voluntarily identified themselves as group members compared to those who have not self-identified; 2) the effects of goals spill over to non-goal related tasks; and 3) editors exposed to group role models in COTW are more likely to perform similarly to the models on group-relevant citizenship behaviors. Finally, we discuss design and managerial implications based on our findings.

In this paper, we introduce a method to measure shared leadership in Wikipedia as a step in developing a new model of online leadership. We show that editors with varying degrees of engagement and from peripheral as well as central roles all act like leaders, but that core and peripheral editors show different profiles of leadership behavior. Specifically, we developed machine learning models to automatically identify four types of leadership behaviors from 4 million messages sent between Wikipedia editors. We found strong evidence of shared leadership in Wikipedia, with editors in peripheral roles producing a large proportion of leadership behaviors.