VLDB 2026 Research / reviewers in the wild / expert
Elizabeth Gerber
dblp:34/4583 · also Elizabeth M. Gerber
· DBLP profile ↗
48ranked-venue papers
6as first author
7since 2021 · last 2025
0000-0001-8914-071XORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 48 · 6 first-author · 7 since 2021Databases, data management, data science and information retrieval · 4Applied, interdisciplinary, general and emerging computing · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Pressure to use AI for college admissions: implications for adolescent self-concept and intelligent coaching design
Aidan Z. Fitzsimons, Elizabeth Gerber, Duri Long |
Creativity & Cognition | 2 |
| 2025 | What Remotely Matters? Understanding Individual, Team, and Organizational Factors in Remote Work at ScaleabstractAlthough knowledge workers are increasingly able to adopt remote and hybrid working arrangements and work productively, many organizations continue to question the effectiveness of remote work and focus on its concerns and challenges. Previous CSCW research shows that remote workers have limited awareness of other workers, require more explicit coordination, and feel excluded from in-person colleagues. Research also shows that adopting work practices and technologies that are remote work-friendly can offset many of these challenges. Identifying which effective practices and challenges are most helpful or hurtful to remote workers-and how workplace attributes (e.g., team structure; communication frequency; tool use) affect them-could strengthen organizations' strategies and policies for remote work. Through a theoretically-informed survey of 1,526 U.S. knowledge workers, we find many factors prior research has argued as essential to remote work, such as knowing your teammates personally, to be the least important for remote workers, and show how workplace attributes influence those perceptions. We provide theoretical and practical implications for future research for organizations that wish to support remote and hybrid work modalities. Kapil Garg, Diego Gómez-Zará, Elizabeth Gerber, Darren Gergle, Noshir S. Contractor, Michael Massimi |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2025 | AI That Helps Us Help Each Other: A Proactive System for Scaffolding Mentor-Novice Collaboration in Entrepreneurship CoachingabstractEntrepreneurship requires navigating open-ended, ill-defined problems: identifying risks, challenging assumptions, and making strategic decisions under deep uncertainty. Novice founders often struggle with these metacognitive demands, while mentors face limited time and visibility to provide tailored support. We present a human-AI coaching system that combines a domain-specific cognitive model of entrepreneurial risk with a large language model (LLM) to proactively scaffold both novice and mentor thinking. The system proactively poses diagnostic questions that challenge novices' thinking and helps both novices and mentors plan for more focused and emotionally attuned meetings. Critically, mentors can inspect and modify the underlying cognitive model, shaping the logic of the system to reflect their evolving needs. Through an exploratory field deployment, we found that using the system supported novice metacognition, reduced mentors' cognitive load, and improved meeting depth, intentionality, and focus--while also surfaced key tensions around trust, misdiagnosis, and expectations of AI. We contribute design principles for proactive AI systems that scaffold metacognition and human-human collaboration in complex, ill-defined domains, offering implications for similar domains like healthcare, education, and knowledge work. Evey Jiaxin Huang, Matthew W. Easterday, Elizabeth Gerber |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2023 | On Hackathons: A Multidisciplinary Literature ReviewabstractThe number of hackathon events worldwide has nearly quadrupled in the last five years. Despite exponential growth across diverse industries and increasing interest across academic disciplines, our integrated understanding of the phenomena of hackathons is limited. We conduct the first multidisciplinary literature review of publications from 1999 to 2022 to understand the conceptualization of the phenomena over time. We find that hackathon research can be categorized into 4 core areas (purpose, format, processes, and outcomes). Research was first driven by a purpose (innovation, learning, and collaboration), followed by an examination of how formats adjust to purpose to influence what happens (processes) and what is produced (outcomes), and critical reviews of the hackathon phenomena. We contribute a unifying framework with these four core areas to inform future directions of hackathon research and practice, as well as a discussion of the need for longitudinal and multidisciplinary research of hackathons. Connie W. Chau, Elizabeth Gerber |
CHI | 2 |
| 2023 | Intelligent Coaching Systems: Understanding One-to-many Coaching for Ill-defined Problem SolvingabstractOne-to-many coaching is a common, yet difficult, coaching technique used in environments with many novices learning to solve ill-defined problems. Intelligent systems might be designed to support 1-to-many coaching but designing such systems requires a 1-to-many coaching model that details novices' challenges, coaches' strategies, and coaches' goals. To build such a model, we conducted interaction analysis on 24 1-to-many coaching sessions with novices developing new products in a university incubator and conducted retrospective analyses with 3 coaches and 30 novices. We contribute a model that demonstrates that coaches in a 1-to-many setting not only need to help novices develop metacognitive skills (just as in 1-to-1 coaching), but also need to utilize the presence and expertise of a group of novices to learn from each other, to mitigate their fear of failures, and provide them accountability. Our model informs design implications for future intelligent coaching systems to (1) assist coaches in monitoring and comparing many novices' progress, learning, and expertise; (2) provide novices with checklists, templates, and scaffolds to help them self-evaluate, seek-help, and summarize learning; (3) showcase failures and growth; and (4) publicize planning and progress to provide accountability. Evey Jiaxin Huang, Daniel Rees Lewis, Shubhanshi Gaudani, Matthew W. Easterday, Elizabeth Gerber |
Proc. ACM Hum. Comput. Interact. | 5 |
| 2021 | Understanding Gender Differences in Pricing Strategies in Online Labor MarketplacesabstractThe growing online gig economy provides ways for women to participate in a flexible, remote workforce and close the offline gender pay and participation gap. While women in online labor marketplaces earn about as much overall as men, women set lower bill rates suggesting gender differences in pricing strategies. In this study, we surveyed 392 freelancers in the United States (US) on the popular marketplace platform, Upwork, to understand strategies used to set hourly bill rates. We did not find gender differences in pricing strategies that were significantly related to bill rate. Instead, we found that other factors, such as full-time freelancer status and level of self-esteem, may help explain gender differences in bill rates. To better support equity and fairness in the growing gig economy, CHI researchers must identify, assess, and address the complex interaction between societal conditions in online labor markets. Eureka Foong, Elizabeth Gerber |
CHI | 2 |
| 2021 | CrowdFolio: Understanding How Holistic and Decomposed Workflows Influence Feedback on Online PortfoliosabstractFreelancers increasingly earn their livelihood through online marketplaces. To attract new clients, freelancers continuously curate their online portfolios to convey their unique skills and style. However, many lack access to rapid, regular, and inexpensive feedback needed to improve their portfolios. Existing crowd feedback systems, which collect feedback on individual creative projects (i.e., decomposed approach), could fill this need, but it is unclear how they might support feedback on multiple projects (i.e., holistic approach). In a between-subjects study with 30 freelancers, we compared decomposed and holistic feedback collection approaches using CrowdFolio, a crowd feedback system for portfolios. The holistic approach helped freelancers discover new ways to describe their work, while the decomposed approach provided detailed insight about the visual attractiveness of projects. This study contributes evidence that portfolio feedback systems, regardless of collection approach, can positively support professional development by impacting how freelancers portray themselves online and reflect on their identity. Eureka Foong, Joy Kim, Mira Dontcheva, Elizabeth Gerber |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2020 | Next Steps for Human-Computer IntegrationabstractHuman-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration. Florian 'Floyd' Mueller, Pedro Lopes 0001, Paul Strohmeier, Wendy Ju, Caitlyn E. Seim, Martin Weigel 0001, Suranga Nanayakkara, Marianna Obrist, Zhuying Li 0001, Joseph La Delfa, Jun Nishida, Elizabeth Gerber, Dag Svanæs, Jonathan Grudin, Stefan Greuter, Kai Kunze, Thomas Erickson, Steven Greenspan, Masahiko Inami, Joe Marshall, Harald Reiterer, Katrin Wolf 0001, Jochen Meyer 0001, Thecla Schiphorst, Dakuo Wang, Pattie Maes |
CHI | 12 |
| 2019 | How We Make ImpactabstractNobel laureate, Herb Simon described design as changing existing situations into preferred situations. But Simon did not specify a medium by which we must design. As a technologist tackling challenges, I often over-rely on technology as my medium and limit my potential for impact. We can design technology, but we can also design services, processes, and organizations and much more. By more broadly understanding the media by which we can design, we can more systematically build the future in which we want to live. Elizabeth Gerber |
UIST | 1 |
| 2019 | Distributed Apprenticeship in Online CommunitiesabstractSupporting learning in online communities is an important direction for the future of human–computer interaction as people increasingly leverage social technologies to support professional growth and development. However, few have studied how people leverage the socio-technical affordances of online informal workplace communities to develop professional skills in the absence of dedicated expert guidance. We draw from theories of apprenticeship to introduce an emergent theory of distributed apprenticeship, which outlines how community expectations of transparency and mutual support allow for instruction to be directed by a distributed network of nonexperts. We develop distributed apprenticeship through a qualitative study of crowdfunding entrepreneurs, where novices leverage social interactions with community members to develop a wide range of entrepreneurial skills. We then generalize distributed apprenticeship to other workplace contexts and provide design implications for online communities where people develop professional skills with minimal dedicated formal guidance. Julie Hui, Matthew W. Easterday, Elizabeth Gerber |
Hum. Comput. Interact. | 3 |
| 2018 | IntroAssist: A Tool to Support Writing Introductory Help RequestsabstractWriting introductory help requests is a key part of develop-ing new professional connections, such as through email and other online messaging systems. This paper presents the design and an experimental evaluation of IntroAssist-a web-based tool that leverages cognitive apprenticeship in-structional methods to support writing introductory help requests through an expert-informed checklist, tagged peer examples, self-tagging, and suggested word limit. In a study of IntroAssist with novice entrepreneurs, we find that 1) expert raters consider help requests written with the tool as more effective, 2) participants are able to perform introduc-tory help seeking skills after the tool is removed, and 3) participants report being more likely to send help requests written with the tool. We present implications for the de-velopment of systems that support the initiation of profes-sional relationships. Julie Hui, Darren Gergle, Elizabeth Gerber |
CHI | 3 |
| 2018 | Women (Still) Ask For Less: Gender Differences in Hourly Rate in an Online Labor MarketplaceabstractIn many traditional labor markets, women earn less on average compared to men. However, it is unclear whether this discrepancy persists in the online gig economy, which bears important differences from the traditional labor market (e.g., more flexible work arrangements, shorter-term engagements, reputation systems). In this study, we collected self-determined hourly bill rates from the public profiles of 48,019 workers in the United States (48.8% women) on Upwork, a popular gig work platform. The median female worker set hourly bill rates that were 74% of the median man's hourly bill rates, a gap than cannot be entirely explained by online and offline work experience, education level, and job category. However, in some job categories, we found evidence of a more complex relationship between gender and earnings: women earned more overall than men by working more hours, outpacing the effect of lower hourly bill rates. To better support equality in the rapidly growing gig economy, we encourage continual evaluation of the complex gender dynamics on these platforms and discuss whose responsibility it is to address inequalities. Eureka Foong, Nicholas Vincent, Brent J. Hecht, Elizabeth Gerber |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2018 | CheerOn: Facilitating Online Social Support for Novice Project-Based Learning TeamsabstractNovices learn innovation best through project-based learning (PBL), working in face-to-face teams to tackle real-world problems. Yet, real-world projects are complex, stressful, and especially challenging for novices. Online communities could provide social support to motivate novices, but it is unclear how to design online communities to support face-to-face PBL teams. Here we ask:How might we design an online system that enlists external supporters to provide online social support to motivate PBL students?Our need-finding study found that PBL students received infrequent social support, rarely engaged in help-seeking, and perceived little progress until the end of their projects. Based on these findings, we designedCheerOn, an online social support system that prompts novice student teams to externalize progress allowing external, online supporters to offer social support. We testedCheerOnwith 3 PBL teams and 15 external supporters over a 6-week course. We found that external supporters provided instrumental, informational, and emotional support that strengthened students’ bonds to the community, which increased help-seeking. Supporters also provided appraisal support, which increased students’ perceived value of their work. Supporters were more likely to offer informational and instrumental support when they were promoted or saw a clear need for help; supporters who received gratitude from students were more likely to offer emotional support in return; and supporters who were closely connected to the community were more likely to offer appraisal and instrumental support. Theoretically, this research contributes to our understanding of how hybrid face-to-face and online communities can impact the behavior of PBL students, specifically towards the facilitation of help-seeking behavior, as well as increased understanding of how different types of social support (i.e., appraisal, emotional, informational, and instrumental) can impact the participation of PBL students and supporters. Practically, this research contributes to our understanding of how to design socio-technical systems that facilitate social support for offline novice PBL students working, expanding the instructional resources available for preparing novices in PBL environments. Emily Harburg, Daniel Rees Lewis, Matthew W. Easterday, Elizabeth Gerber |
ACM Trans. Comput. Hum. Interact. | 4 |
| 2017 | Listen to Others, Listen to Yourself: Combining Feedback Review and Reflection to Improve Iterative DesignabstractFeedback from diverse audiences can contain ambiguity and contradictions, making it difficult to interpret and act on. To promote deeper interpretation of feedback, we tested the effects of combining a reflection activity and reviewing external feedback for an iterative design task. Designers (N=90) created a design and revised it after a) performing a reflection activity before reviewing feedback, b) performing the reflection after reviewing feedback, c) performing the reflection only, or d) reviewing the feedback only. We measured design quality, depth of revision, perceived effort, and confidence; and categorized the content produced from the reflections. We found that performing reflection after feedback review led to the largest increase in perceived quality for the revised designs, and performing reflection and feedback review regardless of the order resulted in the most extensive revision. Our results also showed that performing the reflection alone yielded outcomes that were similar to when only reviewing feedback, and either activity led to better outcomes than the control condition (no feedback or reflection). Designers stated that the reflection helped them recall their goals, question their choices, and prioritize revisions. We argue that designers should perform a lightweight, explicit reflection to enhance their iterative process, and discuss implications for feedback platforms. Yu-Chun (Grace) Yen, Steven Dow, Elizabeth Gerber, Brian P. Bailey |
Creativity & Cognition | 3 |
| 2017 | Online Feedback Exchange: A Framework for Understanding the Socio-Psychological FactorsabstractTo meet the demand for authentic, timely, and affordable feedback, researchers have explored technologies to connect designers with feedback providers online. While researchers have implemented mechanisms to improve the content of feedback, most systems for online feedback exchange do not support an end-to-end cycle, from help-seeking to sense-making to action. Building on extant literature in learning sciences, design, organizational behavior, and online communities, we propose a conceptual framework to highlight critical processes that affect online feedback exchange. We contribute research questions for future feedback systems and argue that online feedback systems must be able to support designers through five activities that happen before, during, and after the feedback exchange. Our framework suggests that systems should address broader socio-psychological factors, such as how intent should be communicated online, how dialogue can support the interpretation of feedback, and how to balance the tradeoffs of anonymizing feedback providers. Eureka Foong, Steven Dow, Brian P. Bailey, Elizabeth Gerber |
CHI | 4 |
| 2017 | Critique Style Guide: Improving Crowdsourced Design Feedback with a Natural Language ModelabstractDesigners are increasingly leveraging online crowds; yet, online contributors may lack the expertise, context, and sensitivity to provide effective critique. Rubrics help feedback providers but require domain experts to write them and may not generalize across design domains. This paper introduces and tests a novel semi-automated method to support feedback providers by analyzing feedback language. In our first study, 52 students from two design courses created design solutions and received feedback from 176 online providers. Instructors, students, and crowd contributors rated the helpfulness of each feedback response. From this data, an algorithm extracted a set of natural language features (e.g., specificity, sentiment etc.) that correlated with the ratings. The features accurately predicted the ratings and remained stable across different raters and design solutions. Based on these features, we produced a critique style guide with feedback examples - automatically selected for each feature - to help providers revise their feedback through self-assessment. In a second study, we tested the validity of the guide through a between-subjects experiment (n=50). Providers wrote feedback on design solutions with or without the guide. Providers generated feedback with higher perceived helpfulness when using our style-based guidance. Markus Krause, Thomas Garncarz, Jiaojiao Song, Elizabeth Gerber, Brian P. Bailey, Steven Dow |
CHI | 4 |
| 2017 | Reappropriating Hackathons: The Production Work of the CHI4Good Day of ServiceabstractThe popularity of hackathons has increased as technology pervades more facets of our lives. Originally designed for programmers, hackathons are now being appropriated by new stakeholders across diverse sectors. Yet with this evolution in hackathons, we no longer adequately understand what is produced and, thereby, the value of these events. We conducted an interview study with 22 stakeholders - participants, representatives of nonprofit organizations, and organizers - of the CHI4Good Day of Service to understand what is produced through philanthropic hackathons. Whereas traditional hackathons are oriented around the production of code or prototypes, our analysis of interview data suggests that the production work of philanthropic hackathons also includes technical capacity and expertise, expanded social networks, an exposure to design process, affective experiences, and an opportunity for participants to shape their identities against a cross-sectoral, interdisciplinary backdrop. We conclude by reflecting on implications for the CHI community in carrying out philanthropic events styled after hackathons. Emily Porter, Chris Bopp, Elizabeth Gerber, Amy Voida |
CHI | 3 |
| 2017 | From in the Class or in the Wild?: Peers Provide Better Design Feedback Than External CrowdsabstractAs demand for design education increases, instructors are struggling to provide timely, personalized feedback for student projects. Gathering feedback from classroom peers and external crowds offer scalable approaches, but there is little evidence of how they compare. We report on a study in which students (n=127) created early- and late-stage prototypes as part of nine-week projects. At each stage, students received feedback from peers and external crowds: their own social networks, online communities, and a task market. We measured the quality, quantity and valence of the feedback and the actions taken on it, and categorized its content using a taxonomy of critique discourse. The study found that peers produced feedback that was of higher perceived quality, acted upon more, and longer compared to the crowds. However, crowd feedback was found to be a viable supplement to peer feedback and students preferred it for projects targeting specialized audiences. Feedback from all sources spanned only a subset of the critique categories. Instructors may fill this gap by further scaffolding feedback generation. The study contributes insights for how to best utilize different feedback sources in project-based courses. Helen Wauck, Yu-Chun (Grace) Yen, Wai-Tat Fu, Elizabeth Gerber, Steven Dow, Brian P. Bailey |
CHI | 4 |
| 2017 | Developing Makerspaces as Sites of EntrepreneurshipabstractMakerspaces are providing new opportunities for entrepre-neurial development. Based on participant observation and a series of in-depth interviews in an entrepreneurial mak-erspace, we found that the socio-technical environment of a makerspace promotes entrepreneurship by leveraging community-based values of social support, transparency, exploration, and empowerment. The openness of a shared space, access to social technologies, and community of social support helped members develop entrepreneurial skills and self-efficacy. We add to CSCW literature on expanding the role of makerspaces as places of career development through entrepreneurship Julie Hui, Elizabeth Gerber |
CSCW | 2 |
| 2017 | Understanding Trust amid Delays in CrowdfundingabstractTrust is essential for beginning and maintaining relationships online where assessing uncertainties and risks is difficult. While product delays have been shown to reduce trust in e-commerce settings, we understand little about the effect of delays on trust in the increasingly popular context of crowdfunding. In a mixed method study, we examine what factors influence backers' trust in crowdfunding when rewards are delayed. Based on in-depth interviews with crowdfunding participants, we found that a rich set of factors influenced backers' trust including backers' role identity and domain knowledge, backer's research on a creator's background, creators' communication during delays, and duration of delays. To better understand the factors affecting delays, we conducted a regression analysis with 4,089 delayed projects and found that the funding goal, number of backers, percent raised, number of reward levels, and creator's previous crowdfunding experience are associated with the duration of delay. We discuss design implications for managing delays and maintaining trust in crowdfunding. Yongsung Kim, Aaron D. Shaw, Elizabeth Gerber |
CSCW | 4 |
| 2017 | Agile Research Studios: Orchestrating Communities of Practice to Advance Research TrainingabstractUndergraduate research experiences enhance learning and professional development, but providing effective and scalable research training is often limited by practical implementation and orchestration challenges. This paper introduces Agile Research Studios (ARS)--a socio-technical system that expands research training opportunities by supporting research communities of practice without increasing faculty mentoring resources. ARS integrates and advances professional best practices and organizational designs, principles for forming effective learning communities, and design of social technologies to overcome the orchestration challenge of one faculty researcher mentoring 20 or more students. We present the results of a two-year pilot of the Design, Technology, and Research (DTR) program, which used the ARS model to improve the quality of learning, produce research outcomes, and lower the barrier to participation while increasing the number of students who receive authentic research training. Matthew W. Easterday, Elizabeth Gerber, Daniel Rees Lewis, Leesha Maliakal |
CSCW | 3 |
| 2017 | Novice and Expert Sensemaking of Crowdsourced Design FeedbackabstractOnline feedback exchange (OFE) systems are an increasingly popular way to test concepts with millions of target users before going to market. Yet, we know little about how designers make sense of this abundant feedback. This empirical study investigates how expert and novice designers make sense of feedback in OFE systems. We observed that when feedback conflicted with frames originating from the participant's design knowledge, experts were more likely than novices to question the inconsistency, seeking critical information to expand their understanding of the design goals. Our results suggest that in order for OFE systems to be truly effective, they must be able to support nuances in sensemaking activities of novice and expert users. Eureka Foong, Darren Gergle, Elizabeth Gerber |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2016 | Social Network, Web Forum, or Task Market?: Comparing Different Crowd Genres for Design Feedback ExchangeabstractIncreasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback. Yu-Chun (Grace) Yen, Steven Dow, Elizabeth Gerber, Brian P. Bailey |
Conference on Designing Interactive Systems | 3 |
| 2016 | Studying the Effects of Task Notification Policies on Participation and Outcomes in On-the-go CrowdsourcingabstractRecent years have seen the growth of physical crowdsourcing systems (e.g., Uber; TaskRabbit) that motivate large numbers of people to provide new and improved physical tasking and delivery services on-demand. In these systems, opportunistically relying on people to make convenient contributions may lead to incomplete solutions, while directing people to do inconvenient tasks requires high incentives. To increase people's willingness to participate and reduce the need to incentivize participation, we study on-the-go crowdsourcing as an alternative approach that suggests tasks along people’s existing routes that are conveniently on their way. We explore as a first step in this paper the design of task notification policies that decide when, where, and to whom to suggest tasks. Situating our work in the context of practical problems such as package delivery and lost-and-found searches, we conducted controlled experiments that show how small changes in task notification policy can influence individual participation and actions in significant ways that in turn affect system outcomes. We discuss the implications of our findings on the design of future on-the-go crowdsourcing technologies and applications. Yongsung Kim, Emily Harburg, Shana Azria, Aaron D. Shaw, Elizabeth Gerber, Darren Gergle |
HCOMP | 5 |
| 2015 | Critiki: A Scaffolded Approach to Gathering Design Feedback from Paid CrowdworkersabstractFeedback is important to the creative process, but not everyone has a personal crowd of individuals they can turn to for high-quality feedback. We introduce and evaluate Critiki, a novel system for gathering design critiques on crowdfunding project pages from paid crowdworkers. Stemming from previous research on crowdfunding project creators and their need for early-stage design feedback, we design and build a working system which fits the need of this population: rapid and inexpensive feedback. To solve issues with critique quality we describe a scaffolding technique designed to assist crowdworkers in writing high-quality critiques. We evaluate Critiki with two field deployments: 1) A randomized controlled experiment with 450 crowdworkers to evaluate the efficacy of the scaffolding technique and 2) A user study with 31 crowdfunding project creators to determine usability and user satisfaction. We contribute to research on Creativity and Cognition by demonstrating a working creativity support system, empirically evaluating the system, and describing how scaffolding approaches can be designed for other crowdsourced tasks Michael D. Greenberg, Matthew W. Easterday, Elizabeth Gerber |
Creativity & Cognition | 3 |
| 2015 | Building Support Tools to Connect Novice Designers with Professional CoachesabstractCreativity support tools help learners undertake creative work, such as facilitating coaching by creative professionals. How might we design creativity support tools that in-crease learners' access to coaching by creative professionals? This study took place in an extracurricular project-based learning program where students were co-located, and met professional coaches face-to-face once a week but otherwise communicated online. To test an online creativity support tool called the Loft and investigate coach-student communication we collected data from 47 interviews, online log data and field observations. We found that (a) explicit help-seeking was rare outside of meetings, (b) help from professionals was highly-valued but not sought out, and (c) online systems could surface learner struggles and trigger help-giving. Our findings suggested that online creativity platforms can support professional coaching through: (1) structured virtual updates (2) coach thanking, (3) Computer-Supported Group Critique, (4) disclosure of expertise, and (5) help-seeking training. Daniel Rees Lewis, Emily Harburg, Elizabeth Gerber, Matthew W. Easterday |
Creativity & Cognition | 3 |
| 2015 | Understanding the Effects of Crowdfunding on Entrepreneurial Self-EfficacyabstractCrowdfunding is emerging as a new socio-technical system that is changing how entrepreneurs interact with their community of financial supporters. While computer supported cooperative work (CSCW) researchers have begun to explore how this new type of system influences entrepreneurial work, less is understood about how the system influences their psychological experience - specifically self-efficacy, or belief in one's own ability to succeed at a task, which affects persistence, learning, and performance. Drawing on Bandura's Social Cognitive Theory, we conducted a qualitative study of 53 entrepreneurs using crowdfunding and found that crowdfunding can influence self-efficacy through (1) social validation, (2) role modeling, (3) mastery, and (4) physiological states supported by socio-technical features, such as displaying a concrete goal, examples of other's work, or public feedback. Results suggest how socio-technical systems can be designed to help entrepreneurs perform to enhance our economic and social prosperity. Emily Harburg, Julie Hui, Michael D. Greenberg, Elizabeth Gerber |
CSCW | 4 |
| 2015 | Crowdfunding Science: Sharing Research with an Extended AudienceabstractCrowdfunding is changing how, why, and which research projects are pursued. With the increasing number of crowdfunded research projects, it is important to understand what drives scientists to launch crowdfunding campaigns and how it affects their work. To better understand this re-cent phenomenon, we present a grounded theory of how and why scientists crowdfund. Through 27 semi-structured interviews, we find that scientists are motivated to crowd-fund in order to share their work and engage the public in the research process in ways traditional science work has not offered. Scientists also perceive crowdfunding as a more accessible way to get funds quickly compared to existing fundraising mechanisms, such as grant applications. However, they must learn to use more accessible language to successfully communicate their research through social media to a broad audience of non-scientists and professional peers. Based on these findings, we discuss design implications to inform future crowdfunding platforms and sup-port tools. Julie Hui, Elizabeth Gerber |
CSCW | 2 |
| 2015 | Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced FeedbackabstractStudent entrepreneurs struggle to collect feedback on their product pitches in a classroom setting due to a lack of time, money, and access to motivated feedback providers. Online social networks present a unique opportunity for entrepreneurial students to quickly access feedback providers by leveraging their online social capital. In order to better understand how to improve crowdsourced online pitch feedback, we perform an experiment to test the effect of online anonymity on pitch feedback quality and quantity. We also test a communal feedback method—evenly distributing between teams feedback providers from the class’s collective online social networks—which would help all teams benefit from a useful amount of feedback rather than having some teams receive much more feedback than others. We found that feedback providers in the anonymous condition provided significantly more specific criticism and specific praise, which students rated as more useful. Furthermore, we found that the communal feedback method helped all teams receive sufficient feedback to edit their pitches. This research contributes an empirical investigation to the crowdsourcing community of how crowds through online social networks can help student entrepreneurs obtain authentic feedback to improve their work. Julie Hui, Amos Glenn, Rachel Jue, Elizabeth Gerber, Steven Dow |
HCOMP | 4 |
| 2015 | Enabling Physical Crowdsourcing On-the-Go with Context-Sensitive NotificationsabstractThis paper introduces the idea of presenting physical tasks into people’s existing routine so that people can contribute small tasks on-the-go with minimal effort. We present two on-the-go crowdsourcing systems: Libero for package delivery, and CrowdFound for finding lost items. To encourage contributions, we introduce notification techniques that present task opportunities when potential helpers are likely to accept. To direct people to regions where help is most needed, we introduce techniques for tracking a person’s location within a task region and directing their attention based on task history. Evaluation studies demonstrate the feasibility of on-the-go crowdsourcing and investigate questions over the likelihood of task completion, the perceived cost of disruption, and the effectiveness of tracking and coordination. Yongsung Kim, Emily Harburg, Shana Azria, Elizabeth Gerber, Darren Gergle |
HCOMP | 4 |
| 2014 | Computer supported novice group critiqueabstractGroups of novice critiquers can sometimes provide feedback of the same quality as a single expert. Unfortunately, we do not know how to create systems for novice group critique in design education. We tested whether 4 principles: write-first scripts, critique prompts, interactive critiquing & formative framing, allow us to create systems that combine the advantages of face-to-face and computer-mediated critique. We collected observations and 48 interviews with 12 undergraduate design students who used a computer supported group critique system over 5 critique sessions, analyzed using grounded theory. We found that: (a) the write-first script helped overcome initial learning costs; (b) the interactive critique features created a dual-channel critique that increased the number of critiquers, duration of critique and interactivity; and (c) the system produced a greater volume of useful critique and promoted reciprocity among critiquers. The study provides improved principles for developing computer supported novice group critique systems in design. Matthew W. Easterday, Daniel Rees Lewis, Colin Fitzpatrick, Elizabeth Gerber |
Conference on Designing Interactive Systems | 4 |
| 2014 | Crowd-based design activities: helping students connect with users onlineabstractBy definition, human-centered design relies on interaction with users. While interacting with users within industry can be challenging, fostering these interactions in a classroom setting can be even more difficult. This qualitative study explores the use of crowd-based design activities as a way to support student-user interactions online. We motivate these online methods through a survey of 27 design instructors, who identified common challenges of conducting student-user interactions in physical settings, including coordination constraints and geographical barriers to meeting in person. We then describe our research through design to create and test 10 activities in a classroom setting, including using Twitter for needfinding and using Reddit to brainstorm ideas with experts. Finally, we present an emergent framework outlining the design space for crowd-based design activities where students learn to use input from online crowds to inform their design work. We discuss plans to refine and expand the current set of activities for open access to instructors. Julie Hui, Elizabeth Gerber, Steven Dow |
Conference on Designing Interactive Systems | 2 |
| 2014 | Understanding and leveraging social networks for crowdfunding: opportunities and challengesabstractCrowdfunding provides a new way for creatives to share their work and acquire resources from their social network to influence what new ideas are realized. Yet, we understand very little about this growing phenomenon. Grounded in existing work on social network analysis, we interview 58 crowdfunding project creators to investigate how crowdfunders use their social network to reach their campaign goals. We identified three main challenges, which include understanding network capabilities, activating network connections, and expanding network reach. From our findings, we develop initial design implications for support tools to help crowdfunding project creators better understand and leverage their social network. Julie Hui, Elizabeth Gerber, Darren Gergle |
Conference on Designing Interactive Systems | 2 |
| 2014 | Combining crowdsourcing and learning to improve engagement and performanceabstractCrowdsourcing complex creative tasks remains difficult, in part because these tasks require skilled workers. Most crowdsourcing platforms do not help workers acquire the skills necessary to accomplish complex creative tasks. In this paper, we describe a platform that combines learning and crowdsourcing to benefit both the workers and the requesters. Workers gain new skills through interactive step-by-step tutorials and test their knowledge by improving real-world images submitted by requesters. In a series of three deployments spanning two years, we varied the design of our platform to enhance the learning experience and improve the quality of the crowd work. We tested our approach in the context of LevelUp for Photoshop, which teaches people how to do basic photograph improvement tasks using Adobe Photoshop. We found that by using our system workers gained new skills and produced high-quality edits for requested images, even if they had little prior experience editing images. Mira Dontcheva, Robert R. Morris, Joel Brandt, Elizabeth Gerber |
CHI | 4 |
| 2014 | Learning to fail: experiencing public failure online through crowdfundingabstractOnline crowdfunding platforms like Kickstarter are gaining attention among novice creatives as an effective platform for funding their ventures and engaging in creative work with others. However, a focus on financial success of crowdfunding has obscured the fact that over 58% of crowdfunding projects fail to achieve their funding goals. This population of failed creatives however, gives us an audience to study public creative failure in an online environment. We draw inspiration from work in organizational behavior on failure, and work in Human Computer Interaction (HCI) on online behavior, to study online public failure. Using a mixed-methods approach with data scraped from Kickstarter and interview data with failed crowdfunding project creators, we answer the following question: What do project creators on crowdfunding platforms learn and change through the process of failing? We find that creators who relaunch their projects succeed 43% of the time, and that most individuals find failure to be a positive experience. We conclude the paper with a series of design implications for future creative platforms where public failure is part of the creative process. Michael D. Greenberg, Elizabeth Gerber |
CHI | 2 |
| 2014 | Understanding the role of community in crowdfunding workabstractCrowdfunding provides a new opportunity for entrepre-neurs to launch ventures without having to rely on traditional funding mechanisms, such as banks and angel investing. Despite its rapid growth, we understand little about how crowdfunding users build ad hoc online communities to undertake this new way of performing entrepreneurial work. To better understand this phenomenon, we performed a qualitative study of 47 entrepreneurs who use crowdfunding platforms to raise funds for their projects. We identify community efforts to support crowdfunding work, such as providing mentorship to novices, giving feedback on campaign presentation, and building a repository of example projects to serve as models. We also identify where community efforts and technologies succeed and fail at support-ing the work in order to inform the design of crowdfunding support tools and systems. Julie Hui, Michael D. Greenberg, Elizabeth Gerber |
CSCW | 3 |
| 2014 | Pair research: matching people for collaboration, learning, and productivityabstractTo increase productivity, informal learning, and collaborations within and across research groups, we have been experimenting with a new kind of interaction that we call {em pair research}, in which members are paired up weekly to work together on each other's projects. In this paper, we present a system for making pairings and present results from two deployments. Results show that members used pair research in a wide variety of ways including pair programming, user testing, brainstorming, and data collection and analysis. Pair research helped members get things done and share their expertise with others. Rob Miller 0001, Eric Gilbert, Elizabeth Gerber |
CSCW | 4 |
| 2014 | WeDo: End-To-End Computer Supported Collective Action
Andrés Monroy-Hernández, Aaron D. Shaw, Sean A. Munson, Elizabeth Gerber, Benjamin Mako Hill, Peter Kinnaird, Shelly Farnham, Patrick Minder |
ICWSM | 5 |
| 2013 | Affect and Creative Performance on Crowdsourcing PlatformsabstractPerformance on crowd sourcing platforms varies greatly, especially for tasks requiring significant cognitive effort or creative insight. Researchers have proposed several techniques to address these problems, yet few have considered the role of affect, despite the well-established link between positive affect and creative performance. In this paper, we examine two affective techniques to boost creativity on crowd sourcing platforms - affective priming and affective pre-screening. Across three experiments, we find divergent results, depending on which technique is used. We find that not all happy crowd workers are alike. Those that are primed to feel happy exhibit enhanced creative performance, whereas those that merely report feeling happy exhibit impaired creative performance. We examine these findings in light of preexisting research on creativity, affect, and mood saliency. Lastly, we show how our findings have implications not only for crowd sourcing platforms, but also for other human-computer interaction scenarios that involve affect and creative performance. Robert R. Morris, Mira Dontcheva, Adam Finkelstein, Elizabeth Gerber |
ACII | 4 |
| 2013 | A pilot study of using crowds in the classroomabstractIndustry relies on higher education to prepare students for careers in innovation. Fulfilling this obligation is especially difficult in classroom settings, which often lack authentic interaction with the outside world. Online crowdsourcing has the potential to change this. Our research explores if and how online crowds can support student learning in the classroom. We explore how scalable, diverse, immediate (and often ambiguous and conflicting) input from online crowds affects student learning and motivation for project-based innovation work. In a pilot study with three classrooms, we explore interactions with the crowd at four key stages of the innovation process: needfinding, ideating, testing, and pitching. Students reported that online crowds helped them quickly and inexpensively identify needs and uncover issues with early-stage prototypes, although they favored face-to-face interactions for more contextual feed-back. We share early evidence and discuss implications for creating a socio-technical infrastructure to more effectively use crowdsourcing in education. Steven Dow, Elizabeth Gerber, Audris Wong |
CHI | 2 |
| 2013 | The future of crowd workabstractPaid crowd work offers remarkable opportunities for improving productivity, social mobility, and the global economy by engaging a geographically distributed workforce to complete complex tasks on demand and at scale. But it is also possible that crowd work will fail to achieve its potential, focusing on assembly-line piecework. Can we foresee a future crowd workplace in which we would want our children to participate? This paper frames the major challenges that stand in the way of this goal. Drawing on theory from organizational behavior and distributed computing, as well as direct feedback from workers, we outline a framework that will enable crowd work that is complex, collaborative, and sustainable. The framework lays out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation. Aniket Kittur, Jeffrey V. Nickerson, Michael S. Bernstein, Elizabeth Gerber, Aaron D. Shaw, John Zimmerman, Matthew Lease, John Joseph Horton |
CSCW | 4 |
| 2013 | Crowdfunding: Motivations and deterrents for participationabstractCrowdfunding is changing how, why, and which ideas are brought into existence. With the increasing number of crowdfunded projects, it is important to understand what drives people to either create or fund these projects. To shed light on this new social phenomenon, we present a grounded theory of motivation informed by the first cross-platform qualitative study of the crowdfunding community. By performing 83 semistructured interviews, we uncover creator motivations, which include the desire to raise funds, expand awareness of work, connect with others, gain approval, maintain control, and learn; and supporter motivations, which include the desire to collect rewards, help others, support causes, and be part of a community. We also explore deterrents to crowdfunding participation, including, among creators, fear of failure, and, for supporters, lack of trust. Based on these findings, we provide three emergent design principles to inform the design of effective crowdfunding platforms and support tools. Elizabeth Gerber, Julie Hui |
ACM Trans. Comput. Hum. Interact. | 1 |
| 2012 | Work in progress: Developing an Innovation Self-efficacy surveyabstractInnovation is critical to our economic and social prosperity. We rely on industry, university, and government employees to develop, modify, and implement innovative ideas while navigating ambiguous problem contexts, overcoming setbacks, and persisting in competition with courses of action. Research has shown that self-efficacy, or an individual's belief in their ability, influences the pursuit of and persistence in challenging work. This suggests that self-efficacy is critical for innovation. Despite resource-intensive efforts to foster innovation in organizations, we inadequately understand how to measure the impact of these interventions on individuals' judgment of their own innovation ability. We present the initial development and validation of the first survey measure for Innovation Self-efficacy (ISE), or the belief in one's ability to innovate. Elizabeth Gerber, Caitlin Kennedy Martin, Elizabeth Kramer, Jennie Braunstein, Adam R. Carberry |
FIE | 1 |
| 2011 | Tech break up: a research method for understanding people's attachment to their technologyabstractTech Break Ups are an early stage research method that enables researchers to gain insights into ways in which people are attached to technology through improvised "break-ups." Informants verbally reflect on and formally end their relationships with technology that they previously used. The method was used in a study to understand peoples' relationship to technologies that enable creative work and reasons and moments for change in the relationship. This paper provides an overview of the method and its execution. The study reveals three primary factors of detachment for technologies that support creative work: changing self-identity, creative process, and creative ability. This case study suggests that the Tech Break Up Method can provide insights into product attachment to inform the design of new technology while simultaneously providing informants with an immediate positive experience through direct emotional expression to technology. Elizabeth Gerber |
Creativity & Cognition | 1 |
| 2011 | Affective computational priming and creativityabstractWhile studies have shown that affect influences creativity, few investigate how affect influences creative performance with creativity support tools. Drawing from methods commonly used in psychology research, we present affective computational priming, a new method for manipulating affect using digitally embedded stimuli. We present two studies that explore computational techniques for inducing positive, neutral, and negative affect and examine their impact on idea generation with creativity support tools. Our results suggest that positive affective computational priming positively influences the quality of ideas generated. We discuss opportunities for future HCI research and offer practical applications of affective computational priming. Sheena Lewis, Mira Dontcheva, Elizabeth Gerber |
CHI | 3 |
| 2010 | Momentum: getting and staying on topic during a brainstormabstractDespite the prevalent use of group brainstorming for problem solving and decision-making within organizations, brainstorming sessions often lack focus and fail to produce quality ideas. We describe Momentum, a tool that elicits topic-oriented responses prior to a group brainstorm. In an exploratory study, we found qualitative differences in task focus, quality and rate of ideation, and efficiency of storytelling between users and non-users of the tool. Patti Bao, Elizabeth Gerber, Darren Gergle, David Hoffman |
CHI | 2 |
| 2009 | Using improvisation to enhance the effectiveness of brainstormingabstractGroup brainstorming is a popular ideation method for design teams, yet brainstorming outcomes vary greatly. The method depends on individuals working collectively to generate ideas, and so group dynamics determine whether the method succeeds or fails. This paper explores how interaction designers used techniques from theatrical improvisation, or improv, to adhere to the rules of brainstorming thereby enhancing group interactions while collaborating. The usefulness of improvisation for brainstorming stems from the similarity of the goals of improvisation and brainstorming, the similarity of the recurrent problems that actors and designers encounter when collaborating, and the distinctness of the ways each have devised to resolve the problems that block the group's performance. This paper reflects on the individual- and group-level outcomes for design students and practitioners while brainstorming. Elizabeth Gerber |
CHI | 1 |
| 2007 | Improvisation principles and techniques for designabstractExisting research addresses how designers create tools to support improvisation, yet little research explores how improvisation offers tools to support design work. This paper explores the potential relationship between improvisation and design, examining how design can benefit from improvisation. The paper argues that improvisation can build perspectives and skills that are critical for designers, such as creative collaboration, fostering innovation, supporting spontaneity, learning through error, and presenting ideas. The paper reviews the use of improvisation activities by designers in a multi-case study. The applications are analyzed to demonstrate individual and group level outcomes in design work. Elizabeth Gerber |
CHI | 1 |