EDBT 2026 Demo / reviewers in the wild / expert
Karrie Karahalios
dblp:64/4500 · also Karrie G. Karahalios
· DBLP profile ↗
123ranked-venue papers
3as first author
32since 2021 · last 2026
0000-0001-8788-3405ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 96 · 3 first-author · 25 since 2021Databases, data management, data science and information retrieval · 17 · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 11 · 4 since 2021Applied, interdisciplinary, general and emerging computing · 8 · 1 since 2021Artificial intelligence and machine learning · 7 · 4 since 2021Systems, architecture and hardware · 1 · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Supporting Learners' Use of Imperfect Generative Pedagogical Chatbots: The Role of Chatbot Response Uncertainty and Reduced VerbosityabstractGenerative chatbots promise to scale personalized learning. Most publicly available generative chatbots are designed to provide confident and eloquent responses by default, even when hallucinating. Prior work has observed that learners using such chatbots often engage shallowly and fail to detect chatbot errors due to overtrust, cognitive overload, and prioritization of short-term gains. To address these challenges, this work examines two chatbot design options in a STEM learning context: introducing verbal uncertainty and reducing response verbosity. Using Bayesian causal inference and thematic analysis in a quasi-experimental setting, we found that a less verbose chatbot improved detection of errors with logical fallacies, but did not increase the use of alternative resources. A chatbot that always expressed uncertainty reduced the adoption of incorrect chatbot responses, but had mixed effects on learning outcomes, suggesting the need to increase signal credibility and maintain learners’ engagement in the learning process despite chatbot disuse. Tiffany Wenting Li, Yifan Song 0007, Hari Sundaram, Karrie Karahalios |
CHI | 4 |
| 2026 | Control in Context: How Smart Home Users Navigate Proxy-based SchemesabstractA homeowner controls their smart home devices along a spectrum of approaches, ranging from physical device control to various proxy-based control modalities. This paper studies how and why users move along this spectrum in their day-to-day lives, building upon existing research that focused only on specific interactions. We surveyed smart home owners (N = 43 users), and conducted follow-up interviews with a subset of the survey participants (N = 8). Our studies allow us to both distill specific contexts and experiences of smart home owners as they navigate the control spectrum, as well as to describe how their experiences (both positive and negative) shape their tendencies to control devices in a particular way. These insights lead us to propose practical implications for designers and researchers of smart home management systems, including the need to support flexible control scheme transitions, reduce switching costs, and account for temporal and spatial heterogeneity in the evaluation and design of control systems. Ali Zaidi, Anna Karanika, Ti-Chung Cheng, Yi-Shyuan Chiang, Camille Cobb, Indranil Gupta, Karrie Karahalios |
CHI | 7 |
| 2025 | From Sociotechnical Gaps to Solutions: Designing AI Tools with Parents to Address Special Education Advocacy Barriers in IEP Processes
Ali Zaidi, Karrie Karahalios |
Conference on Designing Interactive Systems | 2 |
| 2025 | "I'd Never Actually Realized How Big An Impact It Had Until Now": Perspectives of University Students with Disabilities on Generative Artificial Intelligence
Alex Atcheson, Omar Khan 0004, Brian Siemann, Anika Jain, Karrie Karahalios |
CHI | 5 |
| 2025 | Organize, Then Vote: Exploring Cognitive Load in Quadratic Survey InterfacesabstractQuadratic Surveys (QSs) elicit more accurate preferences than traditional methods like Likert-scale surveys. However, the cognitive load associated with QSs has hindered their adoption in digital surveys for collective decision-making. We introduce a two-phase "organize-then-vote" QS to reduce cognitive load. As interface design significantly impacts survey results and accuracy, our design scaffolds survey takers' decision-making while managing the cognitive load imposed by QS. In a 2x2 between-subject in-lab study on public resource allotment, we compared our interface with a traditional text interface across a QS with 6 (short) and 24 (long) options. Two-phase interface participants spent more time per option and exhibited shorter voting edit distances. We qualitatively observed shifts in cognitive effort from mechanical operations to constructing more comprehensive preferences. We conclude that this interface promoted deeper engagement, potentially reducing satisficing behaviors caused by cognitive overload in longer QSs. This research clarifies how human-centered design improves preference elicitation tools for collective decision-making. Ti-Chung Cheng, Yutong Zhang 0011, Yi-Hung Chou, Vinay Koshy, Tiffany Wenting Li, Karrie Karahalios, Hari Sundaram |
CHI | 6 |
| 2025 | Placebo Effect of Control Settings in Feeds Are Not Always StrongabstractPeer Reviewed Silas Hsu, Vinay Koshy, Kristen Vaccaro, Christian Sandvig, Karrie Karahalios |
CHI | 5 |
| 2025 | Can Learners Navigate Imperfect Generative Pedagogical Chatbots? An Analysis of Chatbot Errors on LearningabstractGenerative pedagogical chatbots offer a promising solution to transform personalized learning at scale, but their benefits are at risk because of the potential of providing inaccurate information. We have a limited understanding of how effectively learners handle factual chatbot errors and how these errors affect learners with varying backgrounds. This study addresses these questions in an ecologically valid open-ended online STEM learning environment. Using Bayesian causal inference and thematic analysis on survey and interview data from a quasi-experimental setting, we found that most participants struggled to detect factual errors even with access to reading materials and the Internet. Undetected errors harmed learning outcomes and self-efficacy, underscoring the need to help learners evaluate chatbot responses. By analyzing participants' evaluation strategies, we identified challenges during error management and suggested ideas on designing effective supporting resources and learner empowerment. Finally, we revealed differential impacts of chatbot errors across learners and called for personalized support and deployment. Tiffany Wenting Li, Yifan Song 0007, Hari Sundaram, Karrie Karahalios |
L@S | 4 |
| 2025 | Venire: A Machine Learning-Guided Panel Review System for Community Content ModerationabstractResearch into community content moderation often assumes that moderation teams govern with a single, unified voice. However, recent work has found that moderators disagree with one another at modest, but concerning rates. The problem is not the root disagreements themselves. Subjectivity in moderation is unavoidable, and there are clear benefits to including diverse perspectives within a moderation team. Instead, the crux of the issue is that, due to resource constraints, moderation decisions end up being made by individual decision-makers. The result is decision-making that is inconsistent, which is frustrating for community members. To address this, we develop Venire, an ML-backed system for panel review on Reddit. Venire uses a machine learning model trained on log data to identify the cases where moderators are most likely to disagree. Venire fast-tracks these cases for multi-person review. Ideally, Venire allows moderators to surface and resolve disagreements that would have otherwise gone unnoticed. We conduct three studies through which we design and evaluate Venire: a set of formative interviews with moderators, technical evaluations on two datasets, and a think-aloud study in which moderators used Venire to make decisions on real moderation cases. Quantitatively, we demonstrate that Venire is able to improve decision consistency and surface latent disagreements. Qualitatively, we find that Venire helps moderators resolve difficult moderation cases more confidently. Venire represents a novel paradigm for human-AI content moderation, and shifts the conversation from replacing human decision-making to supporting it. Vinay Koshy, Frederick Choi, Yi-Shyuan Chiang, Hari Sundaram, Eshwar Chandrasekharan, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 6 |
| 2024 | Not What it Used to Be: Characterizing Content and User-base Changes in Newly Created Online CommunitiesabstractAttracting new members is vital to the health of many online communities. Yet, prior qualitative work suggests that newcomers to online communities can be disruptive – either due to a lack of awareness around existing community norms or to differing expectations around how the community should operate. Consequently, communities may have to navigate a trade-off between growth and development of community identity. We evaluate the presence of this trade-off through a longitudinal analysis of two years of commenting data for each of 1,620 Reddit communities. We find that, on average, communities become less linguistically distinctive as they grow. These changes appear to be driven almost equally by newcomers and returning users. Surprisingly, neither heavily moderated communities nor communities undergoing major user-base diversification are any more or less likely to maintaining distinctiveness. Taken together, our results complicate the assumption that growth is inherently beneficial for online communities. Alex Atcheson, Vinay Koshy, Karrie Karahalios |
CHI | 3 |
| 2024 | Choosing What You Want Versus Getting What You Want: An Experiment with Choice in Video Ad PlacementabstractUser agency and control serve as cornerstones of design in HCI, with numerous studies finding that choice improves user experiences. However, few studies examine how users benefit from the act of choosing, independent from the fulfillment of their chosen option; making this distinction is crucial for refining guidelines on when to provide user control. In our experiment on YouTube, participants randomly experienced either a pre-roll ad, a mid-roll ad, or a choice between the two. Participants then rated their subjective experiences. Mid-roll ads negatively affected experience ratings, but ratings between those choosing a pre-roll ad and those assigned a pre-roll ad were similar. That is, the right ad timing had a much larger impact than choosing an ad timing. The findings suggest that user interfaces should not offer choices solely for the sake of offering choices, and suggest scenarios where automation would be preferable to fine-grained user control. Silas Hsu, Karrie Karahalios |
CHI | 2 |
| 2024 | Enhancing Child Vocalization Classification with Phonetically-Tuned Embeddings for Assisting Autism Diagnosis
Jialu Li 0002, Mark Hasegawa-Johnson, Karrie Karahalios |
INTERSPEECH | 3 |
| 2024 | Visualization for improving foreign language pronunciation
Charlotte Yoder, Karrie Karahalios, Mark Hasegawa-Johnson, Shreyansh Agrawal |
INTERSPEECH | 2 |
| 2024 | MIRACLE: An Online, Explainable Multimodal Interactive Concept Learning SystemabstractWe present MIRACLE, a system for online, interpretable visual concept and video action recognition. Through a chat interface, users query the recognition system with an uploaded image or video. For images, MIRACLE returns concept predictions from its structured knowledge base, justifying its predictions with heatmaps and natural language-based attribute detections. For videos, MIRACLE predicts an action and justifies its prediction with time varying entity-entity relations. With its ability to learn new concepts in an online, few-shot manner and its support of dynamic changes to its knowledge base, MIRACLE represents a step forward in interpretable multimodal learning systems. Ansel Blume, Khanh Duy Nguyen, Zhenhailong Wang, Yangyi Chen, Michal Shlapentokh-Rothman, Xiaomeng Jin, Zhen Zhu 0006, Jiateng Liu, Kuan-Hao Huang, Mankeerat Sidhu, Xuanming Zhang, Vivian Liu, Raunak Sinha, Te-Lin Wu, Abhaysinh Zala, Elias Stengel-Eskin, Da Yin, Utkarsh Mall, Zhou Yu 0005, Kai-Wei Chang 0001, Camille Cobb, Karrie Karahalios, Lydia B. Chilton, Mohit Bansal, Nanyun Peng 0001, Carl Vondrick, Derek Hoiem, Heng Ji 0001 |
ACM Multimedia | 24 |
| 2023 | Inform the Uninformed: Improving Online Informed Consent Reading with an AI-Powered ChatbotabstractInformed consent is a core cornerstone of ethics in human subject research. Through the informed consent process, participants learn about the study procedure, benefits, risks, and more to make an informed decision. However, recent studies showed that current practices might lead to uninformed decisions and expose participants to unknown risks, especially in online studies. Without the researcher’s presence and guidance, online participants must read a lengthy form on their own with no answers to their questions. In this paper, we examined the role of an AI-powered chatbot in improving informed consent online. By comparing the chatbot with form-based interaction, we found the chatbot improved consent form reading, promoted participants’ feelings of agency, and closed the power gap between the participant and the researcher. Our exploratory analysis further revealed the altered power dynamic might eventually benefit study response quality. We discussed design implications for creating AI-powered chatbots to offer effective informed consent in broader settings. Ziang Xiao, Tiffany Wenting Li, Karrie Karahalios, Hari Sundaram |
CHI | 3 |
| 2023 | Am I Wrong, or Is the Autograder Wrong? Effects of AI Grading Mistakes on LearningabstractErrors in AI grading and feedback often have an intractable set of causes and are, by their nature, difficult to completely avoid. Since inaccurate feedback potentially harms learning, there is a need for designs and workflows that mitigate these harms. To better understand the mechanisms by which erroneous AI feedback impacts students’ learning, we conducted surveys and interviews that recorded students’ interactions with a short-answer AI autograder for “Explain in Plain English” code reading problems. Using causal modeling, we inferred the learning impacts of wrong answers marked as right (false positives, FPs) and right answers marked as wrong (false negatives, FNs). We further explored explanations for the learning impacts, including errors influencing participants’ engagement with feedback and assessments of their answers’ correctness, and participants’ prior performance in the class. Tiffany Wenting Li, Silas Hsu, Maxwell Fowler, Zhilin Zhang 0004, Craig B. Zilles, Karrie Karahalios |
ICER (1) | 6 |
| 2023 | What should I Ask: A Knowledge-driven Approach for Follow-up Questions Generation in Conversational Surveys
Yubin Ge, Ziang Xiao, Jana Diesner, Heng Ji 0001, Karrie Karahalios, Hari Sundaram |
PACLIC | 5 |
| 2023 | It Is All About Criticism: Understanding the Effect of Social Media Discourse on Legal Crowdfunding CampaignsabstractLegal crowdfunding is an emerging domain where lawyers and individuals raise funds to fight legal actions. To study how prospective donors can verify the credibility of legal campaigns, we analyzed the conversations surrounding these campaigns on Facebook. We discovered three primary themes associated with the perceptions of the contributors of legal campaigns: supporters posting admiring and appreciative comments, supporters posting critical and disapproving comments, and opponents posting critical and disapproving comments. We observed that while supporters criticized campaigns' opponents, biased media, and dishonest authorities, opponents criticized campaign owners, campaigns' objectives and opaque logistics. To understand the impact of these perspectives on donors, we followed up with an online survey study where we presented a legal campaign with its corresponding social media conversations. We found that critical comments impacted donation decisions more than appreciative comments. We concluded with design implications to better support potential donors to make more informed donation decisions. Sanorita Dey, Brittany R. L. Duff, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2023 | Inform, Explain, or Control: Techniques to Adjust End-User Performance Expectations for a Conversational Agent Facilitating Group Chat DiscussionsabstractA conversational agent (CA) effectively facilitates online group discussions at scale. However, users may have expectations about how well the CA would perform that do not match with the actual performance, compromising technology acceptance. We built a facilitator CA that detects a member who has low contribution during a synchronous group chat discussion and asks the person to participate more. We designed three techniques to set end-user expectations about how accurately the CA identifies an under-contributing member: 1)information: explicitly communicating the accuracy of the detection algorithm, 2)explanation: providing an overview of the algorithm and the data used for the detection, and 3)adjustment: enabling users to gain a feeling of control over the algorithm. We conducted an online experiment with 163 crowdworkers in which each group completed a collaborative decision-making task and experienced one of the techniques. Through surveys and interviews, we found that the explanation technique was the most effective strategy overall as it reduced user embarrassment, increased the perceived intelligence of the CA, and helped users better understand the detection algorithm. In contrast, the information technique reduced members' contributions and the adjustment technique led to a more negative perceived discussion experience. We also discovered that the interactions with other team members diluted the effects of the techniques on users' performance expectations and acceptance of the CA. We discuss implications for better designing expectation-setting techniques for AI-team collaboration such as ways to improve collaborative decision outcomes and quality of contributions. Hyo Jin Do, Ha Kyung Kong, Pooja Tetali, Karrie Karahalios, Brian P. Bailey |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2023 | Composing Team Compositions: An Examination of Instructors' Current Algorithmic Team Formation PracticesabstractInstructors using algorithmic team formation tools must decide which criteria (e.g., skills, demographics, etc.) to use to group students into teams based on their teamwork goals, and have many possible sources from which to draw these configurations (e.g., the literature, other faculty, their students, etc.). However, tools offer considerable flexibility and selecting ineffective configurations can lead to teams that do not collaborate successfully. Due to such tools' relative novelty, there is currently little knowledge of how instructors choose which of these sources to utilize, how they relate different criteria to their goals for the planned teamwork, or how they determine if their configuration or the generated teams are successful. To close this gap, we conducted a survey (N=77) and interview (N=21) study of instructors using CATME Team-Maker and other criteria-based processes to investigate instructors' goals and decisions when using team formation tools. The results showed that instructors prioritized students learning to work with diverse teammates and performed "sanity checks" on their formation approach's output to ensure that the generated teams would support this goal, especially focusing on criteria like gender and race. However, they sometimes struggled to relate their educational goals to specific settings in the tool. In general, they also did not solicit any input from students when configuring the tool, despite acknowledging that this information might be useful. By opening the "black box" of the algorithm to students, more learner-centered approaches to forming teams could therefore be a promising way to provide more support to instructors configuring algorithmic tools while at the same time supporting student agency and learning about teamwork. Emily M. Hastings, Vidushi Ojha, Benedict V. Austriaco, Karrie Karahalios, Brian P. Bailey |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2023 | Measuring User-Moderator Alignment on r/ChangeMyViewabstractSocial media sites like Reddit, Discord, and Clubhouse utilize a community-reliant approach to content moderation. Under this model, volunteer moderators are tasked with setting and enforcing content rules within the platforms' sub-communities. However, few mechanisms exist to ensure that the rules set by moderators reflect the values of their community. Misalignments between users and moderators can be detrimental to community health. Yet little quantitative work has been done to evaluate the prevalence or nature of user-moderator misalignment. Through a survey of 798 users on r/ChangeMyView, we evaluate user-moderator alignment at the level of policy-awareness (does users know what the rules are?), practice-awareness (do users know how the rules are applied?) and policy-/practice-support (do users agree with the rules and how they are applied?). We find that policy-support is high, while practice-support is low -- using a hierarchical Bayesian model we estimate the correlation between community opinion and moderator decisions to range from .14 to .45 across subreddit rules. Surprisingly, these correlations were only slightly higher when users were asked to predict moderator actions, demonstrating low awareness of moderation practices. Our findings demonstrate the need for careful analysis of user-moderator alignment at multiple levels. We argue that future work should focus on building tools to empower communities to conduct these analyses themselves. Vinay Koshy, Tanvi Bajpai, Eshwar Chandrasekharan, Hari Sundaram, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 5 |
| 2022 | Re-imagining the Power of Priming and Framing Effects in the Context of Political Crowdfunding CampaignsabstractIn recent years, political crowdfunding campaigns have emerged through which politicians raise money to fund their election campaigns. Divisive issues discussed in these campaigns may not only motivate donations but also could have a broader priming effect on people’s social opinions. In the U.S., more than one-third of the population with moderate opinions show a tendency to swing their opinion based on recent and more accessible events. In this paper, we ask: can such campaigns further prime people’s responses to partisan topics, even when we discuss those topics in a non-political context? To answer this question, we analyzed the influence of exposure to a political candidate’s crowdfunding campaign on responses to a subsequently seen, unrelated scientific topic that is not inherently political but is seen as partisan in the U.S. (climate change). We found that exposure to an attitude-inconsistent political candidate’s crowdfunding campaign (a campaign that is counter to someone’s existing political beliefs) can have a significant priming effect on subsequently seen politically charged topics. This effect may occur due to the activation of in-group identity by the candidate’s partisan campaign. Guided by these findings, we investigated elements that can mitigate this self-categorization effect. We found that carefully designed content following framing techniques such as schema framing and threat/safety framing can mitigate people’s sense of self-categorization toward non-political topics. Sanorita Dey, Brittany R. L. Duff, Karrie Karahalios |
CHI | 3 |
| 2022 | A Learner-Centered Technique for Collectively Configuring Inputs for an Algorithmic Team Formation ToolabstractThe configuration that an instructor enters into an algorithmic team formation tool determines how students are grouped into teams, impacting their learning experiences. One way to decide the configuration is to solicit input from the students. Prior work has investigated the criteria students prefer for team formation, but has not studied how students prioritize the criteria or to what degree students agree with each other. This paper describes a workflow for gathering student preferences for how to weight the criteria entered into a team formation tool, and presents the results of a study in which the workflow was implemented in four semesters of the same project-based design course. In the most recent semester, the workflow was supplemented with an online peer discussion to learn about students' rationale for their selections. Our results show that students want to be grouped with other students who share the same course commitment and compatible schedules the most. Students prioritize demographic attributes next, and then task skills such as programming needed for the project work. We found these outcomes to be consistent in each instance of the course. Instructors can use our results to guide team formation in their own project-based design courses and replicate our workflow to gather student preferences for team formation in any course. Emily M. Hastings, Sneha R. Krishna Kumaran, Karrie Karahalios, Brian P. Bailey |
SIGCSE (1) | 3 |
| 2022 | Our Browser Extension Lets Readers Change the Headlines on News Articles, and You Won't Believe What They Did!abstractHeadlines play a critical role in how users perceive articles. But many headline publishers craft headlines in ways that either attract clicks in an attempt to earn ad revenue, or misinform users or manipulate their opinions for malicious intents. Such headlines can do harm since many users simply skim and share headlines without reading the articles in full. We present an exploratory browser extension that empowers users to suggest headlines they deem better for news articles. Users can view headlines suggested by other users that they follow as they browse websites. We conducted a study of 27 users who used the extension for one week to read news and suggest headlines. We found that users saw value in the tool and used it to change headlines that they found in need of improvement. We characterize the changes that people make to headlines if enabled. We also report on a followup study we conducted with 312 participants to evaluate headlines suggested by the tool. The purpose of the study was to examine whether headlines suggested by untrained users could be preferred over original headlines by professional editors. We found that a substantial number of the suggested headlines were indeed preferred. Our work explores the designs for, and opportunities and consequences of, empowering news consumers by giving them control over the content curation process. Farnaz Jahanbakhsh, Amy X. Zhang, Karrie Karahalios, David R. Karger |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2022 | Designing a Medical Crowdfunding Website from Sense of Community TheoryabstractA sense of community is important in encouraging people to contribute to a variety of causes and the communities that support them. Researchers have identified website design features that can engender a sense of community on sites to promote contributions. However, most findings about design features are based on observational empirical research testing single features at a time or on standard practice and rarely use integrated theories to provide rationale for their design suggestions. This work investigates ways to re-design an entire website---with a simulated medical crowdfunding interface entitled Community Journey---informed by Sense of Community Theory to increase site visitors' sense of community and contributions. A between-subjects experiment revealed that the Community Journey interface increased potential supporters' sense of community and their overall willingness to contribute via monetary donations, campaign shares, personal messages, and offline support. Think-aloud interviews identified the interface features responsible for the overall increase in willingness to contribute. Finally, we suggest theory driven design implications for creating websites to build a strong support community and to encourage various contributions. Jennifer G. Kim, Robert E. Kraut, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2022 | Power Dynamics and Value Conflicts in Designing and Maintaining Socio-Technical Algorithmic ProcessesabstractHow do power dynamics and value conflicts affect our ability to design and maintain socio-technical algorithmic processes? In this paper, we study the SIGCHI student volunteer (SV) selection process that uses a weighted semi-randomized algorithm to recruit a desired pool of volunteers. Our interviews with the community members showed that the process is complex and socio-technical; the algorithm's outputs are interpreted and adjusted by the conference organizers to reflect the community values while ensuring the selection of effective volunteers to help with organizing the conference. This provides a stage in which the power dynamics and value conflicts among the stakeholders play salient roles in determining how the process was perceived and envisioned. For instance, non-organizers of the conference found the algorithm used in the selection process to be a power-balancer that places a check on the organizers who oversee the process. However, even with a participatory process to elicit the algorithm's weights, the power dynamics and value conflicts between the participants made it difficult to reach a consensus on what the SV selection process should consider and prioritize. Our findings highlight the importance of value transparency -- the type of transparency that focuses on explaining why a decision was made rather than how it was made -- as a mechanism for resolving such conflicts. Based on our findings, we lay out design recommendations that can guide communities to better design and maintain algorithmic socio-technical processes over time in the face of power dynamics and value conflicts. Joon Sung Park 0001, Karrie Karahalios, Niloufar Salehi, Motahhare Eslami |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2022 | Deconstructing Categorization in Visualization Recommendation: A Taxonomy and Comparative StudyabstractVisualization recommendation (VisRec) systems provide users with suggestions for potentially interesting and useful next steps during exploratory data analysis. These recommendations are typically organized into categories based on their analytical actions, i.e., operations employed to transition from the current exploration state to a recommended visualization. However, despite the emergence of a plethora of VisRec systems in recent work, the utility of the categories employed by these systems in analytical workflows has not been systematically investigated. Our article explores the efficacy of recommendation categories by formalizing a taxonomy of common categories and developing a system, Frontier, that implements these categories. Using Frontier, we evaluate workflow strategies adopted by users and how categories influence those strategies. Participants found recommendations that add attributes to enhance the current visualization and recommendations that filter to sub-populations to be comparatively most useful during data exploration. Our findings pave the way for next-generation VisRec systems that are adaptive and personalized via carefully chosen, effective recommendation categories. Doris Jung Lin Lee, Vidya Setlur, Melanie Tory, Karrie Karahalios, Aditya G. Parameswaran |
IEEE Trans. Vis. Comput. Graph. | 4 |
| 2021 | Attitudes Surrounding an Imperfect AI AutograderabstractDeployment of AI assessment tools in education is widespread, but work on students’ interactions and attitudes towards imperfect autograders is comparatively lacking. This paper presents students’ perceptions surrounding a ∼ 90% accurate automated short-answer grader that determined homework and exam credit in a college-level computer science course. Using surveys and interviews, we investigated students’ knowledge about the autograder and their attitudes. Silas Hsu, Tiffany Wenting Li, Zhilin Zhang 0004, Maxwell Fowler, Craig B. Zilles, Karrie Karahalios |
CHI | 6 |
| 2021 | "We Just Use What They Give Us": Understanding Passenger User Perspectives in Smart HomesabstractWith a plethora of off-the-shelf smart home devices available commercially, people are increasingly taking a do-it-yourself approach to configuring their smart homes. While this allows for customization, users responsible for smart home configuration often end up with more control over the devices than other household members. This separates those who introduce new functionality to the smart home (pilot users) from those who do not (passenger users). To investigate the prevalence and impact of pilot-passenger user relationships, we conducted a Mechanical Turk survey and a series of one-hour interviews. Our results suggest that pilot-passenger relationships are common in multi-user households and shape how people form habits around devices. We find from interview data that smart homes reflect the values of their pilot users, making it harder for passenger users to incorporate their devices into daily life. We conclude the paper with design recommendations to improve passenger and pilot user experience. Vinay Koshy, Joon Sung Park 0001, Ti-Chung Cheng, Karrie Karahalios |
CHI | 4 |
| 2021 | "I can show what I really like.": Eliciting Preferences via Quadratic VotingabstractSurveys are a common instrument to gauge self-reported opinions from the crowd for scholars in the CSCW community, the social sciences, and many other research areas. Researchers often use surveys to prioritize a subset of given options when there are resource constraints. Over the past century, researchers have developed a wide range of surveying techniques, including one of the most popular instruments, the Likert ordinal scale, to elicit individual preferences. However, the challenge to elicit accurate and rich self-reported responses with surveys in a resource-constrained context still persists today. In this study, we examine Quadratic Voting (QV), a voting mechanism powered by the affordances of a modern computer and straddles ratings and rankings approaches, as an alternative online survey technique. We argue that QV could elicit more accurate self-reported responses compared to the Likert scale when the goal is to understand relative preferences under resource constraints. We conducted two randomized controlled experiments on Amazon Mechanical Turk, one in the context of public opinion polling and the other in a human-computer interaction user study. Based on our Bayesian analysis results, a QV survey with a sufficient amount of voice credits, aligned significantly closer to participants' incentive-compatible behaviors than a Likert scale survey, with a medium to high effect size. In addition, we extended QV's application scenario from typical public policy and education research to a problem setting familiar to the CSCW community: a prototypical HCI user study. Our experiment results, QV survey design, and QV interface serve as a stepping stone for CSCW researchers to further explore this surveying methodology in their studies and encourage decision-makers from other communities to consider QV as a promising alternative. Ti-Chung Cheng, Tiffany Wenting Li, Yi-Hung Chou, Karrie Karahalios, Hari Sundaram |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2021 | StreamSketch: Exploring Multi-Modal Interactions in Creative Live StreamsabstractCreative live streams, where artists or designers demonstrate their creative process, have emerged as a unique and popular genre of live streams due to the real-time interactivity they afford. However, streamer-viewer interactions on most live streaming platforms only enable users to utilize text and emojis to communicate, which limits what viewers can convey and share in real time. To investigate the design space of potential visual and non-textual modalities within creative live streams, we first analyzed existing Twitch extensions and conducted a formative study with streamers who share creative activities to uncover key challenges that these streamers face. We then designed and implemented a prototype system, StreamSketch, which enables viewers and streamers to interact during live streams using multiple modalities, including freeform sketches and text. The prototype was evaluated by two professional artist streamers and their viewers during six streaming sessions. Overall, streamers and viewers found that StreamSketch provided increased engagement and new affordances compared to the traditional text-only modality, and highlighted how efficiency, moderation, and tool integration were continued challenges. Zhicong Lu, Rubaiat Habib Kazi, Li-Yi Wei, Mira Dontcheva, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 5 |
| 2021 | Contestability For Content ModerationabstractContent moderation systems for social media have had numerous issues of bias, in terms of race, gender, and ability among many others. One proposal for addressing such issues in automated decision making is by designing for contestability, whereby users can shape and influence how decisions are made. In this study, we conduct a series of participatory design workshops with participants from communities that have experienced problems with social media content moderation in the past. Together with participants, we explore the idea of designing for contestability in content moderation and find that users' designs suggest three fruitful, practical avenues: adding representation, improving communication, and designing with compassion. We conclude with design recommendations drawn from participants' proposals, and reflect on the challenges that remain. Kristen Vaccaro, Ziang Xiao, Kevin Hamilton, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2021 | NOAH: Interactive Spreadsheet Exploration with Dynamic Hierarchical OverviewsabstractSpreadsheet systems are by far the most popular platform for data exploration on the planet, supporting millions of rows of data. However, exploring spreadsheets that are this large via operations such as scrolling or issuing formulae can be overwhelming and error-prone. Users easily lose context and suffer from cognitive and mechanical burdens while issuing formulae on data spanning multiple screens. To address these challenges, we introduce dynamic hierarchical overviews that are embedded alongside spreadsheets. Users can employ this overview to explore the data at various granularities, zooming in and out of the spreadsheet. They can issue formulae over data subsets without cumbersome scrolling or range selection, enabling users to gain a high or low-level perspective of the spreadsheet. An implementation of our dynamic hierarchical overview, NOAH, integrated within DataSpread, preserves spreadsheet semantics and look and feel, while introducing such enhancements. Our user studies demonstrate that NOAH makes it more intuitive, easier, and faster to navigate spreadsheet data compared to traditional spreadsheets like Microsoft Excel and spreadsheet plug-ins like Pivot Table, for a variety of exploration tasks; participants made fewer mistakes in NOAH while being faster in completing the tasks. Sajjadur Rahman, Mangesh Bendre, Shichu Zhu, Zhaoyuan Su, Karrie Karahalios, Aditya G. Parameswaran |
Proc. VLDB Endow. | 6 |
| 2020 | Enriched Social Translucence in Medical CrowdfundingabstractSocial translucence theory argues that online collaboration systems should make contributors' activities visible to better achieve a common goal. Currently in medical crowdfunding sites, various non-monetary contributions integral to the success of a campaign, such as campaign promotions and offline support, are less visible than monetary contributions. Our work investigates ways to enrich social translucence in medical crowdfunding by aggregating and visualizing non-monetary contributions that reside outside of the current crowdfunding space. Three different styles of interactive visualizations were built and evaluated with medical crowdfunding beneficiaries and contributors. Our results reveal the perceived benefits and challenges of making the previously invisible non-monetary contributions visible using various design features in the visualizations. We discuss our findings based on the social translucence framework--visibility, awareness, and accountability--and suggest design guidelines for crowdfunding platform designers. Jennifer G. Kim, Ha Kyung Kong, Hwajung Hong, Karrie Karahalios |
Conference on Designing Interactive Systems | 4 |
| 2020 | LIFT: Integrating Stakeholder Voices into Algorithmic Team FormationabstractTeam formation tools assume instructors should configure the criteria for creating teams, precluding students from participating in a process affecting their learning experience. We propose LIFT, a novel learner-centered workflow where students propose, vote for, and weigh the criteria used as inputs to the team formation algorithm. We conducted an experiment (N=289) comparing LIFT to the usual instructor-led process, and interviewed participants to evaluate their perceptions of LIFT and its outcomes. Learners proposed novel criteria not included in existing algorithmic tools, such as organizational style. They avoided criteria like gender and GPA that instructors frequently select, and preferred those promoting efficient collaboration. LIFT led to team outcomes comparable to those achieved by the instructor-led approach, and teams valued having control of the team formation process. We provide instructors and designers with a workflow and evidence supporting giving learners control of the algorithmic process used for grouping them into teams. Emily M. Hastings, Albatool A. Alamri, Andrew Kuznetsov, Christine Pisarczyk, Karrie Karahalios, Darko Marinov, Brian P. Bailey |
CHI | 5 |
| 2020 | Awareness, Navigation, and Use of Feed Control Settings OnlineabstractControl settings are abundant and have significant effects on user experiences. One example of an impactful but understudied area is feed settings. In this study, we investigated awareness, navigation, and use of feed settings. We began by creating a taxonomy of feed settings on social media and search sites. Via an online survey, we measured awareness of Facebook feed settings. An in-person interview study then investigated how people navigated to and chose to set feed settings on their own feeds. We discovered that many participants did not believe ad personalization feed settings existed. Furthermore, we discovered a misalignment in the expectation and the function of settings, especially of ad personalization settings for many participants. Despite all participants struggling to find at least one setting, participants overall wanted to use settings: 94% altered at least one setting they encountered. From these results, we discuss implications and suggest design guidelines for settings. Silas Hsu, Kristen Vaccaro, Yin Yue, Aimee Rickman, Karrie Karahalios |
CHI | 5 |
| 2020 | Addressing Cognitive and Emotional Barriers in Parent-Clinician Communication through Behavioral Visualization WebtoolsabstractEffective communication between clinicians and parents of young children with developmental delays can decrease parents' anxiety, help them handle bad news, and improve their adherence to proposed interventions. However, parents have reported dissatisfaction regarding their current communication with clinicians, and they face cognitive and emotional challenges when discussing their child's developmental delays. In this paper, we present visualization as a facilitator of parent-clinician communication and how it could address existing communication challenges. Parents and clinicians anticipated visualization webtools would aid their communication by helping parents gain a better understanding of their child, acting as objective evidence, and highlighting the strength of the child as well as important medical concepts. In addition, visualization can act as a longitudinal record, helping parents track, explore, and share their child's developmental progress. Finally, we propose visualization as a tool to guide parents in their transition from feeling emotional and disempowered to advocating with confidence. Ha Kyung Kong, Karrie Karahalios |
CHI | 2 |
| 2020 | Random, Messy, Funny, Raw: Finstas as Intimate Reconfigurations of Social MediaabstractAmong many young people, the creation of a finsta-a portmanteau of "fake" and "Instagram" which describes secondary Instagram accounts-provides an outlet to share emotional, low-quality, or indecorous content with their close friends. To study why people create and maintain finstas, we conducted a qualitative study through interviews with finsta users and content analysis of video bloggers exposing their finsta on YouTube. We found that one way that young people deal with mounting social pressures is by reconfiguring online platforms and changing their purposes, norms, expectations, and currencies. Carving out smaller spaces accessible only to close friends allows users the opportunity for a more unguarded, vulnerable, and unserious performance. Drawing on feminist theory, we term this process intimate reconfiguration. Through this reconfiguration finsta users repurpose an existing and widely-used social platform to create opportunities for more meaningful and reciprocal forms of social support. Sijia Xiao, Danaé Metaxa, Joon Sung Park 0001, Karrie Karahalios, Niloufar Salehi |
CHI | 4 |
| 2020 | Auditing Race and Gender Discrimination in Online Housing Markets
Joshua Asplund, Motahhare Eslami, Hari Sundaram, Christian Sandvig, Karrie Karahalios |
ICWSM | 5 |
| 2020 | Recommendation for video advertisements based on personality traits and companion contentabstractPeople encounter video ads every day when they access online content. While ads can be annoying or greeted with resistance, they can also be seen as informative and enjoyable. We asked the question, what might make an ad more enjoyable? And, do people with different personality traits prefer to watch different ads --- could it be possible to better match ads and people? To answer these questions, we conducted an online study where we asked people to watch video ads of different emotional sentiments. We also measured their personality traits through an online survey. We found that the sentiment of people's preferred video ads varies significantly based on their personality traits. Additionally, we investigated when these ads are accompanied by content, how the emotional state induced by accompanying content affects people's ad preferences. We found that there was a complex relationship between people's emotional state induced by accompanying content and their ad preference when an ad highlighted either an alertness or calmness sentiment. However, when an ad highlighted activeness and amusement, the relationship was not significant. Overall, our results show that people's personality traits and their emotional states are two key elements that predict the tone of their preferred video ads. Sanorita Dey, Brittany R. L. Duff, Niyati Chhaya, Wai Fu, Viswanathan (Vishy) Swaminathan, Karrie Karahalios |
IUI | 6 |
| 2020 | Benchmarking Spreadsheet SystemsabstractSpreadsheet systems are used for storing and analyzing data across domains by programmers and non-programmers alike.While spreadsheet systems have continued to support increasingly large datasets, they are prone to hanging and freezing while performing computations even on much smaller ones. We present a benchmarking study that evaluates and compares the performance of three popular systems, Microsoft Excel, LibreOffice Calc, and Google Sheets, on a range of canonical spreadsheet computation operations. We find that spreadsheet systems lack interactivity for several operations, on datasets well below their advertised scalability limits. We further evaluate whether spreadsheet systems adopt database optimization techniques such as indexing, intelligent data layout, and incremental and shared computation,to efficiently execute computation operations. We outline several ways future spreadsheet systems can be redesigned to offer interactive response times on large datasets. Sajjadur Rahman, Kelly Mack, Mangesh Bendre, Karrie Karahalios, Aditya G. Parameswaran |
SIGMOD Conference | 5 |
| 2020 | ShapeSearch: A Flexible and Efficient System for Shape-based Exploration of TrendlinesabstractIdentifying trendline visualizations with desired patterns is a common task during data exploration. Existing visual analytics tools offer limited flexibility, expressiveness, and scalability for such tasks, especially when the pattern of interest is under-specified and approximate. We propose ShapeSearch, an efficient and flexible pattern-searching tool, that enables the search for desired patterns via multiple mechanisms: sketch, natural-language, and visual regular expressions. We develop a novel shape querying algebra, with a minimal set of primitives and operators that can express a wide variety of shape search queries, and design a natural- language and regex-based parser to translate user queries to the algebraic representation. To execute these queries within interactive response times, ShapeSearch uses a fast shape algebra execution engine with query-aware optimizations, and perceptually-aware scoring methodologies. We present a thorough evaluation of the system, including a user study, a case study involving genomics data analysis, as well as performance experiments, comparing against state-of-the-art trendline shape matching approaches-that together demonstrate the usability and scalability of ShapeSearch. Tarique Siddiqui, Paul Luh, Zesheng Wang 0001, Karrie Karahalios, Aditya G. Parameswaran |
SIGMOD Conference | 4 |
| 2020 | UIST+CSCW: A Celebration of Systems Research in Collaborative and Social ComputingabstractThis joint panel between UIST and CSCW brings together leading researchers at the intersection of the conferences-systems researchers in collaborative and social computing-to engage in a discussion and retrospective. Pairs of panelists will represent each decade since the founding of the conferences, sharing a brief retrospective that surveys the most influential papers of that decade, the zeitgeist of the problems that were popular that decade and why, and what each decade's work has to say to the decades that came before and after. The panel is intended as a space to celebrate advances in the field, and reflect on the burdens and opportunities that it faces ahead. Michael S. Bernstein, Irene Greif, Wendy E. Mackay, Hiroshi Ishii 0001, Jonathan Grudin, Karrie Karahalios, Meredith Ringel Morris, Aniket Kittur, Jaime Teevan, Amy X. Zhang, Niloufar Salehi |
UIST | 6 |
| 2020 | "It's all about conversation": Challenges and Concerns of Faculty and Students in the Arts, Humanities, and the Social Sciences about Education at ScaleabstractAs colleges and universities continue their commitment to increasing access to higher education through offering education online and at scale, attention on teaching open-ended subjects online and at scale, mainly the arts, humanities, and the social sciences, remains limited. While existing work in scaling open-ended courses primarily focuses on the evaluation and feedback of open-ended assignments, there is a lack of understanding of how to effectively teach open-ended, university-level courses at scale. To better understand the needs of teaching large-scale, open-ended courses online effectively in a university setting, we conducted a mixed-methods study with university instructors and students, using surveys and interviews, and identified five critical pedagogical elements that distinguish the teaching and learning experiences in an open-ended course from that in a non-open-ended course. An overarching theme for the five elements was the need to support students' self-expression. We further uncovered open challenges and opportunities when incorporating the five critical pedagogical elements into large-scale, open-ended courses online in a university setting, and suggested six future research directions: (1) facilitate in-depth conversations, (2) create a studio-friendly environment, (3) adapt to open-ended assessment, (4) scale individual open-ended feedback, (5) establish trust for self-expression, and (6) personalize instruction and harness the benefits of student diversity. Tiffany Wenting Li, Karrie Karahalios, Hari Sundaram |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2020 | "At the End of the Day Facebook Does What ItWants": How Users Experience Contesting Algorithmic Content ModerationabstractInterest has grown in designing algorithmic decision making systems for contestability. In this work, we study how users experience contesting unfavorable social media content moderation decisions. A large-scale online experiment tests whether different forms of appeals can improve users' experiences of automated decision making. We study the impact on users' perceptions of the Fairness, Accountability, and Trustworthiness of algorithmic decisions, as well as their feelings of Control (FACT). Surprisingly, we find that none of the appeal designs improve FACT perceptions compared to a no appeal baseline. We qualitatively analyze how users write appeals, and find that they contest the decision itself, but also more fundamental issues like the goal of moderating content, the idea of automation, and the inconsistency of the system as a whole. We conclude with suggestions for -- as well as a discussion of the challenges of -- designing for contestability. Kristen Vaccaro, Christian Sandvig, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2020 | You can't always sketch what you want: Understanding Sensemaking in Visual Query SystemsabstractVisual query systems (VQSs) empower users to interactively search for line charts with desired visual patterns, typically specified using intuitive sketch-based interfaces. Despite decades of past work on VQSs, these efforts have not translated to adoption in practice, possibly because VQSs are largely evaluated in unrealistic lab-based settings. To remedy this gap in adoption, we collaborated with experts from three diverse domains-astronomy, genetics, and material science-via a year-long user-centered design process to develop a VQS that supports their workflow and analytical needs, and evaluate how VQSs can be used in practice. Our study results reveal that ad-hoc sketch-only querying is not as commonly used as prior work suggests, since analysts are often unable to precisely express their patterns of interest. In addition, we characterize three essential sensemaking processes supported by our enhanced VQS. We discover that participants employ all three processes, but in different proportions, depending on the analytical needs in each domain. Our findings suggest that all three sensemaking processes must be integrated in order to make future VQSs useful for a wide range of analytical inquiries. Doris Jung Lin Lee, John Lee 0005, Tarique Siddiqui, Karrie Karahalios, Aditya G. Parameswaran |
IEEE Trans. Vis. Comput. Graph. | 5 |
| 2019 | User Attitudes towards Algorithmic Opacity and Transparency in Online Reviewing PlatformsabstractAlgorithms exert great power in curating online information, yet are often opaque in their operation, and even existence. Since opaque algorithms sometimes make biased or deceptive decisions, many have called for increased transparency. However, little is known about how users perceive and interact with potentially biased and deceptive opaque algorithms. What factors are associated with these perceptions, and how does adding transparency into algorithmic systems change user attitudes? To address these questions, we conducted two studies: 1) an analysis of 242 users' online discussions about the Yelp review filtering algorithm and 2) an interview study with 15 Yelp users disclosing the algorithm's existence via a tool. We found that users question or defend this algorithm and its opacity depending on their engagement with and personal gain from the algorithm. We also found adding transparency into the algorithm changed users' attitudes towards the algorithm: users reported their intention to either write for the algorithm in future reviews or leave the platform. Motahhare Eslami, Kristen Vaccaro, Min Kyung Lee, Amit Elazari Bar On, Eric Gilbert, Karrie Karahalios |
CHI | 6 |
| 2019 | Trust and Recall of Information across Varying Degrees of Title-Visualization MisalignmentabstractVisualizations are emerging as a means of spreading digital misinformation. Prior work has shown that visualization interpretation can be manipulated through slanted titles that favor only one side of the visual story, yet people still think the visualization is impartial. In this work, we study whether such effects continue to exist when titles and visualizations exhibit greater degrees of misalignment: titles whose message differs from the visually cued message in the visualization, and titles whose message contradicts the visualization. We found that although titles with a contradictory slant triggered more people to identify bias compared to titles with a miscued slant, visualizations were persistently perceived as impartial by the majority. Further, people's recall of the visualization's message more frequently aligned with the titles than the visualization. Based on these results, we discuss the potential of leveraging textual components to detect and combat visual-based misinformation with text-based slants. Ha Kyung Kong, Zhicheng Liu 0001, Karrie Karahalios |
CHI | 3 |
| 2019 | Understanding Visual Cues in Visualizations Accompanied by Audio NarrationsabstractIt is often assumed that visual cues, which highlight specific parts of a visualization to guide the audience's attention, facilitate visualization storytelling and presentation. This assumption has not been systematically studied. We present an in-lab experiment and a Mechanical Turk study to examine the effects of integral and separable visual cues on the recall and comprehension of visualizations that are accompanied by audio narration. Eye-tracking data in the in-lab experiment confirm that cues helped the viewers focus on relevant parts of the visualization faster. We found that in general, visual cues did not have a significant effect on learning outcomes, but for specific cue techniques (e.g. glow) or specific chart types (e.g heatmap), cues significantly improved comprehension. Based on these results, we discuss how presenters might select visual cues depending on the role of the cues and the visualization type. Ha Kyung Kong, Zhicheng Liu 0001, Karrie Karahalios |
CHI | 4 |
| 2019 | Dimensional Analysis of Laughter in Female Conversational SpeechabstractHow do people hear laughter in expressive, unprompted speech? What is the range of expressivity and function of laughter in this speech, and how can laughter inform the recognition of higher-level expressive dimensions in a corpus? This paper presents a scalable method for collecting natural human description of laughter, transforming the description to a vector of quantifiable laughter dimensions, and deriving baseline classifiers for the different dimensions of expressive laughter. Then, it explores the impact of leveraging nuances of laughter in the recognition of higher-level, general expressive dimensions, discovered in the same way, such as genuine happiness, sarcasm, nervous reflection, and more. The performance of the low-level laughter classifiers is presented, along with the performance of the high-level laughter-aware and laughter-unaware classifiers. Mary Pietrowicz, Carla Agurto, Jonah Casebeer, Mark Hasegawa-Johnson, Karrie Karahalios, Guillermo A. Cecchi |
ICASSP | 5 |
| 2019 | Faster, Higher, Stronger: Redesigning Spreadsheets for ScaleabstractSpreadsheet tools are ubiquitous for interactive adhoc data management and analysis. With increasing dataset sizes, spreadsheet tools fall short-they freeze during heavy computation within the sheet (interactivity); they are hard to navigate when datasets go beyond a certain size (navigability); they only support cell-at-a-time computation, severely limiting analysis capabilities (expressiveness). We have been developing DATASPREAD to holistically unify databases and spreadsheets to leverage the benefits of both, with a spreadsheet-like front-end and a database-like backend. We demonstrate three key features of DATASPREAD to address the aforementioned spreadsheet scalability challenges in interactivity, navigability, and expressiveness1. Our demonstration will let attendees perform typical analysis tasks on Microsoft Excel and DATASPREAD side-by-side, providing a clear understanding of the improvements offered by DATASPREAD over traditional spreadsheet tools. Mangesh Bendre, Tana Wattanawaroon, Sajjadur Rahman, Kelly Mack, Shichu Zhu, Ping-Jing Yang, Kevin Chen-Chuan Chang, Karrie Karahalios, Aditya G. Parameswaran |
ICDE | 11 |
| 2019 | Search bias quantification: investigating political bias in social media and web searchabstractUsers frequently use search systems on the Web as well as online social media to learn about ongoing events and public opinion on personalities. Prior studies have shown that the top-ranked results returned by these search engines can shape user opinion about the topic (e.g., event or person) being searched. In case of polarizing topics like politics, where multiple competing perspectives exist, the political bias in the top search results can play a significant role in shaping public opinion towards (or away from) certain perspectives. Given the considerable impact that search bias can have on the user, we propose a generalizable search bias quantification framework that not only measures the political bias in ranked list output by the search system but also decouples the bias introduced by the different sources—input data and ranking system. We apply our framework to study the political bias in searches related to 2016 US Presidential primaries in Twitter social media search and find that both input data and ranking system matter in determining the final search output bias seen by the users. And finally, we use the framework to compare the relative bias for two popular search systems—Twitter social media search and Google web search—for queries related to politicians and political events. We end by discussing some potential solutions to signal the bias in the search results to make the users more aware of them. Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh 0001, Krishna P. Gummadi, Karrie Karahalios |
Inf. Retr. J. | 7 |
| 2019 | Quantifying Voter Biases in Online Platforms: An Instrumental Variable ApproachabstractIn content-based online platforms, use of aggregate user feedback (say, the sum of votes) is commonplace as the "gold standard" for measuring content quality. Use of vote aggregates, however, is at odds with the existing empirical literature, which suggests that voters are susceptible to different biases-reputation (e.g., of the poster), social influence (e.g., votes thus far), and position (e.g., answer position). Our goal is to quantify, in an observational setting, the degree of these biases in online platforms. Specifically, what are the causal effects of different impression signals-such as the reputation of the contributing user, aggregate vote thus far, and position of content-on a participant's vote on content? We adopt an instrumental variable (IV) framework to answer this question. We identify a set of candidate instruments, carefully analyze their validity, and then use the valid instruments to reveal the effects of the impression signals on votes. Our empirical study using log data from Stack Exchange websites shows that the bias estimates from our IV approach differ from the bias estimates from the ordinary least squares (OLS) method. In particular, OLS underestimates reputation bias (1.6-2.2x for gold badges) and position bias (up to 1.9x for the initial position) and overestimates social influence bias (1.8-2.3x for initial votes). The implications of our work include: redesigning user interface to avoid voter biases; making changes to platforms' policy to mitigate voter biases; detecting other forms of biases in online platforms. Himel Dev, Karrie Karahalios, Hari Sundaram |
Proc. ACM Hum. Comput. Interact. | 2 |
| 2019 | VidLyz: An Interactive Approach to Assist Novice Entrepreneurs in Making Persuasive Campaign VideosabstractVideos are essential for successful crowdfunding campaigns. However, without knowledge of the underlying persuasion factors, novice entrepreneurs may find it difficult to optimize their videos for success. This paper presents VidLyz, a novel assistive tool that allows users to explore the implications of audience-engagement persuasion factors in the context of campaign videos through contrasting examples, crowd-sourced subjective ratings, and feedback on engagement factors. VidLyz promotes active thinking about the impact of audience persuasion factors in making an effective campaign video by guiding novice entrepreneurs in planning materials of their videos on their own, with consideration of the product category, the target audience, and audience-product interactions. To evaluate our system, we collected subjective ratings and feedback on persuasion factors for 140 Kickstarter campaign videos from 2100 crowd workers and presented them through our prototype VidLyz tool. A user study with 45 novice users and five previous campaign creators found that our tool was useful for understanding the implication and relative importance of the persuasion factors of the campaign videos. The interactive and active thinking elements of VidLyz promoted novice users to make coherent and persuasive pre-production plan (using storyboards) for their proposed campaign videos. A follow-up user study showed that these storyboards had a higher likelihood of getting funded by crowd workers than those with less persuasive pre-production plan. We concluded with design implications to better support novice entrepreneurs. Sanorita Dey, Brittany R. L. Duff, Wai-Tat Fu, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2019 | A Slow Algorithm Improves Users' Assessments of the Algorithm's AccuracyabstractWith computational algorithms making an increasing number of deeply consequential, and often problematic judgments on our behalf, there is a growing interest in slowing down technology to encourage users to reflect on judgments made by algorithms. Prior work in slow technology has established slowness as an agent of reflection and serendipity; however, it has been unclear whether this waiting time actually helps users gain useful insight or any other benefits as they make judgments using an algorithm. To this end, we conducted a series of online and in-person between-subject user studies in which we isolate the impact of an algorithm's speed on how users incorporate the algorithm's advice when making judgments in the context of simple visual recognition tasks. We find that our participants followed good quality algorithms more and bad quality algorithms somewhat less if the response time of the algorithm is slower. Furthermore, qualitative analysis of the in-person study interviews reveals that the waiting was not time wasted, but was often used to reflect on the task and the estimation process of themselves and the algorithm, and to compare and reevaluate the two processes. Based on these findings, we outline design implications of future algorithmic systems. Joon Sung Park 0001, Rick Barber, Alex Kirlik, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2019 | Should We Use an Abstract Comic Form to Persuade?: Experiments with Online Charitable DonationabstractThis paper examines the use of the abstract comic form for persuading online charitable donations. Persuading individuals to contribute to charitable causes online is hard and responses to the appeals are typically low; charitable donations share the structure of public goods dilemmas where the rewards are distant and non-exclusive. In this paper, we examine if comics in abstract form are more persuasive than in the plain text form. Drawing on a rich literature on comics, we synthesized a three-panel abstract comic to create our appeal. We conducted a between-subject study with 307 participants from Amazon Mechanical Turk on the use of abstract comic form to appeal for charitable donations. As part of our experimental procedure, we sought to persuade individuals to contribute to a real charity focused on Autism research with monetary costs. We compared the average amount of donation to the charity under three conditions: the plain text message, an abstract comic that includes the plain text, and an abstract comic that additionally includes the social proof. We use Bayesian modeling to analyze the results, motivated by model transparency and its use in small-sized studies. Our experiments reveal that the message in abstract comic form elicited significantly more donations than text form (medium to large effect size=0.59). Incorporating social proof in the abstract comic message did not show a significant effect. Our studies have design implications: non-profits and governmental agencies interested in alleviating public goods dilemmas that share a similar structure to our experiment (single-shot task, distant, non-exclusive reward) ought to consider including messages in the abstract comic form as part of their online fund-raising campaign. Ziang Xiao, Po-Shiun Ho, Karrie Karahalios, Hari Sundaram |
Proc. ACM Hum. Comput. Interact. | 4 |
| 2018 | Effects of Socially Stigmatized Crowdfunding Campaigns in Shaping OpinionsabstractDonation-based crowdfunding platforms have an increasing number of campaigns on socially stigmatized topics. These platforms' widespread online reachability and the large flow of monetary donations have the potential to shape individuals' opinions by influencing their perceptions. However, little research has been done to investigate whether these campaigns impact individuals' opinions and how. We conducted an experiment to explore how an attitude-inconsistent campaign on fairness and equality for LGBTIQ people influenced participants' opinion on this topic. Although all the participants changed their perceived opinions after reading the support for the campaigns, participants opposing equality were less inclined to change their attitude than participants supporting equality. To examine this difference further, we conducted another experiment where participants were exposed to both attitude-consistent and attitude-inconsistent campaigns with varying levels of social support. Participants opposing equality showed less sensitivity to the level of social support, and wanted to donate significantly more money to anti-equality campaigns compared to those who supported equality. Results demonstrate the complex role of crowdfunding campaigns in shaping individuals' opinions on stigmatized topics. Sanorita Dey, Karrie Karahalios, Wai-Tat Fu |
CHI | 2 |
| 2018 | Communicating Algorithmic Process in Online Behavioral AdvertisingabstractAdvertisers develop algorithms to select the most relevant advertisements for users. However, the opacity of these algorithms, along with their potential for violating user privacy, has decreased user trust and preference in behavioral advertising. To mitigate this, advertisers have started to communicate algorithmic processes in behavioral advertising. However, how revealing parts of the algorithmic process affects users' perceptions towards ads and platforms is still an open question. To investigate this, we exposed 32 users to why an ad is shown to them, what advertising algorithms infer about them, and how advertisers use this information. Users preferred interpretable, non-creepy explanations about why an ad is presented, along with a recognizable link to their identity. We further found that exposing users to their algorithmically-derived attributes led to algorithm disillusionment---users found that advertising algorithms they thought were perfect were far from it. We propose design implications to effectively communicate information about advertising algorithms. Motahhare Eslami, Sneha R. Krishna Kumaran, Christian Sandvig, Karrie Karahalios |
CHI | 4 |
| 2018 | Understanding Identity Presentation in Medical CrowdfundingabstractPeople desire to present themselves favorably to others. However, medical crowdfunding beneficiaries are often expected to present their dire medical conditions and financial straits to solicit financial support. To investigate how beneficiaries convey their situation on medical crowdfunding pages and how contributors perceive the presented information, we interviewed both medical crowdfunding beneficiaries and contributors. While beneficiaries emphasized the serious of their medical situations to signal their deservedness of support, contributor participants gave less attention to that content. Rather, they focused on their impression of the beneficiary's character formed by various features of contributions such as the contributor's names, messages, and shared pictures. These contribution features further signaled common connections between the beneficiary and contributors and each contributor's unique involvement in the beneficiary's medical journey. However, the contribution amount resulted in judgement about other contributors. We suggest design opportunities and challenges that apply these results to the design of medical crowdfunding interfaces. Jennifer G. Kim, Hwajung Hong, Karrie Karahalios |
CHI | 3 |
| 2018 | Frames and Slants in Titles of Visualizations on Controversial TopicsabstractSlanted framing in news article titles induce bias and influence recall. While recent studies found that viewers focus extensively on titles when reading visualizations, the impact of titles in visualization remains underexplored. We study frames in visualization titles, and how the slanted framing of titles and the viewer's pre-existing attitude impact recall, perception of bias, and change of attitude. When asked to compose visualization titles, people used five existing news frames, an open-ended frame, and a statistics frame. We found that the slant of the title influenced the perceived main message of a visualization, with viewers deriving opposing messages from the same visualization. The results did not show any significant effect on attitude change. We highlight the danger of subtle statistics frames and viewers' unwarranted conviction of the neutrality of visualizations. Finally, we present a design implication for the generation of visualization titles and one for the viewing of titles. Ha Kyung Kong, Zhicheng Liu 0001, Karrie Karahalios |
CHI | 3 |
| 2018 | The Illusion of Control: Placebo Effects of Control SettingsabstractAlgorithmic prioritization is a growing focus for social media users. Control settings are one way for users to adjust the prioritization of their news feeds, but they prioritize feed content in a way that can be difficult to judge objectively. In this work, we study how users engage with difficult-to-validate controls. Via two paired studies using an experimental system -- one interview and one online study -- we found that control settings functioned as placebos. Viewers felt more satisfied with their feed when controls were present, whether they worked or not. We also examine how people engage in sensemaking around control settings, finding that users often take responsibility for violated expectations -- for both real and randomly functioning controls. Finally, we studied how users controlled their social media feeds in the wild. The use of existing social media controls had little impact on user's satisfaction with the feed; instead, users often turned to improvised solutions, like scrolling quickly, to see what they want. Kristen Vaccaro, Dylan Huang, Motahhare Eslami, Christian Sandvig, Kevin Hamilton, Karrie Karahalios |
CHI | 6 |
| 2018 | To Label or Not to Label: The Effect of Stance and Credibility Labels on Readers' Selection and Perception of News ArticlesabstractSocial media sites use different labels to help users find and select news feeds. For example, Blue Feed, Red Feed, a news feed created by the Wall Street Journal, use stance labels to separate news articles with opposing political ideologies to help people explore diverse opinions. To combat the spread of fake news, Facebook has experimented with putting credibility labels on news articles to help readers decide whether the content is trustworthy. To systematically understand the effects of stance and credibility labels on online news selection and consumption, we conducted a controlled experiment to study how these labels influence the selection, perceived extremeness, and level of agreement of news articles. Results show that stance labels may intensify selective exposure - a tendency for people to look for agreeable opinions -- and make people more vulnerable to polarized opinions and fake news. We found, however, that the effect of credibility labels on reducing selective exposure and recognizing fake news is limited. Although originally designed to encourage exposure to opposite viewpoints, stance labels can make fake news articles look more trustworthy, and they may lower people's perception of the extremeness of fake news articles. Our results have important implications on the subtle effects of stance and credibility labels on online news consumption. Mingkun Gao, Ziang Xiao, Karrie Karahalios, Wai-Tat Fu |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2018 | Structure or Nurture?: The Effects of Team-Building Activities and Team Composition on Team OutcomesabstractHow can instructors group students into teams that interact and learn effectively together? One strand of research advocates for grouping students into teams with "good" compositions such as skill diversity. Another strand argues for deploying team-building activities to foster interpersonal relations like psychological safety. Our work synthesizes these two strands of research. We describe an experiment (N=249) that compares how team composition vs. team-building activities affect student team outcomes. In two university courses, we composed student teams either randomly or using a criteria-based team formation tool. Teams further performed team-building activities that promoted either team or task outcomes. We collected project scores, and used surveys to measure psychological safety, perceived performance, and team satisfaction. Surprisingly, the criteria-based teams did not statistically differ from the random teams on any of the measures taken, despite having compositions that better satisfied the criteria defined by the instructor. Our findings argue that, for instructors deploying a team formation tool, creating an expectation among team members that their team can perform well is as important as tuning the criteria in the tool. We also found that student teams reported high levels of psychological safety, but these levels appeared to develop organically and were not affected by the activities or compositional strategies tested. We distill these and other findings into implications for the design and deployment of team formation tools for learning environments. Emily M. Hastings, Farnaz Jahanbakhsh, Karrie Karahalios, Darko Marinov, Brian P. Bailey |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2018 | Editors' Message
Karrie Karahalios, Andrés Monroy-Hernández, Airi Lampinen, Geraldine Fitzpatrick |
Proc. ACM Hum. Comput. Interact. | 1 |
| 2018 | ShapeSearch: Flexible Pattern-based Querying of Trend Line VisualizationsabstractFinding visualizations with desired patterns is a common goal during data exploration. However, due to the limited expressiveness and flexibility of existing visual analytics systems, pattern-based querying of visualizations has largely been a manual process. We demonstrate ShapeSearch, a system that enables users to express their desired patterns in trend lines using multiple flexible mechanisms --- including natural language and visual regular expressions, and automates the search via an optimized execution engine. Internally, the system leverages an expressive shape query algebra that supports a range of operators and primitives for representing ShapeSearch queries. In our demonstration, conference attendees will learn how the various components of ShapeSearch help accelerate scientific discovery by automating the search for meaningful patterns in trend lines in domains such as genomics and material science. Tarique Siddiqui, Paul Luh, Zesheng Wang 0001, Karrie Karahalios, Aditya G. Parameswaran |
Proc. VLDB Endow. | 4 |
| 2017 | Understanding the Effects of Endorsements in Scientific CrowdfundingabstractUnderstanding the factors that persuade backers to donate to research projects has become increasingly important with the rising popularity of scientific crowdfunding. Although there are many similarities between enterprise and scientific crowdfunding, some factors differentiate these two forms of crowdfunding. One such factor is the use of endorsements. The endorsement helps backers gain trust based on expert opinions about the competency of the researchers and the usefulness of the projects. We analyzed 810 endorsements from scientific campaigns posted on Experiment.com and derived a taxonomy of topics discussed in the endorsements. A regression analysis revealed that when endorsers explained the skills of the campaign owners, the probability of success of the campaign improved; on the contrary, when endorsers reiterated the goal of the project, the campaign was less likely to succeed. We conclude with design implications formulated from our findings to better support scientific crowdfunding. Sanorita Dey, Karrie Karahalios, Wai-Tat Fu |
CHI | 2 |
| 2017 | You Want Me to Work with Who?: Stakeholder Perceptions of Automated Team Formation in Project-based CoursesabstractInstructors are increasingly using algorithmic tools for team formation, yet little is known about how these tools are applied or how students and instructors perceive their use. We studied a representative team formation tool (CATME) in eight project-based courses. An instructor uses the tool to form teams by surveying students' working styles, skills, and demographics; then configuring these criteria as input into an algorithm that assigns teams. We surveyed students (N=277) in the courses to gauge their perceptions of the strengths and weaknesses of the tool and ideas for improving it. We also interviewed instructors (N=13) different from those who taught the eight courses to learn about their criteria selections and perceptions of the tool. Students valued the rational basis for forming teams but desired a stronger voice in criteria selection and explanations as to why they were assigned to a particular team. Instructors appreciated the efficiency of team formation but wanted to view exemplars of criteria used in similar courses. This work contributes recommendations for deploying team formation tools in educational settings and for better satisfying the goals of all stakeholders. Farnaz Jahanbakhsh, Wai-Tat Fu, Karrie Karahalios, Darko Marinov, Brian P. Bailey |
CHI | 3 |
| 2017 | Fast-Forwarding to Desired Visualizations with Zenvisage
Tarique Siddiqui, John Lee 0005, Albert Kim, Edward Xue, Xiaofo Yu, Sean Zou, Lijin Guo, Changfeng Liu, Chaoran Wang, Karrie Karahalios, Aditya G. Parameswaran |
CIDR | 10 |
| 2017 | The Art and Science of Persuasion: Not All Crowdfunding Campaign Videos are The SameabstractTo successfully raise money using crowdfunding, it is important for a campaign to communicate ideas or products effectively to the potential backers. One of the lesser explored but powerful components of a crowdfunding campaign is the campaign video. To better understand how videos affect campaign outcomes, we analyzed videos from 210 Kickstarter campaigns across three different project categories. In a mixed-methods study, we asked 3150 Amazon Mechanical Turk (MTurk) workers to evaluate the campaign videos. We found six recurrent factors from a qualitative analysis as well as quantitative analysis. Analysis revealed product related and video related factors that were predictive of the final outcome of campaigns over and above the static project representation features identified in previous studies. Both the qualitative and quantitative analysis showed that videos influenced perception differently for projects in different categories, and the differential perception was important for predicting successes of the projects. For example, in technology campaigns, projects perceived to have a lower level of complexity were more likely to be successful; but in design and fashion campaigns, projects perceived to have a higher level of complexity - which perhaps reflected craftsmanship - were more likely to be successful. We conclude with design implications to better support the video making process. Sanorita Dey, Brittany R. L. Duff, Karrie Karahalios, Wai-Tat Fu |
CSCW | 3 |
| 2017 | "Not by Money Alone": Social Support Opportunities in Medical Crowdfunding CampaignsabstractMedical crowdfunding helps patients receive financial support from their distributed social networks online. However, little is known about who the patient's supporters are, what support they provide, and why. To address this, we interviewed fifteen people involved in medical crowdfunding, including both beneficiaries and supporters. We found that support networks were larger than beneficiaries expected, with strangers offering support. Supporters offered not only monetary but also volunteering contributions including campaign creation, promotion, and external support. However, the emphasis medical crowdfunding interfaces place on monetary contributions led to social issues. Beneficiaries' close friends felt pressured to donate money they could not afford to give. And beneficiaries promoting the campaign worried they would be judged for requesting money. To mitigate these concerns, we suggest making the variety of volunteering contributions more visible and discuss the design challenges of including such signals in existing systems. Jennifer G. Kim, Kristen Vaccaro, Karrie Karahalios, Hwajung Hong |
CSCW | 3 |
| 2017 | Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social MediaabstractSearch systems in online social media sites are frequently used to find information about ongoing events and people. For topics with multiple competing perspectives, such as political events or political candidates, bias in the top ranked results significantly shapes public opinion. However, bias does not emerge from an algorithm alone. It is important to distinguish between the bias that arises from the data that serves as the input to the ranking system and the bias that arises from the ranking system itself. In this paper, we propose a framework to quantify these distinct biases and apply this framework to politics-related queries on Twitter. We found that both the input data and the ranking system contribute significantly to produce varying amounts of bias in the search results and in different ways. We discuss the consequences of these biases and possible mechanisms to signal this bias in social media search systems' interfaces. Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh 0001, Krishna P. Gummadi, Karrie Karahalios |
CSCW | 7 |
| 2017 | Discovering dimensions of perceived vocal expression in semi-structured, unscripted oral history accountsabstractWhat do people hear in expressive, unprompted speech? And how can their descriptions be transformed into a representative set of dimensions of vocal expression? This paper presents a methodology for collecting user description of vocal expression, transforms the user descriptions into a set of measurable expressive dimensions, and derives a representative feature set and baseline classifiers across these dimensions. The resulting classifiers recognized the top 13 dimensions over an oral history corpus, with a maximum unweighted recall score of 80.5%. Mary Pietrowicz, Mark Hasegawa-Johnson, Karrie Karahalios |
ICASSP | 3 |
| 2017 | "Be Careful; Things Can Be Worse than They Appear": Understanding Biased Algorithms and Users' Behavior Around Them in Rating Platforms
Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, Kevin Hamilton |
ICWSM | 3 |
| 2017 | Internal and External Visual Cue Preferences for Visualizations in PresentationsabstractAbstract Presenters, such as analysts briefing to an executive committee, often use visualizations to convey information. In these cases, providing clear visual guidance is important to communicate key concepts without confusion. This paper explores visual cues that guide attention to a particular area of a visualization. We developed a visual cue taxonomy distinguishing internal from external cues, designed a web tool based on the taxonomy, and conducted a user study with 24 participants to understand user preferences in choosing visual cues. Participants perceived internal cues (e.g., transparency, brightness, and magnification) as the most useful visual cues and often combined them with other internal or external cues to emphasize areas of focus for their audience. Interviews also revealed that the choice of visual cues depends on not only the chart type, but also the presentation setting, the audience, and the function cues are serving. Considering the complexity of choosing visual cues, we provide design implications for improving the organization, consistency, and integration of visual cues within existing workflows. Ha Kyung Kong, Zhicheng Liu 0001, Karrie Karahalios |
Comput. Graph. Forum | 3 |
| 2017 | Editor's Note/Chairs' WelcomeabstractWelcome to this issue of the Proceedings of the ACM on Human-Computer Interaction, which will focus on contributions from the research community Computer-Supported Cooperative Work and Social Computing (CSCW). This diverse research community explores how different types of social groups affect, and are affected by, information and communication technology. The topics explored by this community can include social media use, crowdsourcing and micro-work, societal effects of computing, and much more. Like many other HCI communities, CSCW approaches these topics with a broad range of scientific techniques, theoretical perspectives and technology platforms. The call for papers for this issue on CSCW attracted 385 submissions, from Asia, Canada, Australia, Europe, Africa, and the United States. After the first round of reviewing, 207 (54%) papers were invited to the Revise and Resubmit phase. The editorial committee worked hard over August 2017 to arrive at final decisions, with a Virtual Committee meeting held to discuss those papers that needed collective deliberation. In the end, 105 papers (27%) were accepted. This issue exists because of the dedicated volunteer effort of 101 senior editors who served as Associate Chairs (ACs), and 885 expert reviewers to ensure high quality and insightful reviews for all papers in both rounds. Reviewers and committee members were kept constant for papers that submitted to both rounds. Senior members of the editorial group also helped shepherd some papers, reflecting the deep commitment of this research community. We are excited by the compelling and thought-provoking work that resulted in this PACMHCI CSCW issue and look forward to equally high quality submissions for the next submission cycle from this research community in the Spring of 2018. For those interested in this area, this group holds their next annual conference November 3-7, 2018 in New York City's Hudson River (Jersey City). That conference will provide many opportunities to share ideas with other researchers and practitioners from institutions around the world. Karrie Karahalios, Geraldine Fitzpatrick, Andrés Monroy-Hernández |
Proc. ACM Hum. Comput. Interact. | 1 |
| 2017 | A Comparative Study of Visualizations with Different Granularities of Behavior for Communicating about AutismabstractThis paper presents a comparative study of two webtools developed to capture engagement via visualization of coordinated communication behavior in children with autism. A clear preference arose for different tasks based on behavior granularity emphasis in the two visualizations. The survey and interview results further revealed the importance of showing behavior patterns, rather than displaying a single behavior without context, in behavior visualization. Based on the results, we propose three granularity-related features to incorporate into behavioral visualizations for communication in clinical settings: separating modalities, coordinating dyadic interactions, and displaying micro-behaviors. Ha Kyung Kong, John Lee 0005, Karrie Karahalios |
Proc. ACM Hum. Comput. Interact. | 3 |
| 2017 | I've Seen "Enough": Incrementally Improving Visualizations to Support Rapid Decision MakingabstractData visualization is an effective mechanism for identifying trends, insights, and anomalies in data. On large datasets, however, generating visualizations can take a long time, delaying the extraction of insights, hampering decision making, and reducing exploration time. One solution is to use online sampling-based schemes to generate visualizations faster while improving the displayed estimates incrementally, eventually converging to the exact visualization computed on the entire data. However, the intermediate visualizations are approximate, and often fluctuate drastically, leading to potentially incorrect decisions. We propose sampling-based incremental visualization algorithms that reveal the "salient" features of the visualization quickly---with a 46× speedup relative to baselines---while minimizing error, thus enabling rapid and error-free decision making. We demonstrate that these algorithms are optimal in terms of sample complexity, in that given the level of interactivity, they generate approximations that take as few samples as possible. We have developed the algorithms in the context of an incremental visualization tool, titled I nc V isage , for trendline and heatmap visualizations. We evaluate the usability of I nc V isage via user studies and demonstrate that users are able to make effective decisions with incrementally improving visualizations, especially compared to vanilla online-sampling based schemes. Sajjadur Rahman, Maryam Aliakbarpour, Hidy Kong, Eric Blais, Karrie Karahalios, Aditya G. Parameswaran, Ronitt Rubinfeld |
Proc. VLDB Endow. | 5 |
| 2016 | EnGaze: Designing Behavior Visualizations with and for Behavioral ScientistsabstractJoint attention is widely recognized as an important developmental milestone for children, and experts consider a lack of joint attention a defining characteristic of autism spectrum disorders (ASDs). While clinicians and researchers agree on the importance of joint attention, their definitions and methods for assessing joint attention vary. In this paper, we present the design process and the evaluation of EnGaze, a visualization-based Web tool for dyadic communicative behavior that highlights commonly discussed features of joint attention. While such visualization styles are not yet the norm in the clinical practices of behavioral and developmental psychology, we argue they should be and find that the introduction of these visual artifacts helped clinicians and researchers conceptualize their personal joint attention rules. Researchers envisioned a number of uses for EnGaze in their personal workflow, including identifying atypical communication patterns and providing a visual record for tracking behavior. The contributions of this paper are 1) an interactive visualization for exploring joint attention, 2) the documentation of an iterative design process for a clinical visualization tool, 3) illustrations of EnGaze use cases with active practitioners in the behavioral sciences, and (4) the discussions surrounding the implications of introducing such visualizations in behavioral science communities. Ha Kyung Kong, John Lee 0005, Karrie Karahalios |
Conference on Designing Interactive Systems | 4 |
| 2016 | Plexlines: Tracking Socio-communicative Behaviors Using Timeline Visualizations
John Lee 0005, Ha Kyung Kong, Karrie Karahalios, Sanny Lin |
AMIA | 3 |
| 2016 | Parental Perceptions, Experiences, and Desires of Music Therapy
Ha Kyung Kong, Karrie Karahalios |
AMIA | 2 |
| 2016 | First I "like" it, then I hide it: Folk Theories of Social FeedsabstractMany online platforms use curation algorithms that are opaque to the user. Recent work suggests that discovering a filtering algorithm's existence in a curated feed influences user experience, but it remains unclear how users reason about the operation of these algorithms. In this qualitative laboratory study, researchers interviewed a diverse, non-probability sample of 40 Facebook users before, during, and after being presented alternative displays of Facebook's News Feed curation algorithm's output. Interviews revealed 10 "folk theories' of automated curation, some quite unexpected. Users who were given a probe into the algorithm's operation via an interface that incorporated "seams,' visible hints disclosing aspects of automation operations, could quickly develop theories. Users made plans that depended on their theories. We conclude that foregrounding these automated processes may increase interface design complexity, but it may also add usability benefits. Motahhare Eslami, Karrie Karahalios, Christian Sandvig, Kristen Vaccaro, Aimee Rickman, Kevin Hamilton, Alex Kirlik |
CHI | 2 |
| 2016 | The Power of Collective Endorsements: Credibility Factors in Medical Crowdfunding CampaignsabstractTraditional medical fundraising charities have been relying on third-party watchdogs and carefully crafting their reputation over time to signal their credibility to potential donors. As medical fundraising campaigns migrate to online platforms in the form of crowdfunding, potential donors can no longer rely on the organization's traditional methods for achieving credibility. Individual fundraisers must establish credibility on their own. Potential donors, therefore, seek new factors to assess the credibility of crowdfunding campaigns. In this paper, we investigate current practices in assessing the credibility of online medical crowdfunding campaigns. We report results from a mixed-methods study that analyzed data from social media and semi-structured interviews. We discovered eleven factors associated with the perceived credibility of medical crowdfunding. Of these, three communicative/emotional factors were unique to medical crowdfunding. We also found a distinctive validation practice, the collective endorsement. Close-connections' online presence and external online communities come together to form this collective endorsement in online medical fundraising campaigns. We conclude by describing how fundraisers can leverage collective endorsements to improve their campaigns' perceived credibility. Jennifer G. Kim, Ha Kyung Kong, Karrie Karahalios, Wai-Tat Fu, Hwajung Hong |
CHI | 3 |
| 2016 | A Comparative Study of Query-biased and Non-redundant Snippets for Structured Search on Mobile DevicesabstractTo investigate what kind of snippets are better suited for structured search on mobile devices, we built an experimental mobile search application and conducted a task-oriented interactive user study with 36 participants. Four different versions of a search engine result page (SERP) were compared by varying the snippet type (query-biased vs. non-redundant) and the snippet length (two vs. four lines per result). We adopted a within-subjects experiment design and made each participant do four realistic search tasks using different versions of the application. During the study sessions, we collected search logs, "think-aloud" comments, and post-task surveys. Each session was finalized with an interview. We found that with non-redundant snippets the participants were able to complete the tasks faster and find more relevant results. Most participants preferred non-redundant snippets and wanted to see more information about each result on the SERP for any snippet type. Yet, the participants felt that the version with query-biased snippets was easier to use. We conclude with a set of practical design recommendations. Nikita Spirin, Alexander Kotov 0001, Karrie Karahalios, Vassil Mladenov, Pavel A. Izhutov |
CIKM | 3 |
| 2016 | The Elements of Fashion StyleabstractThe outfits people wear contain latent fashion concepts capturing styles, seasons, events, and environments. Fashion theorists have proposed that these concepts are shaped by design elements such as color, material, and silhouette. A dress may be "bohemian" because of its pattern, material, trim, or some combination of them: it is not always clear how low-level elements translate to high-level styles. In this paper, we use polylingual topic modeling to learn latent fashion concepts jointly in two languages capturing these elements and styles. Using this latent topic formation we can translate between these two languages through topic space, exposing the elements of fashion style. We train the polylingual topic model (PLTM) on a set of more than half a million outfits collected from Polyvore, a popular fashion-based social net- work. We present novel, data-driven fashion applications that allow users to express their needs in natural language just as they would to a real stylist and produce tailored item recommendations for these style needs. Kristen Vaccaro, Sunaya Shivakumar, Ziqiao Ding, Karrie Karahalios, Ranjitha Kumar |
UIST | 4 |
| 2016 | MapWatch: Detecting and Monitoring International Border Personalization on Online MapsabstractMaps have long played a crucial role in enabling people to conceptualize and navigate the world around them. However, maps also encode the world-views of their creators. Disputed international borders are one example of this: governments may mandate that cartographers produce maps that conform to their view of a territorial dispute. Today, online maps maintained by private corporations have become the norm. However, these new maps are still subject to old debates. Companies like Google and Bing resolve these disputes by localizing their maps to meet government requirements and user preferences, i.e., users in different locations are shown maps with different international boundaries. We argue that this non-transparent personalization of maps may exacerbate nationalistic disputes by promoting divergent views of geopolitical realities. Gary Soeller, Karrie Karahalios, Christian Sandvig, Christo Wilson |
WWW | 2 |
| 2016 | Effortless Data Exploration with zenvisage: An Expressive and Interactive Visual Analytics SystemabstractData visualization is by far the most commonly used mechanism to explore and extract insights from datasets, especially by novice data scientists. And yet, current visual analytics tools are rather limited in their ability to operate on collections of visualizations---by composing, filtering, comparing, and sorting them---to find those that depict desired trends or patterns. The process of visual data exploration remains a tedious process of trial-and-error. We propose zenvisage, a visual analytics platform for effortlessly finding desired visual patterns from large datasets. We introduce zenvisage's general purpose visual exploration language, ZQL ("zee-quel") for specifying the desired visual patterns, drawing from use-cases in a variety of domains, including biology, mechanical engineering, climate science, and commerce. We formalize the expressiveness of ZQL via a visual exploration algebra---an algebra on collections of visualizations---and demonstrate that ZQL is as expressive as that algebra. zenvisage exposes an interactive front-end that supports the issuing of ZQL queries, and also supports interactions that are "short-cuts" to certain commonly used ZQL queries. To execute these queries, zenvisage uses a novel ZQL graph-based query optimizer that leverages a suite of optimizations tailored to the goal of processing collections of visualizations in certain pre-defined ways. Lastly, a user survey and study demonstrates that data scientists are able to effectively use zenvisage to eliminate error-prone and tedious exploration and directly identify desired visualizations. Tarique Siddiqui, Albert Kim, John Lee 0005, Karrie Karahalios, Aditya G. Parameswaran |
Proc. VLDB Endow. | 4 |
| 2015 | "I always assumed that I wasn't really that close to [her]": Reasoning about Invisible Algorithms in News FeedsabstractOur daily digital life is full of algorithmically selected content such as social media feeds, recommendations and personalized search results. These algorithms have great power to shape users' experiences, yet users are often unaware of their presence. Whether it is useful to give users insight into these algorithms' existence or functionality and how such insight might affect their experience are open questions. To address them, we conducted a user study with 40 Facebook users to examine their perceptions of the Facebook News Feed curation algorithm. Surprisingly, more than half of the participants (62.5%) were not aware of the News Feed curation algorithm's existence at all. Initial reactions for these previously unaware participants were surprise and anger. We developed a system, FeedVis, to reveal the difference between the algorithmically curated and an unadulterated News Feed to users, and used it to study how users perceive this difference. Participants were most upset when close friends and family were not shown in their feeds. We also found participants often attributed missing stories to their friends' decisions to exclude them rather than to Facebook News Feed algorithm. By the end of the study, however, participants were mostly satisfied with the content on their feeds. Following up with participants two to six months after the study, we found that for most, satisfaction levels remained similar before and after becoming aware of the algorithm's presence, however, algorithmic awareness led to more active engagement with Facebook and bolstered overall feelings of control on the site. Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, Christian Sandvig |
CHI | 6 |
| 2015 | Acoustic correlates for perceived effort levels in expressive speechabstractActors and other vocal performers vary their speech across the continuum of vocal effort to express ideas, emphasize thoughts, communicate emotions, and create drama. They are experts at vocal expression. To analyze this range of expression across effort levels, we curated a corpus of professional actors ’ Hamlet soliloquy performances and present an acoustic feature set and classification model suitable for tracking actors ’ expressive speech from extreme to extreme – from whispered, to breathy, through modal, to resonant speech. Mary Pietrowicz, Mark Hasegawa-Johnson, Karrie Karahalios |
INTERSPEECH | 3 |
| 2015 | DataTone: Managing Ambiguity in Natural Language Interfaces for Data VisualizationabstractAnswering questions with data is a difficult and time-consuming process. Visual dashboards and templates make it easy to get started, but asking more sophisticated questions often requires learning a tool designed for expert analysts. Natural language interaction allows users to ask questions directly in complex programs without having to learn how to use an interface. However, natural language is often ambiguous. In this work we propose a mixed-initiative approach to managing ambiguity in natural language interfaces for data visualization. We model ambiguity throughout the process of turning a natural language query into a visualization and use algorithmic disambiguation coupled with interactive ambiguity widgets. These widgets allow the user to resolve ambiguities by surfacing system decisions at the point where the ambiguity matters. Corrections are stored as constraints and influence subsequent queries. We have implemented these ideas in a system, DataTone. In a comparative study, we find that DataTone is easy to learn and lets users ask questions without worrying about syntax and proper question form. Mira Dontcheva, Eytan Adar, Zhicheng Liu 0001, Karrie Karahalios |
UIST | 5 |
| 2014 | People Search within an Online Social Network: Large Scale Analysis of Facebook Graph Search Query LogsabstractPopular online social networks (OSN) generate hundreds of terabytes of new data per day and connect millions of users. To help users cope with the immense scale and influx of new information, OSNs provide a search functionality. However, most of the search engines in OSNs today only support keyword queries and provide basic faceted search capabilities overlooking serendipitous network exploration and search for relationships between OSN entities. This results in siloed information and a limited search space. In 2013 Facebook introduced its innovative Graph Search product with the goal to take the OSN search experience to the next level and facilitate exploration of the Facebook Graph beyond the first degree. In this paper we explore people search on Facebook by analyzing an anonymized social graph, anonymized user profiles, and large scale anonymized query logs generated by users of Facebook Graph Search. We uncover numerous insights about people search across several demographics. We find that named entity and structured queries complement each other across one's duration on Facebook, that females search for people proportionately more than males, and that users submit more queries as they gain more friends. We introduce the concept of a lift predicate and highlight how a graph distance varies with the search goal. Based on these insights, we present a set of design implications to guide the research and development of the OSN search in the future. Nikita Spirin, Junfeng He, Mike Develin, Karrie Karahalios, Maxime Boucher |
CIKM | 4 |
| 2014 | The role of network distance in linkedin people searchabstractLinkedIn is the world's largest professional network, with over 300 million members. One of the primary activities on the site is people search, for which LinkedIn members are both the users and the corpus. This paper presents insights about people search behavior on LinkedIn, based on a log analysis and a user study. In particular, it examines the role that network distance plays in name searches and non-name searches. For name searches, users primarily click on only one of the results, and closer network distance leads to higher click-through rates. In contrast, for non-name searches, users are more likely to click on multiple results that are not in their existing connections, but with whom they have shared connections. The results show that, while network distance contributes significantly to LinkedIn search engagement in general, its role varies dramatically depending on the type of search query. Shih-Wen Huang, Daniel Tunkelang, Karrie Karahalios |
SIGIR | 3 |
| 2013 | ACES: a cross-discipline platform and method for communication and language researchabstractWhile conducting research focused on individuals with impairments is vitally important, such experiments often have high costs (time and money), and researchers may be limited in the instructions they can give, or participant feedback they can gather (due to the impairment). We present how an impairment emulation system (ACES) can be used by researchers in the behavioral sciences. By repurposing this new technology within the context of a "traditional" psychology experiment, we were able to analyze impaired linguistic and communication in a manner that was not possible without a system such as ACES. Our experiment on 96 participants provided strong support for a theory in the aphasia psychology community, and uncovered new understandings of how people communicate when one interlocutor's speech is distorted with aphasia. These findings illustrate a new direction of HCI research that directly helps researchers in Psychology, Communication, and Speech and Hearing Science. Joshua M. Hailpern, Marina Danilevsky, Andrew Harris, Sunah Suh, Reed LaBotz, Karrie Karahalios |
CSCW | 6 |
| 2013 | CrowdBand: An Automated Crowdsourcing Sound Composition SystemabstractCrowdBand, a sound composition system, demonstrates how a crowd can create works that meet requested criteria and fulfill the aesthetic character given by keyword description and examples. CrowdBand allows flexibility in music composition in terms of duration of the music, completion time and cost of music composition by giving the requestor two modes - thrifty and normal. CrowdBand’s workflow divides the composition task into three sections: requesting fundamental sounds, assembling sounds into compositions, and evaluating the results. Based on the crowd workers’ responses, we conclude that crowdsourced workers who are non-musicians can design sound and create novel sound compositions through CrowdBand. We also conclude that CrowdBand gives the musically-untrained crowd workers the ability to use common compositional techniques, such as sound layering, vertical stacking of sounds to create harmonic effects, related melodic lines (contrapuntal techniques), and transitions between aesthetic notions, or sound themes. Finally, we show improved, faster results with successive simplification and examples. Mary Pietrowicz, Danish Chopra, Amin Sadeghi, Puneet Chandra, Brian P. Bailey, Karrie Karahalios |
HCOMP | 6 |
| 2013 | Likeness and Dealbreakers: Interpreting Interpersonal Compatibility from Online Music Profiles
Mo Kudeki, Karrie Karahalios |
INTERACT (3) | 2 |
| 2012 | Designing visualizations to facilitate multisyllabic speech with children with autism and speech delaysabstractThe ability of children to combine syllables represents an important developmental milestone. This ability is often delayed or impaired in a variety of clinical groups, including children with autism spectrum disorders (ASD) and speech delays (SPD). Prior work has demonstrated successful use of computer-based voice visualizations to facilitate speech production and vocalization in children with and without ASD/SPD. While prior work has focused on increasing frequency of speech-like vocalizations or accuracy of speech sound production, we believe that there is a potential new direction of research: exploration of real-time visualizations to shape multisyllabic speech. Over two years we developed VocSyl, a real-time voice visualization system. Rather than building visualizations based on what adult clinicians and software designers may think is needed, we designed VocSyl using the Task Centered User Interface Design (TCUID) methodology throughout the design process. Children with ASD and SPD, targeted users of the software, were directly involved in the development process, allowing us to focus on what these children demonstrate they require. This paper presents the results of our TCUID design cycle of VocSyl, as well as design guidelines for future work with children with ASD and SPD. Joshua M. Hailpern, Andrew Harris, Reed LaBotz, Brianna Birman, Karrie Karahalios |
Conference on Designing Interactive Systems | 5 |
| 2012 | Discovery-based games for learning softwareabstractWe propose using discovery-based learning games to teach people how to use complex software. Specifically, we developed Jigsaw, a learning game that asks players to solve virtual jigsaw puzzles using tools in Adobe Photoshop. We conducted an eleven-person lab study of the prototype, and found the game to be an effective learning medium that can complement demonstration-based tutorials. Not only did the participants learn about new tools and techniques while actively solving the puzzles in Jigsaw, but they also recalled techniques that they had learned previously but had forgotten. Mira Dontcheva, Diana M. Joseph, Karrie Karahalios, Mark W. Newman, Mark S. Ackerman |
CHI | 4 |
| 2012 | Theme issue on autism and technology
Gillian R. Hayes, Karrie Karahalios |
Pers. Ubiquitous Comput. | 2 |
| 2011 | ACES: aphasia emulation, realism, and the turing testabstractTo an outsider it may appear as though an individual with aphasia has poor cognitive function. However, the problem resides in the individual's receptive and expressive language, and not in their ability to think. This misperception, paired with a lack of empathy, can have a direct impact on quality of life and medical care. Hailpern's 2011 paper on ACES demonstrated a novel system that enabled users (e.g., caregivers, therapists, family) to experience first hand the communication-distorting effects of aphasia. While their paper illustrated the impact of ACES on empathy, it did not validate the underlying distortion emulation. This paper provides a validation of ACES' distortions through a Turing Test experiment with participants from the Speech and Hearing Science community. It illustrates that text samples generated with ACES distortions are generally not distinguishable from text samples originating from individuals with aphasia. This paper explores ACES distortions through a `How Human' is it test, in which participants explicitly rate how human- or computer-like distortions appear to be. Joshua M. Hailpern, Marina Danilevsky, Karrie Karahalios |
ASSETS | 3 |
| 2011 | ACES: promoting empathy towards aphasia through language distortion emulation softwareabstractIndividuals with aphasia, an acquired communication disorder, constantly struggle against a world that does not understand them. This lack of empathy and understanding negatively impacts their quality of life. While aphasic individuals may appear to have lost cognitive functioning, their impairment relates to receptive and expressive language, not to thinking processes. We introduce a novel system and model, Aphasia Characteristics Emulation Software (ACES), enabling users (e.g., caregivers, speech therapists and family) to experience, firsthand, the communication-distorting effects of aphasia. By allowing neurologically typical individuals to "walk in another's shoes," we aim to increase patience, awareness and understanding. ACES was grounded in the communication science and psychological literature, and informed by an initial pilot study. Results from an evaluation of 64 participants indicate that ACES provides a rich experience that increases understanding and empathy for aphasia. Joshua M. Hailpern, Marina Danilevsky, Andrew Harris, Karrie Karahalios, Gary S. Dell, Julie Hengst |
CHI | 4 |
| 2011 | YouPivot: improving recall with contextual searchabstractAccording to cognitive science literature, human memory is predicated on contextual cues (e.g., room, music) in the environment. During recall tasks, we associate information/activities/objects with contextual cues. However, computer systems do not leverage our natural process of using contextual cues to facilitate recall. We present a new interaction technique, Pivoting, that allows users to search for contextually related activities and find a target piece of information (often not semantically related). A sample motivation for contextual search would be, 'what was that website I was looking at when Yesterday by The Beatles was last playing?' Our interaction technique is grounded in the cognitive science literature, and is demonstrated in our system YouPivot. In addition, we present a new personal annotation method, called TimeMarks, to further support contextual recall and the pivoting process. In a pilot study, participants were quicker to identify websites, and preferred using YouPivot, compared to current tools. YouPivot demonstrates how principles of human memory can be applied to enhance the search of digital information. Joshua M. Hailpern, Nicholas Jitkoff, Andrew Warr, Karrie Karahalios, Robert Sesek, Nik Shkrob |
CHI | 4 |
| 2011 | Encouraging Initiative in the Classroom with Anonymous Feedback
Tony Bergstrom, Andrew Harris, Karrie Karahalios |
INTERACT (1) | 3 |
| 2010 | The CLOTHO project: predicting application utilityabstractWhen using the computer, each user has some notion that "these applications are important" at a given point in time. We term this subset of applications that the user values as high-utility applications. Identifying high-utility applications is a critical first step for Task Analysis, Time Management/Workflow analysis, and Interruption research. However, existing techniques fail to identify at least 57% of these applications. Our work directly associates measurable computer interaction (CPU consumption, window area, etc.) with the user's perceived application utility without identifying task. In this paper, we present an objective utility function that accurately predicts the user's subjective impressions of application importance, improving existing techniques by 53%. This model of computer usage is based upon 321 hours of real-world data from 22 users (both professional and academic). Unlike existing approaches, our model is not limited by a pre-existing set of applications or known tasks. We conclude with a discussion of the direct implications for improving accuracy in the fields of interruptions, task analysis, and time management systems. Joshua M. Hailpern, Nicholas Jitkoff, Joseph Subida, Karrie Karahalios |
Conference on Designing Interactive Systems | 4 |
| 2010 | Walking in another's shoes: aphasia emulation softwareabstractThe impact of living in a world that does not understand your impairment can be frustrating and a daunting task. Consider how an individual would feel if their family, friends, or doctors did not understand or were not even empathetic to daily struggles brought on by an acquired language disorder such as Aphasia. This work seeks to shed new light on aphasia by creating an instant message client which emulates the effects of aphasia. The goal of this new system is to raise awareness, teach, and increase empathy for caregivers, family members, and doctors/therapists who work with this population on a daily basis. Joshua M. Hailpern, Marina Danilevsky, Karrie Karahalios |
ASSETS | 3 |
| 2010 | Vocsyl: visualizing syllable production for children with ASD and speech delaysabstractCommunication disorders occur across the lifespan and encompass a wide range of conditions that interfere with individuals' abilities to hear (e.g., hearing loss), speak (e.g., voice disorders; motor speech disorders), and/or use language (e.g., specific language impairment; aphasia) to meet their communication needs. Such disorders often compromise the social, recreational, emotional, educational, and vocational aspects of an individual's life. This research examines the development and implementation of new software that facilitates multi-syllabic speech production in children with autism and speech delays. The VocSyl software package utilizes a suite of audio visualizations that represent a myriad of audio features in abstract representations. The goal of these visualizations is to provide children with language impairments a new persistent modality in which to experience and practice speech-language skills. Joshua M. Hailpern, Karrie Karahalios, Laura DeThorne, James Halle |
ASSETS | 2 |
| 2010 | Understanding deja reviewersabstractPeople who review products on the web invest considerable time and energy in what they write. So why would someone write a review that restates earlier reviews? Our work looks to answer this question. In this paper, we present a mixed-method study of deja reviewers, latecomers who echo what other people said. We analyze nearly 100,000 Amazon.com reviews for signs of repetition and find that roughly 10-15% of reviews substantially resemble previous ones. Using these algorithmically-identified reviews as centerpieces for discussion, we interviewed reviewers to understand their motives. An overwhelming number of reviews partially explains deja reviews, but deeper factors revolving around an individual's status in the community are also at work. The paper concludes by introducing a new idea inspired by our findings: a self-aware community that nudges members toward community-wide goals. Eric Gilbert, Karrie Karahalios |
CSCW | 2 |
| 2010 | Widespread Worry and the Stock Market
Eric Gilbert, Karrie Karahalios |
ICWSM | 2 |
| 2009 | Talking points: the differential impact of real-time computer generated audio/visual feedback on speech-like & non-speech-like vocalizations in low functioning children with ASDabstractReal-time computer feedback systems (CFS) have been shown to impact the communication of neurologically typical individuals. Promising new research appears to suggest the same for the vocalization of low functioning children with Autistic Spectrum Disorder (ASD). The distinction between speech-like versus non-speech-like vocalizations has rarely, if ever, been addressed in the HCI community. This distinction is critical as we strive to most effectively and efficiently facilitate speech development in children with ASD, while simultaneously helping decrease vocalizations that do not facilitate positive social interactions. This paper provided an extension of Hailpern et al. in 2009 by examining the influence of a computerized feedback system on both the speech-like and non-speech-like vocalizations of five nonverbal children with ASD. Results were largely positive, in that some form of computerized feedback was able to differentially facilitate speech-like vocalizations relative to nonspeech-like vocalizations in 4 of the 5 children. The main contribution of this work is in highlighting the importance of distinguishing between speech-like versus nonspeech-like vocalizations in the design of feedback systems focused on facilitating speech in similar populations. Joshua M. Hailpern, Karrie Karahalios, Laura DeThorne, James Halle |
ASSETS | 2 |
| 2009 | Conversation clusters: grouping conversation topics through human-computer dialogabstractConversation Clusters explores the use of visualization to highlight salient moments of live conversation while archiving a meeting. Cheaper storage and easy access to recording devices allows extensive archival. However, as the size of the archive grows, retrieving the desired moments becomes increasingly difficult. We approach this problem from a socio-technical perspective and utilize human intuition aided by computer memory. We present computationally detected topics of conversation as visual summaries of discussion and as reference points into the archive. To further bootstrap the system, humans can participate in a dialog with the visualization of the clustering process and shape the development of clustering models. Tony Bergstrom, Karrie Karahalios |
CHI | 2 |
| 2009 | Predicting tie strength with social mediaabstractSocial media treats all users the same: trusted friend or total stranger, with little or nothing in between. In reality, relationships fall everywhere along this spectrum, a topic social science has investigated for decades under the theme of tie strength. Our work bridges this gap between theory and practice. In this paper, we present a predictive model that maps social media data to tie strength. The model builds on a dataset of over 2,000 social media ties and performs quite well, distinguishing between strong and weak ties with over 85% accuracy. We complement these quantitative findings with interviews that unpack the relationships we could not predict. The paper concludes by illustrating how modeling tie strength can improve social media design elements, including privacy controls, message routing, friend introductions and information prioritization. Eric Gilbert, Karrie Karahalios |
CHI | 2 |
| 2009 | Creating a spoken impact: encouraging vocalization through audio visual feedback in children with ASDabstractOne hallmark difficulty of children with Autism Spectrum Disorder (ASD) centers on communication and speech. Research into computer visualizations of voice has been shown to influence conversational patterns and allow users to reflect upon their speech. In this paper, we present the Spoken Impact Project (SIP), an effort to examine the effect of audio and visual feedback on vocalizations in low-functioning children with ASD by providing them with additional means of understanding and exploring their voice. This research spans over 12 months, including the creation of multiple software packages and detailed analysis of more than 20 hours of experimental video. SIP demonstrates the potential of computer generated audio and visual feedback to encourage vocalizations of children with ASD. Joshua M. Hailpern, Karrie Karahalios, James Halle |
CHI | 2 |
| 2009 | Vote and Be Heard: Adding Back-Channel Signals to Social Mirrors
Tony Bergstrom, Karrie Karahalios |
INTERACT (1) | 2 |
| 2009 | Using Social Visualization to Motivate Social ProductionabstractIn this paper we argue that social visualization can motivate contributors to social production projects, such as Wikipedia and open source development. As evidence, we present CodeSaw, a social visualization of open source software development that we studied with real open source communities. CodeSaw mines open source archives to visualize group dynamics that currently lie buried in textual databases. Furthermore, CodeSaw becomes an active social space itself by supporting comments directly inside the visualization. To demonstrate CodeSaw, we apply it to a popular open source project, showing how the visualization reveals group dynamics and individual roles. The paper concludes by presenting evidence that CodeSaw, and social visualization more generally, can motivate contributors to social production projects if the visualization leaves the laboratory and makes it to the community visualized. Eric Gilbert, Karrie Karahalios |
IEEE Trans. Multim. | 2 |
| 2008 | A3: a coding guideline for HCI+autism research using video annotationabstractDue to the profile of strengths and weaknesses indicative of autism spectrum disorders (ASD), technology may play a key role in ameliorating communication difficulties with this population. This paper documents coding guidelines established through cross-disciplinary work focused on facilitating communication development in children with ASD using computerized feedback. The guidelines, referred to as A3 (pronounced A-Cubed) or Annotation for ASD Analysis, define and operationalize a set of dependent variables coded via video annotation. Inter-rater reliability data are also presented from a study currently in-progress, as well as related discussion to help guide future work in this area. The design of the A3 methodology is well-suited for the examination and evaluation of the behavior of low-functioning subjects with ASD who interact with technology. Joshua M. Hailpern, Karrie Karahalios, James Halle, Laura DeThorne, Mary-Kelsey Coletto |
ASSETS | 2 |
| 2008 | VCode and VData: illustrating a new framework for supporting the video annotation workflowabstractDigital tools for annotation of video have the promise to provide immense value to researchers in disciplines ranging from psychology to ethnography to computer science. With traditional methods for annotation being cumbersome, time-consuming, and frustrating, technological solutions are situated to aid in video annotation by increasing reliability, repeatability, and workflow optimizations. Three notable limitations of existing video annotation tools are lack of support for the annotation workflow, poor representation of data on a timeline, and poor interaction techniques with video, data, and annotations. This paper details a set of design requirements intended to enhance video annotation. Our framework is grounded in existing literature, interviews with experienced coders, and ongoing discussions with researchers in multiple disciplines. Our model is demonstrated in a new system called VCode and VData. The benefit of our system is that is directly addresses the workflow and needs of both researchers and video coders. Joey Hagedorn, Joshua M. Hailpern, Karrie Karahalios |
AVI | 3 |
| 2008 | The network in the garden: an empirical analysis of social media in rural lifeabstractHistory repeatedly demonstrates that rural communities have unique technological needs. Yet, we know little about how rural communities use modern technologies, so we lack knowledge on how to design for them. To address this gap, our empirical paper investigates behavioral differences between more than 3,000 rural and urban social media users. Using a dataset collected from a broadly popular social network site, we analyze users' profiles, 340,000 online friendships and 200,000 interpersonal messages. Using social capital theory, we predict differences between rural and urban users and find strong evidence supporting our hypotheses. Namely, rural people articulate far fewer friends online, and those friends live much closer to home. Our results also indicate that the groups have substantially different gender distributions and use privacy features differently. We conclude by discussing design implications drawn from our findings; most importantly, designers should reconsider the binary friend-or-not model to allow for incremental trust-building. Eric Gilbert, Karrie Karahalios, Christian Sandvig |
CHI | 2 |
| 2007 | Isochords: visualizing structure in musicabstractIsochords is a visualization of music that aids in the classification of musical structure. The Isochords visualization highlights the consonant intervals between notes and common chords in music. It conveys information about interval quality, chord quality, and the chord progression synchronously during playback of digital music. Isochords offers listeners a means to grasp the underlying structure of music that, without extensive training, would otherwise remain unobserved or unnoticed. In this paper we present the theory of the Isochords structure, the visualization, and comments from novice and experienced users. Tony Bergstrom, Karrie Karahalios, John C. Hart |
Graphics Interface | 2 |
| 2007 | Seeing More: Visualizing Audio Cues
Tony Bergstrom, Karrie Karahalios |
INTERACT (2) | 2 |
| 2007 | CodeSaw: A Social Visualization of Distributed Software Development
Eric Gilbert, Karrie Karahalios |
INTERACT (2) | 2 |
| 2007 | Tagscape: Navigating the Tag Landscape
Lauren Haynes, Aylin Selcukoglu, Sunah Suh, Karrie Karahalios |
INTERACT (2) | 4 |
| 2006 | I-Living: An Open System Architecture for Assisted LivingabstractAdvances in networking, sensors, and embedded devices have made it feasible to monitor and provide medical and other assistance to people in their homes. Aging populations will benefit from reduced costs and improved healthcare through assisted living based on these technologies. However, these systems challenge current state-of-the-art techniques for usability, reliability, and security. This is a particular challenge for open and extensible systems that combine software and hardware from many vendors and provide information to diverse clinicians. In this paper we present the I-Living architecture for assisted living that allows independent parties work together in a dependable, secure, and low-cost fashion with predictable properties. Our approach is based on an Assisted Living Service Provider (ALSP) who provides a server that collects and maintains encrypted assisted persons (APs)' records. Our ALSP can be a third party distinct from APs, communication providers, and clinicians; or it can be part of an ISP, hospital or similar enterprise. We have explored the architecture by developing a collection of applications and implementing them in a prototype system. Our system shows the feasibility and opportunity of an open approach to assisted living systems. Qixin Wang 0001, Wook Shin, Xue (Steve) Liu, Zheng Zeng 0001, Cham Oh, Bedoor K. AlShebli, Marco Caccamo, Carl A. Gunter, Elsa L. Gunter, Jennifer C. Hou, Karrie Karahalios, Lui Sha |
SMC | 11 |
| 2005 | ChatAmp: Talking with Music and Text
M. Ian Graham, Karrie Karahalios |
INTERACT | 2 |
| 2005 | TextTone: Expressing Emotion Through Text
Ankur Kalra, Karrie Karahalios |
INTERACT | 2 |
| 2005 | Telelogs: a social communication space for urban environmentsabstractThis paper presents a novel idea for a system known as Telelogs. Utilizing the ubiquity of mobile devices, Telelogs functions as a service by which individuals in an urban environment can establish a better sense of community awareness. In addition, this system could serve as a medium through which these individuals can communicate their thoughts and ideas with others within their environment. This would result in a better sense of community solidarity. Telelogs is targeted towards those persons in society that come across one another on a consistent basis but rather than establishing a direct line of communication, they maintain a relationship that could be characterized as one of courteous detachment. In other words, these individuals are known as familiar strangers. There is strong potential for this relationship to be augmented with a mediated communication space. Telelogs would act as a space through which individuals learn more about themselves while reciprocally gaining a better understanding of those persons present in their environment. Telelogs transfers the essence of the blog into an audio form as an extension for mobile devices. A first prototype of Telelogs is presented and accompanied by feedback after a demonstration. In addition, details of a first cell phone implementation follow. Karrie Karahalios |
Mobile HCI | 2 |
| 2004 | Telemurals: linking remote spaces with social catalystsabstractTelemurals is an abstract audio-video installation that seeks to initiate and sustain interaction between and within two remote spaces. Our goal is to improve the social aspects of casual mediated communications by incorporating events into the design of the communication medium that encourage people to engage in interaction when they otherwise would not. We call these events social catalysts, for they encourage people to initiate and sustain interaction. In this paper we discuss the design process and goals of our first Telemurals link between two public spaces, the building of Telemurals, and an ethnographic study describing how the system affected interaction between and within these two spaces based on the theories discussed in this paper. Karrie Karahalios, Judith S. Donath |
CHI | 1 |