Online health communities (OHCs) offer the promise of connecting with supportive peers. Forming these connections first requires finding relevant peers—a process that can be time-consuming. Peer recommendation systems are a computational approach to make finding peers easier during a health journey. By encouraging OHC users to alter their online social networks, peer recommendations could increase available support. But these benefits are hypothetical and based on mixed, observational evidence. To experimentally evaluate the effect of peer recommendations, we conceptualize these systems as health interventions designed to increase specific beneficial connection behaviors. In this paper, we designed a peer recommendation intervention to increase two behaviors: reading about peer experiences and interacting with peers. We conducted an initial feasibility assessment of this intervention by conducting a 12-week field study in which 79 users of CaringBridge.org received weekly peer recommendations via email. Our results support the usefulness and demand for peer recommendation and suggest benefits to evaluating larger peer recommendation interventions. Our contributions include practical guidance on the development and evaluation of peer recommendation interventions for OHCs.

This paper presents reports on a series of experiments with a novel dataset evaluating how well Large Language Models (LLMs) can mark (i.e. grade) open text responses to short answer questions, Specifically, we explore how well different combinations of GPT version and prompt engineering strategies performed at marking real student answers to short answer across different domain areas (Science and History) and grade-levels (spanning ages 5-16) using a new, never-used-before dataset from Carousel, a quizzing platform. We found that GPT-4, with basic few-shot prompting performed well (Kappa, 0.70) and, importantly, very close to human-level performance (0.75). This research builds on prior findings that GPT-4 could reliably score short answer reading comprehension questions at a performance-level very close to that of expert human raters. The proximity to human-level performance, across a variety of subjects and grade levels suggests that LLMs could be a valuable tool for supporting low-stakes formative assessment tasks in K-12 education and has important implications for real-world education delivery.

Following Facebook's introduction of the "Like" in 2009, CaringBridge (a nonprofit health journaling platform) implemented a "Heart" symbol as a single-click reaction affordance in 2012. In 2016, Facebook expanded its Like into a set of emotion-based reactions. In 2021, CaringBridge likewise added three new reactions: "Prayer", "Happy", and "Sad." Through user surveys (N=808) and interviews (N=13), we evaluated this product launch. Unlike Likes on mainstream social media, CaringBridge's single-click Heart was consistently interpreted as a simple, meaningful expression of acknowledgement and support. Although most users accepted the new reactions, the product launch transformed user perceptions of the feature and ignited major disagreement regarding the meanings and functions of reactions in the high stakes context of health crises. Some users found the new reactions to be useful, convenient, and reducing of caregiver burden; others felt they cause emotional harms by stripping communication of meaningful expression and authentic care. Overall, these results surface tensions for small social media platforms that need to survive amidst giants, as well as highlighting crucial trade-offs between the cognitive effort, meaningfulness, and efficiency of different forms of Computer-Mediated Communication (CMC). Our work provides three contributions to support researchers and designers in navigating these tensions: (1) empirical knowledge of how users perceived the reactions launch on CaringBridge; (2) design implications for improving health-focused CMC; and (3) concrete questions to guide future research into reactions and health-focused CMC.

For many automated classification tasks, collecting labeled data is the key barrier to training a useful supervised model. Interfaces for interactive labeling tighten the loop of labeled data collection and model development, enabling a subject-matter expert to quickly establish the feasibility of a classifier to address a problem of interest. These interactive machine learning (IML) interfaces iteratively sample unlabeled data for annotation, train a new model, and display feedback on the model’s estimated performance. Different sampling strategies affect both the rate at which the model improves and the bias of performance estimates. We compare the performance of three sampling strategies in the “early-stage” of label collection, starting from zero labeled data. By simulating a user’s interactions with an IML labeling interface, we demonstrate a trade-off between improving a text classifier’s performance and computing unbiased estimates of that performance. We show that supplementing early-stage sampling with user-guided text search can effectively “seed” a classifier with positive documents without compromising generalization performance—particularly for imbalanced tasks where positive documents are rare. We argue for the benefits of incorporating search alongside active learning in IML interfaces and identify design trade-offs around the use of non-random sampling strategies.

Online health communities offer the promise of support benefits to users, in particular because these communities enable users to find peers with similar experiences. Building mutually supportive connections between peers is a key motivation for using online health communities. However, a user's role in a community may influence the formation of peer connections. In this work, we study patterns of peer connections between two structural health roles: patient and non-professional caregiver. We examine user behavior in an online health community---CaringBridge.org---where finding peers is not explicitly supported. This context lets us use social network analysis methods to explore the growth of such connections in the wild and identify users' peer communication preferences. We investigated how connections between peers were initiated, finding that initiations are more likely between two authors who have the same role and who are close within the broader communication network. Relationships---patterns of repeated interactions---are also more likely to form and be more interactive when authors have the same role. Our results have implications for the design of systems supporting peer communication, e.g. peer-to-peer recommendation systems.

Instrumental support is critical for patients and family caregivers facing life-threatening illnesses, injuries, or chronic conditions (e.g., cancer). We partner with CaringBridge.org—a prominent online health community for journaling about health crises—to conduct a study of instrumental support in the following two phases: a content analysis of 641 journal updates; and a survey of 991 users. Quantitative results show that: (1) patients and family caregivers prefer to receive different types of support than their care networks prefer to provide; (2) people generally have more trust in their closest social connections than acquaintances or businesses to provide instrumental support; and (3) users rate “prayer support” as the most important support category to them. Building on these results, we discuss design implications to accommodate divergent preferences and to expand instrumental support networks. We also discuss the need for future work to empower family caregivers and to support spirituality, an understudied topic in HCI.

Emoji are popular in digital communication, but they are rendered differently on different viewing platforms (e.g., iOS, Android). It is unknown how many people are aware that emoji have multiple renderings, or whether they would change their emoji-bearing messages if they could see how these messages render on recipients' devices. We developed software to expose the multi-rendering nature of emoji and explored whether this increased visibility would affect how people communicate with emoji. Through a survey of 710 Twitter users who recently posted an emoji-bearing tweet, we found that at least 25% of respondents were unaware that the emoji they posted could appear differently to their followers. Additionally, after being shown how one of their tweets rendered across platforms, 20% of respondents reported that they would have edited or not sent the tweet. These statistics reflect millions of potentially regretful tweets shared per day because people cannot see emoji rendering differences across platforms. Our results motivate the development of tools that increase the visibility of emoji rendering differences across platforms, and we contribute our cross-platform emoji rendering software to facilitate this effort.