The SIGCHI and Social Computing research communities have been at the forefront of online safety efforts for youth, ranging from understanding the serious risks youth face online to developing evidence-based interventions for risk protection. Yet, to bring these efforts to bear, we must partner with practitioners, such as industry stakeholders who know how to bring such technologies to market, and youth service providers who work directly with youth. Therefore, we interviewed 33 stakeholders in the space of youth online safety, including industry professionals (n=12), youth service providers (n=11), and researchers (n=10) to understand where their visions toward working together to protect youth online converged and surfaced tensions, as well as how we might reconcile conflicting viewpoints to move forward as one community with synergistic expertise on how to change the current sociotechnical landscape for youth online safety. Overall, we found that non-partisan leadership is necessary to chart actionable, equitable goals to facilitate collaboration between stakeholders, combat feelings of isolation, and foster trust between the stakeholder groups. Based on these findings, we recommend the use of open-innovation methods with their inherent transparency, federated governance models, and clear but inclusive leadership structures to promote collaboration between youth online safety stakeholders. We propose the creation of an open-innovation organization that unifies the diverse voices in youth online safety to develop open-standards and evidence-based design patterns that centralize otherwise fragmented efforts that have fallen short of the goal of effective technological solutions that keep youth safe online.

Recent increases in self-harm and suicide rates among youth have coincided with prevalent social media use; therefore, making these sensitive topics of critical importance to the HCI research community. We analyzed 1,224 direct message conversations (DMs) from 151 young Instagram users (ages 13-21), who engaged in private conversations using self-harm and suicide-related language. We found that youth discussed their personal experiences, including imminent thoughts of suicide and/or self-harm, as well as their past attempts and recovery. They gossiped about others, including complaining about triggering content and coercive threats of self-harm and suicide but also tried to intervene when a friend was in danger. Most of the conversations involved suicide or self-harm language that did not indicate the intent to harm but instead used hyperbolical language or humor. Our results shed light on youth perceptions, norms, and experiences of self-harm and suicide to inform future efforts towards risk detection and prevention.

Accurate real-time risk identification is vital to protecting social media users from online harm, which has driven research towards advancements in machine learning (ML). While strides have been made regarding the computational facets of algorithms for “real-time” risk detection, such research has not yet evaluated these advancements through a human-centered lens. To this end, we conducted a systematic literature review of 53 peer-reviewed articles on real-time risk detection on social media. Real-time detection was mainly operationalized as “early” detection after-the-fact based on pre-defined chunks of data and evaluated based on standard performance metrics, such as timeliness. We identified several human-centered opportunities for advancing current algorithms, such as integrating human insight in feature selection, algorithms’ improvement considering human behavior, and utilizing human evaluations. This work serves as a critical call-to-action for the HCI and ML communities to work together to protect social media users before, during, and after exposure to risks.