Media representations of people with disabilities profoundly influence societal perceptions, yet have historically been absent, stereotyped, or inaccurate. As AI-generated visual media becomes increasingly prevalent, there is a critical opportunity to address these misrepresentations. Responding to the lack of collectively negotiated representation standards, this paper presents our human-centric approach to engaging disability communities meaningfully in AI data practices. Over three months, we worked closely with three disability organizations across the Global North and South to develop the Community Library Creator that introduces design scaffolds to support communities in defining ‘good’ representation and curating community-centric AI datasets; laying the foundations for community-specific evaluation metrics and future model adaptations. We contribute qualitative insights into the complexities of community-led data curation; discuss the value and practical challenges of intersecting human insights with AI requirements; and reflect on human-centered AI approaches that empower communities to share their perspectives and actively shape AI data practices.

Conversational agents (CAs) are a tempting type of computer interface for assisting people’s mental health due to their ability to simulate human-like interactions, however their integration within the broader social context of mental health management remains largely under-explored. Recognizing that managing one’s mental health is often a social rather than individual activity involving close persons such as partners, family, and friends, our research takes a social-orientation to mental health management. Utilizing design cards that depict fictional, yet plausible CA concepts, we present the analysis of an interview study with 24 young adults to understand their views on CAs for both their own use and for a close person. Participants viewed CAs as potentially valuable complements to human support, but expressed concerns about over-reliance and replacement. Our analysis reveal key tensions, design considerations, and opportunities for integrating CAs into mental health ecosystems in ways that respect and enhance existing social support structures.

Nasogastric tubes (NGTs) are feeding tubes that are inserted through the nose into the stomach to deliver nutrition or medication. If not placed correctly, they can cause serious harm, even death to patients. Recent AI developments demonstrate the feasibility of robustly detecting NGT placement from Chest X-ray images to reduce risks of sub-optimally or critically placed NGTs being missed or delayed in their detection, but gaps remain in clinical practice integration. In this study, we present a human-centered approach to the problem and describe insights derived following contextual inquiry and in-depth interviews with 15 clinical stakeholders. The interviews helped understand challenges in existing workflows, and how best to align technical capabilities with user needs and expectations. We discovered the tradeoffs and complexities that need consideration when choosing suitable workflow stages, target users, and design configurations for different AI proposals. We explored how to balance AI benefits and risks for healthcare staff and patients within broader organizational, technical, and medical-legal constraints. We also identified data issues related to edge cases and data biases that affect model training and evaluation; how data documentation practices influence data preparation and labeling; and how to measure relevant AI outcomes reliably in future evaluations. We discuss how our work informs design and development of AI applications that are clinically useful, ethical, and acceptable in real-world healthcare services.

Recent advances in AI combine large language models (LLMs) with vision encoders that bring forward unprecedented technical capabilities to leverage for a wide range of healthcare applications. Focusing on the domain of radiology, vision-language models (VLMs) achieve good performance results for tasks such as generating radiology findings based on a patient’s medical image, or answering visual questions (e.g., “Where are the nodules in this chest X-ray?”). However, the clinical utility of potential applications of these capabilities is currently underexplored. We engaged in an iterative, multidisciplinary design process to envision clinically relevant VLM interactions, and co-designed four VLM use concepts: Draft Report Generation, Augmented Report Review, Visual Search and Querying, and Patient Imaging History Highlights. We studied these concepts with 13 radiologists and clinicians who assessed the VLM concepts as valuable, yet articulated many design considerations. Reflecting on our findings, we discuss implications for integrating VLM capabilities in radiology, and for healthcare AI more generally.

CHI papers researching healthcare human-computer interaction (HCI) are increasingly reporting the use of “thematic analysis” (TA). TA refers to a range of flexible and evolving approaches for qualitative data analysis. Its increased use demonstrates a change in research practices, and with that the emergence of new local standards. We need to understand and reflect upon these emerging local practices, including departures from what is advocated as quality TA practice more generally. Toward this, we conducted a scoping review of a decade of CHI publications (2012 – 2021) that researched healthcare and termed their analysis approach “thematic analysis”; 78 papers reporting a total of 100 TAs were included. We contribute a description of 1) the contexts in which TA is being used, 2) the TA approaches being conducted, and 3) how TA is being reported. Drawing on this, we discuss opportunities to improve research practice when using TA in healthcare HCI.

Self-supervised learning in vision-language processing (VLP) exploits semantic alignment between imaging and text modalities. Prior work in biomedical VLP has mostly relied on the alignment of single image and report pairs even though clinical notes commonly refer to prior images. This does not only introduce poor alignment between the modalities but also a missed opportunity to exploit rich self-supervision through existing temporal content in the data. In this work, we explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model. It is designed to be versatile to arising challenges such as pose variations and missing input images across time. The resulting model excels on downstream tasks both in single- and multi-image setups, achieving state-of-the-art (SOTA) performance on (I) progression classification, (II) phrase grounding, and (III) report generation, whilst offering consistent improvements on disease classification and sentence-similarity tasks. We release a novel multi-modal temporal benchmark dataset, MS-CXR-T, to quantify the quality of vision-language representations in terms of temporal semantics. Our experimental results show the advantages of incorporating prior images and reports to make most use of the data.

Qianchu Liu, Stephanie Hyland, Shruthi Bannur, Kenza Bouzid, Daniel Castro, Maria Wetscherek, Robert Tinn, Harshita Sharma, Fernando Pérez-García, Anton Schwaighofer, Pranav Rajpurkar, Sameer Khanna, Hoifung Poon, Naoto Usuyama, Anja Thieme, Aditya Nori, Matthew Lungren, Ozan Oktay, Javier Alvarez-Valle. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Recent advances in AI and machine learning (ML) promise significant transformations in the future delivery of healthcare. Despite a surge in research and development, few works have moved beyond demonstrations of technical feasibility and algorithmic performance. However, to realize many of the ambitious visions for how AI can contribute to clinical impact requires the closer design and study of AI tools or interventions within specific health and care contexts. This article outlines our collaborative, human-centered approach to developing an AI application that predicts treatment outcomes for patients who are receiving human-supported, internet-delivered Cognitive Behavioral Therapy (iCBT) for symptoms of depression and anxiety. Intersecting the fields of HCI, AI, and healthcare, we describe how we addressed the specific challenges of (1) identifying clinically relevant AI applications ; and (2) designing AI applications for sensitive use contexts like mental health. Aiming to better assist the work practices of iCBT supporters, we share how learnings from an interview study with 15 iCBT supporters surfaced their practices and information needs and revealed new opportunities for the use of AI. Combined with insights from the clinical literature and technical feasibility constraints, this led to the development of two clinical outcome prediction models. To clarify their potential utility for use in practice, we conducted 13 design sessions with iCBT supporters that utilized interface mock-ups to concretize the AI output and derive additional design requirements. Our findings demonstrate how design choices can impact interpretations of the AI predictions as well as supporter motivation and sense of agency. We detail how this analysis and the design principles derived from it enabled the integration of the prediction models into a production interface. Reporting on identified risks of over-reliance on AI outputs and needs for balanced information assessment and preservation of a focus on individualized care, we discuss and reflect on what constitutes a responsible, human-centered approach to AI design in this healthcare context.

Writing about experiences of trauma and other challenges in life is known to provide measurable health benefits. Though writing for an audience may ensure better benefits, confiding one's most troubled memories in others risks a social stigma. Conversational agents can provide a virtual audience that ensures privacy and allows social disclosure. To understand the writing experience with an agent, we created Diarybot, a chatbot assistant for expressive writing. We designed two versions, Basic and Responsive, to explore the writing experience with and without bot follow-up interactions compared to a Google doc baseline. Findings from a 4-day user study with 30 participants reveal that social disclosure with Diarybot can encourage narrative writing, with relative ease and emotional expression in Basic chat. Responsive chat can mediate social acceptance of the bot and provide guidance for self-reflection in the process. We discuss design reflections on social disclosure with agents in pursuit of wellbeing.

AI technologies are often used to aid people in performing discrete tasks with well-defined goals (e.g., recognising faces in images). Emerging technologies that provide continuous, real-time information enable more open-ended AI experiences. In partnership with a blind child, we explore the challenges and opportunities of designing human-AI interaction for a system intended to support social sensemaking. Adopting a research-through-design perspective, we reflect upon working with the uncertain capabilities of AI systems in the design of this experience. We contribute: (i) a concrete example of an open-ended AI system that enabled a blind child to extend his own capabilities; (ii) an illustration of the delta between imagined and actual use, highlighting how capabilities derive from the human-AI interaction and not the AI system alone; and (iii) a discussion of design choices to craft an ongoing human-AI interaction that addresses the challenge of uncertain outputs of AI systems.

There is a dearth of appropriate tools for young learners with mixed visual abilities to engage with computational learning. Addressing this gap, Torino is a physical programming language for teaching computational learning to children ages 7–11 regardless of level of vision. To create code, children connect physical instruction pods and tune their parameter dials to create music, audio stories, or poetry. Currently, the uptake of novel educational technologies to support inclusive education of children with disabilities continues to be limited at scale. We consider how the Torino Learning Environment supports non-specialist teachers to teach computational learning to children with mixed visual abilities in a UK-wide evaluation with 75 children and 30 teachers over a period of three months. We demonstrate how children can successfully learn with a novel physical programming language. We articulate how key design constructs such as persistent program overview and liveness supported non-specialist teachers to co-produce learning for children of different ages, visual and cognitive abilities. We conclude with reflective guidance on evaluating inclusive educational technologies at scale.CCS CONCEPTSHuman-centered computing → Accessibility → Empirical studies in accessibility

Online mental health interventions are increasingly important in providing access to, and supporting the effectiveness of, mental health treatment. While these technologies are effective, user attrition and early disengagement are key challenges. Evidence suggests that integrating a human supporter into such services mitigates these challenges, however, it remains under-studied how supporter involvement benefits client outcomes, and how to maximize such effects. We present our analysis of 234,735 supporter messages to discover how different support strategies correlate with clinical outcomes. We describe our machine learning methods for: (i) clustering supporters based on client outcomes; (ii) extracting and analyzing linguistic features from supporter messages; and (iii) identifying context-specific patterns of support. Our findings indicate that concrete, positive and supportive feedback from supporters that reference social behaviors are strongly associated with better outcomes; and show how their importance varies dependent on different client situations. We discuss design implications for personalized support and supporter interfaces.

Across the world, policy initiatives are being developed to engage children with computer programming and computational thinking. Diversity and inclusion has been a strong force in this agenda, but children with disabilities have largely been omitted from the conversation. Currently, there are no age appropriate tools for teaching programming concepts and computational thinking to primary school children with visual disabilities. We address this gap through presenting the design and implementation of Torino, a tangible programming language for teaching programming concepts to children age 7–11 regardless of level of vision. In this paper, we: (1) describe the design process done in conjunction with children with visual disabilities; (2) articulate the design decisions made; and (3) report insights generated from an evaluation with 10 children with mixed visual abilities that considers how children are able to trace (read) and create (write) programs with Torino. We discuss key design trade-offs: (1) readability versus extensibility; and (2) size versus liveness. We conclude by reflecting upon how an inclusive design approach shaped the final result.

High prevalence of mental illness and the need for effective mental health care, combined with recent advances in AI, has led to an increase in explorations of how the field of machine learning (ML) can assist in the detection, diagnosis and treatment of mental health problems. ML techniques can potentially offer new routes for learning patterns of human behavior; identifying mental health symptoms and risk factors; developing predictions about disease progression; and personalizing and optimizing therapies. Despite the potential opportunities for using ML within mental health, this is an emerging research area, and the development of effective ML-enabled applications that are implementable in practice is bound up with an array of complex, interwoven challenges. Aiming to guide future research and identify new directions for advancing development in this important domain, this article presents an introduction to, and a systematic review of, current ML work regarding psycho-socially based mental health conditions from the computing and HCI literature. A quantitative synthesis and qualitative narrative review of 54 papers that were included in the analysis surfaced common trends, gaps, and challenges in this space. Discussing our findings, we (i) reflect on the current state-of-the-art of ML work for mental health, (ii) provide concrete suggestions for a stronger integration of human-centered and multi-disciplinary approaches in research and development, and (iii) invite more consideration of the potentially far-reaching personal, social, and ethical implications that ML models and interventions can have, if they are to find widespread, successful adoption in real-world mental health contexts.

We propose a computer vision system that can automatically detect people in dynamic real-world scenes, enabling people with vision impairments to have more awareness of, and interactions with, other people in their surroundings. As an initial step, we investigate the feasibility of four camera systems that vary in their placement, field-of-view, and image distortion for: (i) capturing people generally; and (ii) detecting people via a specific person-pose estimator. Based on our findings, we discuss future opportunities and challenges for detecting people in dynamic scenes, and for communicating that information to visually impaired users.

This research takes an orientation to visual impairment (VI) that does not regard it as fixed or determined alone in or through the body. Instead, we consider (dis)ability as produced through interactions with the environment and configured by the people and technology within it. Specifically, we explore how abilities become negotiated through video ethnography with six VI athletes and spectators during the Rio 2016 Paralympics. We use generated in-depth examples to identify how technology can be a meaningful part of ability negotiations, emphasizing how these embed into the social interactions and lives of people with VI. In contrast to treating technology as a solution to a 'sensory deficit', we understand it to support the triangulation process of sense-making through provision of appropriate additional information. Further, we suggest that technology should not try and replace human assistance, but instead enable people with VI to better identify and interact with other people in-situ.

We investigate how technology can support collaborative learning by children with mixed-visual abilities. Responding to a growing need for tools inclusive of children with vision impairments (VI) for the teaching of computer programing to novice learners, we explore Torino -- a physical programing language for teaching programing constructs and computational thinking to children age 7-11. We draw insights from 12 learning sessions with Torino that involved five pairs of children with vision ranging from blindness to full-sight. Our findings show how sense-making of the technology, collaboration, and learning were enabled through an interplay of system design, programing tasks and social interactions, and how this differed between the pairs. The paper contributes insights on the role of touch, audio and visual representations in designs inclusive of people with VI, and discusses the importance and opportunities provided through the 'social' in negotiations of accessibility, for learning, and for self-perceptions of ability and self-esteem.

There has been a surge in artificial intelligence (AI) technologies co-opted by or designed for people with visual disabilities. Researchers and engineers have pushed technical boundaries in areas such as computer vision, natural language processing, location inference, and wearable computing. But what do people with visual disabilities imagine as their own technological future? To explore this question, we developed and carried out tactile ideation workshops with participants in the UK and India. Our participants generated a large and diverse set of ideas, most focusing on ways to meet needs related to social interaction. In some cases, this was a matter of recognizing people. In other cases, they wanted to be able to participate in social situations without foregrounding their disability. It was striking that this finding was consistent across UK and India despite substantial cultural and infrastructural differences. In this paper, we describe a new technique for working with people with visual disabilities to imagine new technologies that are tuned to their needs and aspirations. Based on our experience with these workshops, we provide a set of social dimensions to consider in the design of new AI technologies: social participation, social navigation, social maintenance, and social independence. We offer these social dimensions as a starting point to forefront users' social needs and desires as a more deliberate consideration for assistive technology design.

This special issue on ethical matter(s) contains innovative and reflective scholarship focused on questions of ethics with human computer interaction (HCI) design research. Through this introductory piece, we articulate three trends influencing contemporary design research which are creating spaces of discomfort for many scholars as they acknowledge their positionality and responsibility. We introduce the articles developed for this special issue through the framing of ethical difficulties, dilemmas and paradoxes, recognizing that some issues may not be immediately resolvable. We argue that no matter which structures, principles or heuristics you commit to at the start of an HCI project, ethical conundrums are likely to develop. Through this piece, we are not suggesting that anyone is trying to achieve the status of moral saint within HCI design research. Rather, we offer you work crafted by scholars striving to improve their scholarship, working to address the question of ethical action in HCI research.

There are currently over 1.1 million Syrian refugees in need of healthcare services from an already overstretched Lebanese healthcare system. Access to antenatal care (ANC) services presents a particular challenge. We conducted focus groups with 59 refugees in rural Lebanon to identify contextual and cultural factors that can inform the design of digital technologies to support refugee ANC. Previously identified high utilization of smartphones by the refugee population offers a particular opportunity for using digital technology to support access to ANC as well as health advocacy. Our findings revealed a number of considerations that should be taken into account in the design of refugee ANC technologies, including: refugee health beliefs and experiences, literacy levels, refugee perceptions of negative attitudes of healthcare providers, and hierarchal and familial structures.

This paper describes the challenges and lessons learned in the experience-centered design (ECD) of the Spheres of Wellbeing, a technology to promote the mental health and wellbeing of a group of women, suffering from significant mental health problems and living in a medium secure hospital unit. First, we describe how our relationship with mental health professionals at the hospital and the aspirations for person-centric care that we shared with them enabled us, in the design of the Spheres, to innovate outside traditional healthcare procedures. We then provide insights into the challenges presented by the particular care culture and existing services and practices in the secure hospital unit that were revealed through our technology deployment. In discussing these challenges, our design enquiry opens up a space to make sense of experience living with complex mental health conditions in highly constrained contexts within which the deployment of the Spheres becomes an opportunity to think about wellbeing in similar contexts.

Well-developed interpersonal skills are crucial for all social interactions. However, understanding how interpersonal skills are taught or learned, and how technology can play a part in this, is yet an under-researched area in CSCW and HCI research. To start addressing this gap, our research explores the learning processes of counselling students, for whom developing interpersonal skills forms a fundamental part of their university education. We followed an iterative process to gain an in-depth understanding of a specific counselling program in the UK, combining interviews and low-fidelity technology prompts. Overall, 26 participants comprising tutors, students and expert counsellors took part. Our findings first provide insights into the highly collaborative and social learning process of the students. We highlight the complexity of interpersonal reflection as a crucial process for developing counselling skills, and identify the challenges to learning that students face. Second, we build on this understanding to draw out empirically grounded design considerations around opportunities for technology innovation in this setting.

We consider the role of making in current HCI design practices and how it may affect the wellbeing of those who participate in these processes. Through an exploration of psychological concepts of wellbeing and their connection to making experiences, we suggest that making can facilitate and support both hedonic and eudemonic facets of wellbeing. We illustrate this in the context of three case studies that engaged people in creative making activities as part of user-centered design processes. Based on our experiences, we argue that researchers ought to be mindful of the potential impact our design processes have on our participants and provide considerations for those designing for and with participants where wellbeing is a concern.

In the field of mental health care technologies, very limited attention has been given to the design of interventions for individuals who undergo treatment for severe mental health problems in intense care contexts. Exploring novel designs to engage vulnerable psychiatric patients in therapeutic skills practice and expanding on the potential of technology to promote mental health, the paper introduces the design concept of the Spheres of Wellbeing. A set of interactive artifacts is developed specifically for women with a dual diagnosis of a Learning Disability and Borderline Personality Disorder, living in the medium secure services of a forensic hospital in the UK. The women present a difficult to treat group due to extremely challenging behaviors and a fundamental lack of motivation to engage in therapy. The Spheres are designed to assist the women in practices of mindfulness, to help them tolerate emotional distress and to strengthen their sense of self, all of which are vital components of their specialist treatment Dialectical Behavioral Therapy (DBT). The Spheres are intended to supplement the therapy of the women and to contribute to our understanding of designing technology to enhance mental wellbeing and quality of life more generally.

HCI researchers working in publically funded institutions are increasingly encouraged to engage the public in their research. Mass media is often seen as an effective medium with which to communicate research to large parts of the population. We present an account of three HCI projects that have used engagements with mass media in order to communicate research to the public. We describe the motivations for working with mass media and the mechanics of writing press releases. A grounded theory analysis of online public responses to the projects in the mass media leads us to identify a number of concerns about how research is portrayed by news outlets and thus interpreted by the public. Tensions about technologies and wider societal issues were revealed that might normally be hidden when using traditional user-centred methods. We critically reflect on the efficacy of using the mass media in research and provide guidance for HCI researchers wishing to engage in dialogues with the public in the future.

Panopticon is a video surrogate system that displays multiple sub-sequences in parallel to present a rapid overview of the entire sequence to the user. A novel, precisely animated arrangement slides thumbnails to provide a consistent spatiotemporal layout while allowing any sub-sequence of the original video to be watched without interruption. Furthermore, this output can be generated offline as a highly efficient repeated animation loop, making it suitable for resource-constrained environments, such as web-based interaction. Two versions of Panopticon were evaluated using three different types of video footage with the aim of determining the usability of the proposed system. Results demonstrated an advantage over another surrogate with surveillance footage in terms of search times and this advantage was further improved with Panopticon 2. Eye tracking data suggests that Panopticon's advantage stems from the animated timeline that users heavily rely on.

This two-day workshop will bring together an interdisciplinary group of researchers, designers and practitioners who are interested in the topic of wellbeing in the field of interaction design. Wellbeing is defined as positive mental health, and not only the absence of mental illness, but also the presence of positive psychological functioning. The workshop will provide a platform to share resources, create new ideas for design and build valuable future collaborations. During the first day participants will present their work and exchange their knowledge and experiences in the field. The workshop will utilize a series of interactive activities to support participants in collaboratively constructing a shared understanding of the concept of wellbeing and its challenges in terms of design. On the second day participants will be invited to create low-fidelity prototypes that support an aspect of wellbeing using Microsoft .NET Gadgeteer and other prototyping materials. These practical activities will stimulate discussion so as to contribute to a better understanding of how wellbeing can be facilitated through design, how to address evaluation challenges as well as illustrate related ethical questions.

BinCam is a social persuasive system to motivate reflection and behavioral change in the food waste and recycling habits of young adults. The system replaces an existing kitchen refuse bin and automatically logs disposed of items through digital images captured by a smart phone installed on the underside of the bin lid. Captured images are uploaded to a BinCam application on Facebook where they can be explored by all users of the BinCam system. Engagement with BinCam is designed to fit into the existing structure of users' everyday life, with the intention that reflection on waste and recycling becomes a playful and shared group activity. Results of a user study reveal an increase in both users' awareness of, and reflection about, their waste management and their motivation to improve their waste-related skills. With BinCam, we also explore informational and normative social influences as a source of change (e.g., socially evoked feelings of 'guilt' for non-recycling or food disposal), which has to date been underexplored in persuasive HCI. Design implications for reflection and social persuasion are proposed.

Design and digital technologies to support a sense of self and human relationships for people living with dementia are both urgently needed. We present an enquiry into design for dementia facilitated by a public art commission for an adult mental health unit in a hospital in the UK. The interactive art piece was informed by the notion of personhood in dementia that foregrounds the person's social being and interpersonal relationships as sites where self is maintained and constructed. How clients, clients' family members and staff used the piece is reported and insights related to the notions of home, intimacy, possessions and self are presented. The art piece served as window on both dementia and the institution leading to a number of insights and implications for design.