Large Language Models (LLMs) are increasingly used to generate clinical documentation, yet most systems focus on easing individual users’ clerical burdens rather than supporting multi-stakeholder work. We explore how LLMs expand the design space of clinical multi-stakeholder systems. Through a research-through-design study with 10 dermatologists using a teledermatology prototype, we explore how designing for secondary stakeholders opens new design opportunities and reshapes primary users’ work. Applying the distributed cognition lens to our findings, we conceptualised representational design, i.e., the process of encoding secondary stakeholder perspectives (representations) in LLMs through, among others, prompts, templates, and constraints, while attending to LLMs’ latent knowledge. We envision representational design as supplemental to data- and interaction-level approaches to designing multi-stakeholder artificial intelligence (AI) systems. We demonstrate how LLM-mediated representations cause misalignments between generated outputs and primary users’ notions of acceptability, and how attending to these misalignments sets requirements for human-AI interaction techniques.

Despite recent advancements, real-world use of Artificial Intelligence (AI) in radiology remains low, often due to the mismatch between AI offerings and the situated challenges faced by healthcare professionals. To bridge this gap, we conducted a field study at nine medical sites in Denmark and Kenya with two goals: (1) to understand the challenges faced by radiologists during chest X-ray practice and (2) to envision alternative AI futures that align with collaborative clinical work. This study uniquely grounds the AI design insights in the comprehensive characterisation of diagnostic work across multiple geographical and institutional contexts. Building on ideas articulated by interviewed radiologists (N = 18), we conceptualised five visions that transcend the traditional notions of AI support. These visions emphasise that the clinical usefulness of AI-based systems depends on their configurability and flexibility across three dimensions: type of clinical site, expertise of medical professionals, and situational and patient contexts. Addressing these dependencies requires expanding the clinical AI design space by envisioning futures rooted in the realities of practice rather than solely following the trajectory of AI development.

Artificial Intelligence (AI) repeatedly match or outperform radiologists in lab experiments. However, real-world implementations of radiological AI-based systems are found to provide little to no clinical value. This paper explores how to design AI for clinical usefulness in different contexts. We conducted 19 design sessions and design interventions with 13 radiologists from 7 clinical sites in Denmark and Kenya, based on three iterations of a functional AI-based prototype. Ten sociotechnical dependencies were identified as crucial for the design of AI in radiology. We conceptualised four technical dimensions that must be configured to the intended clinical context of use: AI functionality, AI medical focus, AI decision threshold, and AI Explainability. We present four design recommendations on how to address dependencies pertaining to the medical knowledge, clinic type, user expertise level, patient context, and user situation that condition the configuration of these technical dimensions.

One of the core goals of responsible AI development is ensuring high-quality training datasets. Many researchers have pointed to the importance of the annotation step in the creation of high-quality data, but less attention has been paid to the work that enables data annotation. We define this work as the design of ground truth schema and explore the challenges involved in the creation of datasets in the medical domain even before any annotations are made. Based on extensive work in three health-tech organisations, we describe five external and internal factors that condition medical dataset creation processes. Three external factors include regulatory constraints, the context of creation and use, and commercial and operational pressures. These factors condition medical data collection and shape the ground truth schema design. Two internal factors include epistemic differences and limits of labelling. These directly shape the design of the ground truth schema. Discussions of what constitutes high-quality data need to pay attention to the factors that shape and constrain what is possible to be created, to ensure responsible AI design.

Artificial Intelligence (AI) in medical applications holds great promise. However, the use of Machine Learning-based (ML) systems in clinical practice is still minimal. It is uniquely difficult to introduce clinician-facing ML-based systems in practice, which has been recognised in HCI and related fields. Recent publications have begun to address the sociotechnical challenges of designing, developing, and successfully deploying clinician-facing ML-based systems. We conducted a qualitative systematic review and provided answers to the question: “How can HCI researchers and practitioners contribute to the successful realisation of ML in medical practice?” We reviewed 25 eligible papers that investigated the real-world clinical implications of concrete clinician-facing ML-based systems. The main contributions of this systematic review are: (1) an overview of the technical aspects of ML innovation and their consequences for HCI researchers and practitioners; (2) a description of the different roles that ML-based systems can take in clinical settings; (3) a conceptualisation of the main activities of medical ML innovation processes; (4) identification of five sociotechnical interdependencies that emerge from medical ML innovation; and (5) implications for HCI researchers and practitioners on how to mitigate the sociotechnical challenges of medical ML innovation.

The design of patient-centred e-health services embodies an inherent tension between the concerns of clinicians and those of patients. Clinicians’ concerns are related to professional issues to do with diagnosing and curing disease in accordance with accepted medical standards. In contrast, patients’ concerns typically relate to personal experience and quality of life issues. It is about their identity, their hopes, their fears and their need to maintain a meaningful life. This divergence of concerns presents a fundamental challenge for designers of patient-centred e-health services. We explore this challenge in the context of chronic illness and telecare. Based on insights from medical phenomenology as well as our own experience with designing an e-health service for patients with chronic heart disease, we emphasise the importance – and difficulty – of aligning the concerns of patients and clinicians. To deal with this, we propose a set of concepts for analysing concerns related to the design of e-health services: A concern is (1) meaningful if it is relevant and makes sense to both patients and clinicians, (2) actionable if clinicians or patients – at least in principle – are able to take appropriate action to deal with it, and (3) feasible if it is easy and convenient to do so within the organisational and social context. We conclude with a call for a more participatory and iterative approach to the design of patient-centred e-health services.

The relation between ethnography and design is often discussed in terms of being direct or indirect. The debate on using ethnography in design, models the problem as a matter of mediating between users and designers. This fails to take 'use' serious and renders the problem epistemic i.e. a matter of creating a better understanding or description of the user. Inspired by later developments in Science and Technology Studies I engage an ontological reconceptualization and turn to consider and practice the relation as performative - thus making ethnography, design and users' practices converge. I show this by the case of how the concept of 'medication management' has been performed differently on a combined CSCW and participatory design project in healthcare. It is suggested that through design interventions with working prototypes; prospective analysis and participatory design can be fruitfully assembled in situations of use.