One of the core goals of responsible AI development is ensuring high-quality training datasets. Many researchers have pointed to the importance of the annotation step in the creation of high-quality data, but less attention has been paid to the work that enables data annotation. We define this work as the design of ground truth schema and explore the challenges involved in the creation of datasets in the medical domain even before any annotations are made. Based on extensive work in three health-tech organisations, we describe five external and internal factors that condition medical dataset creation processes. Three external factors include regulatory constraints, the context of creation and use, and commercial and operational pressures. These factors condition medical data collection and shape the ground truth schema design. Two internal factors include epistemic differences and limits of labelling. These directly shape the design of the ground truth schema. Discussions of what constitutes high-quality data need to pay attention to the factors that shape and constrain what is possible to be created, to ensure responsible AI design.

Artificial Intelligence (AI) in medical applications holds great promise. However, the use of Machine Learning-based (ML) systems in clinical practice is still minimal. It is uniquely difficult to introduce clinician-facing ML-based systems in practice, which has been recognised in HCI and related fields. Recent publications have begun to address the sociotechnical challenges of designing, developing, and successfully deploying clinician-facing ML-based systems. We conducted a qualitative systematic review and provided answers to the question: “How can HCI researchers and practitioners contribute to the successful realisation of ML in medical practice?” We reviewed 25 eligible papers that investigated the real-world clinical implications of concrete clinician-facing ML-based systems. The main contributions of this systematic review are: (1) an overview of the technical aspects of ML innovation and their consequences for HCI researchers and practitioners; (2) a description of the different roles that ML-based systems can take in clinical settings; (3) a conceptualisation of the main activities of medical ML innovation processes; (4) identification of five sociotechnical interdependencies that emerge from medical ML innovation; and (5) implications for HCI researchers and practitioners on how to mitigate the sociotechnical challenges of medical ML innovation.

The design of patient-centred e-health services embodies an inherent tension between the concerns of clinicians and those of patients. Clinicians’ concerns are related to professional issues to do with diagnosing and curing disease in accordance with accepted medical standards. In contrast, patients’ concerns typically relate to personal experience and quality of life issues. It is about their identity, their hopes, their fears and their need to maintain a meaningful life. This divergence of concerns presents a fundamental challenge for designers of patient-centred e-health services. We explore this challenge in the context of chronic illness and telecare. Based on insights from medical phenomenology as well as our own experience with designing an e-health service for patients with chronic heart disease, we emphasise the importance – and difficulty – of aligning the concerns of patients and clinicians. To deal with this, we propose a set of concepts for analysing concerns related to the design of e-health services: A concern is (1) meaningful if it is relevant and makes sense to both patients and clinicians, (2) actionable if clinicians or patients – at least in principle – are able to take appropriate action to deal with it, and (3) feasible if it is easy and convenient to do so within the organisational and social context. We conclude with a call for a more participatory and iterative approach to the design of patient-centred e-health services.

The article presents a conceptual framework and a coherent method for design in an organizational context within the participatory design tradition. The MUST method has been developed throughout 10 projects in Danish and American organizations, and it has recently been evaluated and adopted by 3 Danish organizations. The method is based on thorough participation with users and managers, and it combines the use of ethnographic techniques and intervention. The article describes the application area and perspective of the method, presents 6 general principles on which the method is based, and describes 5 main activities providing a stepwise decision-making process in the overall design process. Each of the main activities is illustrated by an example taken from our last project. The article concludes by summing up the main points.