Recent works have shown that tackling offline reinforcement learning (RL) with a conditional policy produces promising results. The Decision Transformer (DT) combines the conditional policy approach and a transformer architecture, showing competitive performance against several benchmarks. However, DT lacks stitching ability -- one of the critical abilities for offline RL to learn the optimal policy from sub-optimal trajectories. This issue becomes particularly significant when the offline dataset only contains sub-optimal trajectories. On the other hand, the conventional RL approaches based on Dynamic Programming (such as Q-learning) do not have the same limitation; however, they suffer from unstable learning behaviours, especially when they rely on function approximation in an off-policy learning setting. In this paper, we propose the Q-learning Decision Transformer (QDT) to address the shortcomings of DT by leveraging the benefits of Dynamic Programming (Q-learning). It utilises the Dynamic Programming results to relabel the return-to-go in the training data to then train the DT with the relabelled data. Our approach efficiently exploits the benefits of these two approaches and compensates for each other's shortcomings to achieve better performance.

Type 1 Diabetes (T1D) self-management requires hundreds of daily decisions. Diabetes technologies that use machine learning have significant potential to simplify this process and provide better decision support, but often rely on cumbersome data logging and cognitively demanding reflection on collected data. We set out to use co-design to identify opportunities for machine learning to support diabetes self-management in everyday settings. However, over nine months of interviews and design workshops with 15 people with T1D, we had to re-assess our assumptions about user needs. Our participants reported confidence in their personal knowledge and rejected machine learning based decision support when coping with routine situations, but highlighted the need for technological support in the context of unfamiliar or unexpected situations (holidays, illness, etc.). However, these are the situations where prior data are often lacking and drawing data-driven conclusions is challenging. Reflecting this challenge, we provide suggestions on how machine learning and other artificial intelligence approaches, e.g., expert systems, could enable decision-making support in both routine and unexpected situations.

Co-design is a widely applied design process with well-documented values, including mutual learning and collective creativity. However, the real-world challenges of conducting multidisciplinary co-design research to inform the design of self-care technologies are not well established. We provide a qualitative account of a multidisciplinary project that aimed to co-design machine learning applications for Type 1 Diabetes (T1D) self-management. Through interviews, we identify not only perceived social, technological and strategic benefits of co-design but also organisational, translational and pragmatic design challenges: participants with T1D experienced difficulties in co-designing systems that met their individual self-care needs as part of group activities; HCI and AI researchers described challenges resulting from applying co-design outcomes to data-driven ML work; and industry collaborators highlighted academic data sharing regulations as cross-organisational challenges that can impede co-design efforts. Based on this understanding, we discuss opportunities for supporting multidisciplinary collaborations and aligning individual health needs with collaborative co-design activities.