Designing Conversational AI systems to support older adults requires these systems to explain their behavior in ways that align with older adults’ preferences and context. While prior work has emphasized the importance of AI explainability in building user trust, relatively little is known about older adults’ requirements and perceptions of AI-generated explanations. To address this gap, we conducted an exploratory Speed Dating study with 23 older adults to understand their responses to contextually grounded AI explanations. Our findings reveal the highly context-dependent nature of explanations, shaped by conversational cues such as the content, tone, and framing of explanation. We also found that explanations are often interpreted as interactive, multi-turn conversational exchanges with the AI, and can be helpful in calibrating urgency, guiding actionability, and providing insights into older adults’ daily lives for their family members. We conclude by discussing implications for designing context-sensitive and personalized explanations in Conversational AI systems.

Visual question answering (VQA) lies at the intersection of language and vision research. It functions as a building block for multimodal conversational AI and serves as a testbed for assessing a model’s capability for open-domain scene understanding. While progress in this area was initially accelerated with the 2015 release of the popular and large dataset "VQA", new datasets are required to continue this research momentum. For example, the 2019 Outside Knowledge VQA dataset "OKVQA" extends VQA by adding more challenging questions that require complex, factual, and commonsense knowledge. However, in our analysis, we found that 41.4% of the dataset needed to be corrected and 10.6% needed to be removed. This paper describes the analysis, corrections, and removals completed and presents a new dataset: OK-VQA Version 2.0. To gain insights into the impact of the changes on OK-VQA research, the paper presents results on state-of-the-art models retrained with this new dataset. The side-by-side comparisons show that one method in particular, Knowledge Augmented Transformer for Vision-and-Language, extends its relative lead over competing methods. The dataset is available online.1

While commercial conversational agents (CA) (i.e. Google assistant, Siri, Alexa) are widely used, these systems have limitations in error-handling, flexibility, personalization and overall dialogue management that are amplified in care coordination settings. In this paper, we synthesize and articulate these limitations through quantitative and qualitative analysis of 56 older adults interacting with a commercial CA deployed in their home for a 10 week period. We look at the CA as a compensatory technology in an older adult's care network. We argue that the CA limitations are rooted in the rigid cue-and-response style of task-oriented interactions common in CAs. We then propose a redesign for CA conversation flow to favor flexibility and personalization that is nonetheless viable within the limitations of current AI and machine learning technologies. We explore design tradeoffs to better support the usability needs of older adults compared to current design optimizations driven by efficiency and privacy goals.

Improving medication management for older adults with Mild Cognitive Impairment (MCI) requires designing systems that support functional independence and provide compensatory strategies as their abilities change. Traditional medication management interventions emphasize forming new habits alongside the traditional path of learning to use new technologies. In this study, we navigate designing for older adults with gradual cognitive decline by creating a conversational “check-in” system for routine medication management. We present the design of MATCHA - Medication Action To Check-In for Health Application, informed by exploratory focus groups and design sessions conducted with older adults with MCI and their caregivers, alongside our evaluation based on a two-phased deployment period of 20 weeks. Our results indicate that a conversational “check-in” medication management assistant increased system acceptance while also potentially decreasing the likelihood of accidental over-medication, a common concern for older adults dealing with MCI.

In the Spring of 2020, closures and safe distancing orders swept much of the United States due to the COVID-19 pandemic. This paper presents a case study of pivoting an in-person empowerment program focused on lifestyle interventions for people newly diagnosed with Mild Cognitive Impairment (MCI) to an online program. Working as rapidly as possible to sustain participant engagement, our design decisions and subsequent iterations point to initial constraints in telehealth capabilities, as well as learning on the fly as new capabilities and requirements emerged. We present the discovery of emergent practices by family members and healthcare providers to meet the new requirements for successful online engagement. For some participants, the online program led to greater opportunities for empowerment while others were hampered by the lack of in-person program support. Providers experienced a sharp learning curve and likewise missed the benefits of in-person interaction, but also discovered new benefits of online collaboration. This work lends insights and potential new avenues for understanding how lifestyle interventions can empower people with MCI and the role of technology in that process.

Conversational agents (CAs) such as Google Home or Alexa offer empowering opportunities for dyads composed of older adults with mild cognitive impairment (MCI) and their care partners. CAs support coordination and planning between the two, and can amplify the support that the care partner needs to provide. In this study, we observed how ten such dyads interacted with a Google Home over 10 weeks. We logged and analyzed 3,878 total interactions, interviewed the dyads to better understand their experiences, and also surveyed their individual preferences and priorities for automated assistance in the home. We found that CAs empowered both the people who had MCI, and their care partners. We observed that the utility of the CA in the day-to-day lives of users largely depended on how much the care partner scaffolded promising functionality, setting it up and contextualizing it for specific needs and desires.