We present a case study of Persona-L, a system that leverages large language models (LLMs) and retrieval-augmented generation (RAG) to model personas of people with Down syndrome. Existing approaches to persona creation can often lead to oversimplified or stereotypical profiles of people with Down syndrome. To that end, we built stereotype detection capabilities into Persona-L as a design probe to open conversations. We then conducted interviews with caregivers and healthcare professionals (N=10) to examine how Down syndrome stereotypes could manifest in the content and delivery of LLM outputs, and interface design. Our findings show the challenges in defining stereotypes, and reveal potential pathways where stereotypes could emerge, including the training data, LLM outputs and interface design. This highlights the need for participatory methods that capture the heterogeneity of lived experiences of people with Down syndrome.

Users feel frustrated when they do not know when to speak with LLM-based agents. Technical delays disrupt the natural rhythm of conversation (turn-taking), yet there is little understanding of how these specific delays impact the back-and-forth flow of interaction. To address this, we analyzed human-agent conversations in social VR to measure timing differences. We used conversation analysis techniques to track specific timing metrics, such as how long it takes to respond (response latencies) and how agents handle interruptions (repair attempts). We found that agents are significantly slower to respond with a median of 4.1 seconds compared to a human’s 1.2 seconds. We identified a “conversational timing drift”, noting that agents struggle with start-up latency, i.e., taking too long to start speaking, and wind-down latency, i.e., failing to stop speaking quickly when a user interrupts them. This is the first study to empirically quantify human-agent conversational latencies within VR. We offer design suggestions to help future agents manage conversational timing better, ultimately improving natural conversation and user experience.

Foreign language speaking anxiety (FLSA) poses a major challenge for English-language learners, suppressing confidence and triggering a cycle of avoidance that hinders language acquisition. To address this, we explored the use of LLM-based embodied conversational agents (ECA) in social virtual reality (VR), which provide personalized support and multimodal interaction in a contextualized environment. We developed three English-language learning scenarios in social VR and conducted a five-day mixed-methods study where participants (N=20) engaged in daily 30-minute role-play practice with an LLM-based ECA to evaluate the efficacy of the system. Quantitative results showed a significant reduction in self-reported FLAS after 3 days, along with subtle gains in speaking proficiency measures. Qualitatively, learners perceived increased confidence, attributing it to the LLM-based ECA’s non-judgmental stance, linguistic scaffolding, affective encouragement, and adaptive feedback. Our findings suggest the potential of LLM-based ECAs in social VR for language learning and offer considerations for future agent design.

Many people struggle with learning a new language when moving to a new country, with traditional tools falling short in providing contextualized learning tailored to each learner's needs.The recent development of large language models (LLMs) and embodied conversational agents (ECAs) in social virtual reality (VR) provides new opportunities to practice language learning in a contextualized and naturalistic way that takes into account the learner's language level and needs.To explore this opportunity, we developed ELLMA-T, 576