AI-powered educational technologies have demonstrated measurable benefits for learners, but their design and evaluation have largely centered on K-12 contexts. As a result, many AI-supported learning systems remain poorly aligned with the needs, constraints, and goals of adult learners. To better understand how AI systems function in adult education, this paper examines the deployment of several AI learning technologies developed within a multidisciplinary, national research institute in the United States focused on adult learning and online education. Drawing on longitudinal deployment data, we conducted a reflexive thematic analysis to identify recurring challenges and design considerations across systems. These insights were synthesized into a set of 19 design guidelines intended to inform future AI-supported adult learning technologies. We demonstrate the utility of these guidelines through a heuristic evaluation of the deployed systems. Lastly, we present a guideline exploration tool that aids in the ideation of technologies by connecting the guidelines to stakeholder statements surfaced in the analysis process.

Online learning at scale has become dramatically more popular over the last decade. While these programs provide affordable and accessible education, low retention and engagement are persistent problems. Virtual Teaching Assistants (VTAs) offer a solution: VTAs such as Jill Watson can answer questions about course logistics and content, amplifying interaction between professors and students, increasing teaching presence, and thereby improving retention and engagement. Using the Community of Inquiry framework, this paper presents what we believe is the first experimental study of the effect a VTA has on student perceptions of teaching presence, social presence, and cognitive presence. Students in a large, online, graduate computer science course were randomly assigned to sections with and without access to Jill Watson. The Community of Inquiry survey was then administered at the end of the semester to measure the three presences. We find that Jill Watson has a small, positive, statistically significant effect on the Design & Organization dimension of teaching presence as well as social presence.