Artificial Intelligence (AI) education is an increasingly popular topic area for K-12 teachers. However, little research has investigated how AI curriculum and tools can be designed to be more accessible to all teachers and learners. In this study, we take a Value-Sensitive Design approach to understanding the role of teacher values in the design of AI curriculum and tools, and identifying opportunities to integrate AI into core curriculum to leverage learners’ interests. We organized co-design workshops with 15 K-12 teachers, where teachers and researchers co-created lesson plans using AI tools and embedding AI concepts into various core subjects. We found that K-12 teachers need additional scaffolding in AI tools and curriculum to facilitate ethics and data discussions, and value supports for learner evaluation and engagement, peer-to-peer collaboration, and critical reflection. We present an exemplar lesson plan that shows entry points for teaching AI in non-computing subjects and reflect on co-designing with K-12 teachers in a remote setting.

Understanding how machines learn is critical for children to develop useful mental models for exploring artificial intelligence (AI) and smart devices that they now frequently interact with. Although children are very familiar with having conversations with conversational agents like Siri and Alexa, children often have limited knowledge about AI and machine learning. We leverage their existing familiarity and present Zhorai, a conversational platform and curriculum designed to help young children understand how machines learn. Children ages eight to eleven train an agent through conversation and understand how the knowledge is represented using visualizations. This paper describes how we designed the curriculum and evaluated its effectiveness with 14 children in small groups. We found that the conversational aspect of the platform increased engagement during learning and the novel visualizations helped make machine knowledge understandable. As a result, we make recommendations for future iterations of Zhorai and approaches for teaching AI to children.

Explainable artificially intelligent (XAI) systems form part of sociotechnical systems, e.g., human+AI teams tasked with making decisions. Yet, current XAI systems are rarely evaluated by measuring the performance of human+AI teams on actual decision-making tasks. We conducted two online experiments and one in-person think-aloud study to evaluate two currently common techniques for evaluating XAI systems: (1) using proxy, artificial tasks such as how well humans predict the AI's decision from the given explanations, and (2) using subjective measures of trust and preference as predictors of actual performance. The results of our experiments demonstrate that evaluations with proxy tasks did not predict the results of the evaluations with the actual decision-making tasks. Further, the subjective measures on evaluations with actual decision-making tasks did not predict the objective performance on those same tasks. Our results suggest that by employing misleading evaluation methods, our field may be inadvertently slowing its progress toward developing human+AI teams that can reliably perform better than humans or AIs alone.

Vast improvements in natural language understanding and speech recognition have paved the way for conversational interaction with computers. While conversational agents have often been used for short goal-oriented dialog, we know little about agents for developing computer programs. To explore the utility of natural language for programming, we conducted a study (n=45) comparing different input methods to a conversational programming system we developed. Participants completed novice and advanced tasks using voice-based, text-based, and voice-or-text-based systems. We found that users appreciated aspects of each system (e.g., voice-input efficiency, text-input precision) and that novice users were more optimistic about programming using voice-input than advanced users. Our results show that future conversational programming tools should be tailored to users' programming experience and allow users to choose their preferred input mode. To reduce cognitive load, future interfaces can incorporate visualizations and possess custom natural language understanding and speech recognition models for programming.