Initial discussion of AI literacy assessment has focused on competency frameworks and learning standards rather than materials for classroom use. Responsible AI for Computational Action (RAICA), a constructionist AI curriculum for middle and high school students, includes assessment materials to support teachers with the evaluation of student AI literacy competencies in their classrooms. These materials include exit tickets used as formative assessments at the end of each lesson and both teacher and student-facing rubrics. After beta-testing a module of the curriculum with nine teachers and 282 students, we reviewed teacher usage data and feedback as well as student responses. The review process surfaced a number of improvements to the materials to better align them with classroom teaching practice. These included clarifying language and adding visual scaffolds. We present the assessment materials and iterative design process used to bridge the gap between the theoretical AI literacy competencies and their practical implementation in classrooms.

As generative AI (GenAI) emerges as a transformative force, clear understanding of high school students' perspectives is essential for GenAI's meaningful integration in high school environments. In this work, we draw insights from a participatory design workshop where we engaged 17 high school students -- a group rarely involved in prior research in this area -- through the design of novel GenAI tools and school policies addressing their key concerns. Students identified challenges and developed solutions outlining their ideal features in GenAI tools, appropriate school use, and regulations. These centered around the problem spaces of combating bias & misinformation, tackling crime & plagiarism, preventing over-reliance on AI, and handling false accusations of academic dishonesty. Building on our participants' underrepresented perspectives, we propose new guidelines targeted at educational technology designers for development of GenAI technologies in high schools. We also argue for further incorporation of student voices in development of AI policies in their schools.

CHI ’25, Yokohama, Japan

Providing personalized, detailed feedback at scale in large undergraduate STEM courses remains a persistent challenge. We present an empirically evaluated practice exam system that integrates AI generated feedback with targeted textbook references, deployed in a large introductory biology course. Our system specifically aims to encourage metacognitive behavior by asking students to explain their answers and declare their confidence. It uses OpenAI's GPT-4o to generate personalized feedback based on this information, while directing them to relevant textbook sections. Through detailed interaction logs from consenting participants across three midterms (541, 342, and 413 students respectively), totaling 28,313 question-student interactions across 146 learning objectives, along with 279 post-exam surveys and 23 semi-structured interviews, we examined the system's impact on learning outcomes and student engagement. Analysis showed that across all midterms, the different feedback types showed no statistically significant differences in performance, though there were some trends suggesting potential benefits worth further investigation. The system's most substantial impact emerged through its required confidence ratings and explanations, which students reported transferring to their actual exam strategies. Approximately 40% of students engaged with textbook references when prompted by feedback---significantly higher than traditional reading compliance rates. Survey data revealed high student satisfaction (M=4.1/5), with 82.1% reporting increased confidence on midterm topics they had practiced, and 73.4% indicating they could recall and apply specific concepts from practice sessions. Our findings demonstrate how thoughtfully designed AI-enhanced systems can scale formative assessment while promoting sustainable study practices and self-regulated learning behaviors, suggesting that embedding structured reflection requirements may be more impactful than sophisticated feedback mechanisms.

Text-to-image generation (TTIG) technologies are Artificial Intelligence (AI) algorithms that use natural language algorithms in combination with visual generative algorithms. TTIG tools have gained popularity in recent months, garnering interest from non-AI experts, including educators and K-12 students. While they have exciting creative potential when used by K-12 learners and educators for creative learning, they are also accompanied by serious ethical implications, such as data privacy, spreading misinformation, and algorithmic bias. Given the potential learning applications, social implications, and ethical concerns, we designed 6-hour learning materials to teach K-12 teachers from diverse subject expertise about the technical implementation, classroom applications, and ethical implications of TTIG algorithms. We piloted the learning materials titled “Demystify text-to-image generative tools for K-12 educators" with 30 teachers across two workshops with the goal of preparing them to teach about and use TTIG tools in their classrooms. We found that teachers demonstrated a technical, applied and ethical understanding of TTIG algorithms and successfully designed prototypes of teaching materials for their classrooms.

Generative AI tools introduce new and accessible forms of media creation for youth. They also raise ethical concerns about the generation of fake media, data protection, privacy and ownership of AI-generated art. Since generative AI is already being used in products used by youth, it is critical that they understand how these tools work and how they can be used or misused. In this work, we facilitated students’ generative AI learning through expression of their imagined future identities. We designed a learning workshop - Dreaming with AI - where students learned about the inner workings of generative AI tools, used text-to-image generation algorithms to create their imaged future dreams, reflected on the potential benefits and harms of generative AI tools and voiced their opinions about policies for the use of these tools in classrooms. In this paper, we present the learning activities and experiences of 34 high school students who engaged in our workshops. Students reached creative learning objectives by using prompt engineering to create their future dreams, gained technical knowledge by learning the abilities, limitations, text-visual mappings and applications of generative AI, and identified most potential societal benefits and harms of generative AI.

App Planner is an interactive support tool for K-12 students, designed to assist in creating mobile applications. By utilizing generative AI, App Planner helps students articulate the problem and solution through guided conversations via a chat-based interface. It assists them in brainstorming and formulating new ideas for applications, provides feedback on those ideas, and stimulates creative thinking. Here we report usability tests from our preliminary study with high-school students who appreciated App Planner for aiding the app design process and providing new viewpoints on human aspects especially the potential negative impact of their creation.

We examine trends in the Learning at Scale conference from 2014 through 2021. We use an original coding scheme to classify all 142 full papers from five angles: setting, approach, pedagogical strategy, population of interest, and dependent variable. We observe a decline of research on MOOCs, an increase in number of settings studied over time, and a consistent focus on assessment strategies. We then examine other conferences to which Learning at Scale authors contribute research. This paper contributes an original coding scheme to classify future Learning at Scale papers; an analysis of the research focuses at Learning at Scale over time; and an evaluation of the mutual influence between Learning at Scale and other venues. These latter two contributions contextualize learning at scale research within Learning at Scale and the broader research community to show how the field has evolved and to help predict its future directions.

The COVID-19 pandemic has forced the transition of workflows across sectors to digital platforms. In education settings, stakeholders previously reluctant to integrate computing technology in the classroom now find themselves with little choice but to embrace it. This move to the digital brings additional challenges in underserved contexts with limited, intermittent, and shared access to mobile or computing devices and the internet. In this rapidly evolving digital landscape, we investigate how educational institutions (schools and non-profit organizations) working with underserved populations in India are managing the transition to online or remote learning. We conducted twenty remote interviews with students, teachers, and administrators from underserved contexts across India. We found that online learning efforts in this setting relied on a resilient human infrastructure comprised of students, teachers, parents, administrators, and non-profit organizations to help navigate and overcome the limitations of available technical infrastructure. Our research aims to articulate lessons for educational technology design in the post-COVID period, outlining areas for improvement in the design of online learning platforms in resource-constrained settings, and identifying elements of online learning that could be retained to strengthen the education system overall.