Extended reality (XR) is well suited to support the situated learning of technical procedures. At the same time, AI-driven intelligent tutoring systems (ITS) can complement XR by providing adaptive pedagogical support. Many domains would benefit from this combination, especially when trainers, equipment, or team members are limited. We present a domain-agnostic XR-based ITS that integrates a training procedure representation (TPR), XR simulation, and an LLM-driven instructor. We demonstrate the tutor's use for tissue sample handling and engine repair, showing how it delivers adaptive feedback, collaborative roleplay, and dynamic scenario management to create realistic and pedagogically meaningful training experiences.

Social skills training targets behaviors necessary for success in social interactions. However, traditional classroom training for such skills is often insufficient to teach effective communication — one-to-one interaction in real-world scenarios is preferred to lecture-style information delivery. This paper introduces a framework that allows instructors to collaborate with large language models to dynamically design realistic scenarios for students to communicate. Our framework uses these scenarios to enable student rehearsal, provide immediate feedback and visualize performance for both students and instructors. Unlike traditional intelligent tutoring systems, instructors can easily co-create scenarios with a large language model without technical skills. Additionally, the system generates new scenario branches in real time when existing options don't fit the student's response.

Reading skills are crucial for students' success in education and beyond. However, reading proficiency among K-12 students has been declining globally, including in Sweden, leaving many underprepared for post-secondary education. Additionally, an increasing number of students have reading disorders, such as dyslexia, which require support. Generative artificial intelligence (genAI) technologies, like ChatGPT, may offer new opportunities to improve reading practices by enhancing the readability of educational texts. This study investigates whether ChatGPT-4 can simplify academic texts and which prompting strategies are most effective. We tasked ChatGPT to re-write 136 academic texts using four prompting approaches: Standard, Meta, Roleplay, and Chain-of-Thought. All four approaches improved text readability, with Meta performing the best overall and the Standard prompt sometimes creating texts that were less readable than the original. This study found variability in the simplified texts, suggesting that different strategies should be used based on the specific needs of individual learners. Overall, the findings highlight the potential of genAI tools, like ChatGPT, to improve the accessibility of academic texts, offering valuable support for students with reading difficulties and promoting more equitable learning opportunities.

Limited training capacity has contributed to a critical shortage of licensed commercial pilots.Adaptive educational technologies and simulators could alleviate current training bottlenecks if these technologies could assess trainee performance and provide appropriate feedback.Agents can be used to assess trainee performance, but there is insufficient guidance on how to provide concurrent feedback in simulation-based learning environments.So, we designed 4 feedback conditions that provide varying degrees of elaboration and used a within-subject study (𝑛 = 20) to compare feedback approaches.Trainee performance was best when they received highly-elaborative feedback that modeled expert behaviour.Variability in participant performance and preferences indicates a need to adapt the feedback type to individual learners and provides insight into the use of concurrent feedback in simulation-based learning environments.Specifically, learners appreciated the expert model because it facilitated a sense of control which was associated with lower negative affect and lower extraneous cognitive load.

Educational technologies have been argued to enhance specific aspects of affect, such as motivation, and through that learner experiences and outcomes. Until recently, affect has been considered separately from cognition. In this study, we investigated how learner affect (valence and activation) was tied to learner cognitive load and behaviours during game-based literacy activities. We employed experience sampling as part of a lab-based case study where 35 English language learners used an adaptive educational game. The results indicated that both positive and negative affect predicted learner cognitive load, with negative affect predicting extraneous (unnecessary) load. These results and the newly identified interaction patterns that accompanied learner affect and cognitive-load trajectories provide insight into the role of affect during learning. They show a need for considering affect when studying cognitive load and have implications for how systems should adapt to learners.

We propose an AI-based pilot trainer to help students learn how to fly aircraft. First, an AI agent uses behavioral cloning to learn flying maneuvers from qualified flight instructors. Later, the system uses the agent's decisions to detect errors made by students and provide feedback to help students correct their errors. This paper presents an instantiation of the pilot trainer. We focus on teaching straight and level flying maneuvers by automatically providing formative feedback to the human student.

Teaching assistants (TAs) are essential members of post-secondary instructional teams, who often have considerable student-facing time. The recently increasing demand for introductory computer-science (CS) courses has resulted in a corresponding increased demand for TAs. The existing TA literature has predominantly focused on investigating the role of training in the TA experience, particularly with respect to performance. This provides little insight into how TAs experience their management. Consequently, we investigate the role of management in the experiences of introductory-level CS TAs. We provide the structure for a formal, tiered management scheme employed in a high-enrolment introductory CS course. This scheme attempts to address the challenges associated with managing TAs at scale. As a case study, we compare the experience of TAs under this new management scheme with that of TAs under an informal, unstructured management style used in smaller introductory CS courses. Specifically, we used questionnaires to look at TAs' self-efficacy and understand their experiences. While we did not find a significant difference in TA self-efficacy between the two management styles, thematic analysis of the open-response data revealed a greater number of challenges reported by the TAs under the tiered management system. TAs characterized this tiered system as "organized" and frequently reported feeling overworked. Across both groups, TAs identified improved communication and additional training as factors that could improve their experience. Our findings suggest self-efficacy was not a sufficient measure to quantify TA experience. We propose framing TA experience based on their motivation and general well-being.

Daniela Teodorescu, Josie Matalski, Delaney Lothian, Denilson Barbosa, Carrie Demmans Epp. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

The combination of tree search and neural networks has achieved super-human performance in challenging domains. We are interested in transferring to humans the knowledge these learning systems generate. We hypothesize the process in which neural-guided tree search algorithms learn how to solve a set of problems can be used to generate curricula for helping human learners. In this paper we show how the Bootstrap learning system can be modified to learn curricula for humans in a puzzle domain. We evaluate our system in two curriculum learning settings. First, given a small set of problem instances, our system orders the instances to ease the learning process of human learners. Second, given a large set of problem instances, our system returns a small ordered subset of the initial set that can be presented to human learners. We evaluate our curricula with a user study where participants learn how to solve a class of puzzles from the game `The Witness.' The user-study results suggest one of the curricula our system generates compares favorably with simple baselines and is competitive with the curriculum from the original `The Witness' game in terms of user retention and effort.

A large part of health education involves acquiring operational skills that require continued cooperation from the sense of sight. Traditional skill-training systems heavily rely on post-task evaluation and cannot automatically identify the tacit knowledge (e.g., visual attention allocation) that is used to support the decision-making and tool manipulation process. Recent research has started addressing this issue by identifying learner behaviours to understand learner needs and respond with personalized real-time support. To enable this type of adaptivity when training people in how to give injections, we conducted a study to investigate user's visual attention. We analyzed the eye-tracking data of 8 skilled medical practitioners and 10 student nurses from when they were performing an injection task on a simulated abdominal pad. Differences in the detected visual attention and eye-scan patterns across expertise levels create an opportunity for using overlay models or other diagnosis approaches to provide adaptive support to nurse trainees.

Leticia Farias Wanderley, Nicole Zhao, Carrie Demmans Epp. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Abstract Knowledge monitoring is a component of metacognition which can help students regulate their own learning. In adaptive learning software, the system’s model of the student can be presented as an open learner model (OLM) which is intended to enable monitoring processes. We explore how presenting alignment, between students’ self-assessed confidence and the system’s model of the student, supports knowledge monitoring. When students can see their confidence and their performance (either combined within one skill meter or expanded as two separate skill meters), their knowledge monitoring and performance improves, particularly for low-achieving students. These results indicate the importance of communicating the alignment between the system’s evaluation of student performance and student confidence in the correctness of their answers as a means to support metacognitive skills.

Seniors are increasingly using the Internet. However, their adoption of available services such as social media is often restricted by their limited experience with new technologies. At the same time, there is significant interest in designing communication applications, especially mobile, that improve seniors' social connectedness. These are mostly implemented as dedicated social networking tools for seniors and their families. A barrier to the full adoption of such tools is the requirement for younger family members to actively manage a platform parallel to the social media tools they already use (e.g., Facebook). We propose PhotoDigest -- a user-centred application that allows seniors to passively engage in their families' social media activities. PhotoDigest automatically harvests families' Facebook photo posts and delivers them to seniors as weekly digests. We conducted a preliminary deployment study and show that PhotoDigest is easily adopted by seniors, does not interfere with younger generations' life routines, and enhances the entire family's social connectedness.

Reading a book together with a family member who has impaired vision or other difficulties reading is an important social bonding activity. However, for the person being read to, there is little support in making these experiences repeatable. While audio can easily be recorded, synchronizing it with the text for later playback requires the use of forced alignment algorithms, which do not perform well on amateur read-aloud speech. We propose a human-in-the-loop approach to augmenting such algorithms, in the form of touch metaphors during collocated read-aloud sessions using tablet e-readers. The metaphor is implemented as a finger-follows-text tracker. We explore how this could better handle the variability of amateur reading, which poses accuracy challenges for existing forced alignment techniques. Data collected from users reading aloud as assisted by touch metaphors show increases in the accuracy of forced alignment algorithms and reveal opportunities for how to better support reading aloud.

We tackle the prediction of instructor intervention in student posts from discussion forums in Massive Open Online Courses (MOOCs). Our key finding is that using automatically obtained discourse relations improves the prediction of when instructors intervene in student discussions, when compared with a state-of-the-art, feature-rich baseline. Our supervised classifier makes use of an automatic discourse parser which outputs Penn Discourse Treebank (PDTB) tags that represent in-post discourse features. We show PDTB relation-based features increase the robustness of the classifier and complement baseline features in recalling more diverse instructor intervention patterns. In comprehensive experiments over 14 MOOC offerings from several disciplines, the PDTB discourse features improve performance on average. The resultant models are less dependent on domain-specific vocabulary, allowing them to better generalize to new courses.

Limited literacy and visual impairment reduce the ability of many to read on their own. Current e-reader solutions rely on either unnatural synthetic voices or professionally produced audio e-books. Neither provide the same enjoyment as having a family member read to a user, especially when the user requires assistive reading (following printed text while listening to it being read). Unfortunately, the support for non-commercial production of such e-books is limited and requires significant effort. We evaluate a novel, assistive mobile interaction technique that facilitates the recording of audio e-books and their synchronization with the read text. We show that a technique based on a finger tracking metaphor provides optimal support with respect to reading speed. These human-in-the-loop, adaptive techniques can now be used to reduce the content-creation burden that is associated with supporting those who cannot read on their own.

Many people who do not know English have moved to English-speaking countries to learn English. Once there, they learn English through formal and informal methods. While considerable work has studied the experiences of English language learners in different learning environments, we have yet to see analytics that detail the experiences of this population within formal and informal learning environments. This study used the experience sampling methodology to capture the information that is needed to detail the communication and affective experiences of advanced English language learners. The collected data reveals differences in how English language learners perceived their communication success based on their learning context, with higher levels of communicative success experienced in formal learning settings. No such differences were found for learners', highly negative, affect. The data suggest a need for additional emotional support within formal and informal learning environments as well as a need for oral communication support within informal contexts.

We introduce an independent open learner model for formative assessment and learning analytics based on developments in technology use in learning, also maintaining more traditional numerical and qualitative feedback options.

Many people cannot communicate effectively with those around them. The causes vary but several tools and strategies can support their communication. These tools, which collectively fall under the banner of Assistive and Augmentative Communication (AAC), are rarely adaptive. Of those that are, few provide context-based or just-in-time vocabulary support to users even though the proliferation of smartphones makes this possible. To meet this need, we developed four algorithms to retrieve relevant vocabulary from Internet-based corpora. We used discourse completion tasks to evaluate each algorithm's ability to identify appropriate vocabulary across a set of specific contexts. The results indicate that our approach identifies appropriate context-specific words that complement general AAC vocabularies: when combined with a typical base vocabulary, the algorithms outperformed the support provided by the base vocabulary alone. They did this by adding small targeted vocabularies.

Video lecture capture is rapidly being deploying in higher-education institutions as a means of increasing student learning, outreach, and experience. Understanding how learners use these systems and relating this use back to pedagogical and institutional goals is a hard issue that has largely been unexplored. This work describes a novel web-based lecture presentation system which contains fine-grained user tracking features. These features, along with student surveys, have been used to help analyse the behaviour of hundreds of students over an academic term, quantifying both the learning approaches of students and their perceptions on learning with lecture capture.