This research full paper describes the augmented reality (AR) application AR-Classroom that combines a physical and virtual environment to teach 3D geometric rotations in an engaging and simplified manner. The AR-Classroom contains a virtual workshop where users can perform rotations by manipulating the application's X, Y, and Z-axis sliders to rotate a virtual LEGO model and a physical workshop where users perform rotations using a physical LEGO model. Guided by previous findings and an iterative approach to usability, the present usability study focused on assessing the usability of the AR-Classroom in its most recent version using a new physical LEGO model (i.e., airplane) and reflecting on how discoverability and usability can be assessed using different types of user experience measures and qualitative analysis. Participants were 22 undergraduate students who completed a pre-test with demographic information, watched a video on geometric transformations, and were randomly assigned to interact with either workshop. While interacting with the AR app, participants were instructed to provide feedback and a single ease-of-use question score. Participants then completed a post-test with two measures of usability. Descriptive statistics of the UX measures were explored, and a thematic analysis was conducted to identify and code themes in human-computer interaction. Findings suggest that the AR-Classroom has reached satisfactory usability, and users can navigate the app's features effectively. Discussion includes insight into which aspects of the app need improvement, how to promote self-directed support in the app, the development of future app efficacy experiments, and how to evaluate the usability of AR technology for learning.

This research full paper describes the AR-Classroom application that utilizes augmented reality (AR) and physical and virtual manipulatives to enable undergraduate students to build intuition about the relation between spatial transformations and their mathematical representations. To further build on the app's usability and functionality, additional features are being prototyped to continue improving the user-app interaction with the AR-Classroom. Some of the challenges the students faced when using AR-Classroom were recalling basic matrix operations without geometric context, basic trigonometric functions and their applications in the two-dimensional space, loss of AR registration for not understanding the AR environment, and User Interface (UI) issues. To address these issues, a conversational Artificial Intelligence (AI)-based multi-sensory and interactive assistance has been added to the AR-Classroom. Integrating sophisticated language processing and response generation of AI with immersive three-dimensional capabilities of AR can create a more engaging learning experience than the previous versions of the app. This integration focuses on creating a symbiosis between AR and AI. It creates an elevated user experience by offering real-time, personalized assistance to students dealing with issues related to understanding mathematical concepts and functionalities of the app. A qualitative exploratory usability study was done to assess the user's interaction with the AI implemented in the AR-Classroom, aiming to explore the AI's ability to guide students in using AR technology and aid in introductory matrix algebra learning, to effectively serve the students' learning. Based on the thematic analysis of the user experiment we found four main themes related to users' perceptions of AR-Classroom AI features usability: (1) AI chatbot ease-of-use, (2) Need for answer elaboration from AI, (3) Desire for visual information, and (4) Increased understanding of the content area. The scores of ease of use indicate AI's ability to guide complex tasks in an AR environment using AI features with less concern for the cognitive load. The overall result suggests the need for further investigation on incorporating AI-guided visual cues in an AR environment.

The AR-Classroom application utilizes augmented reality technology (AR) to make the three-dimensional (3D) rotations underlying matrix algebra visible and interactive. The AR-Classroom has physical and virtual versions, where users can perform rotations using a physical LEGO model or by manipulating the application's x, y, and z axes sliders to rotate a virtual model. Both versions provide 3D matrices, color-coded axes lines, and a green wireframe superimposed onto a LEGO model to represent transformations. To ensure that the AR-Classroom makes learning 3D matrix algebra more engaging and accessible, two usability tests were used to evaluate the discoverability and usability of the app. The benchmark test assessed usability in the AR-Classroom's original format, and recommendations were made to improve the app., such as adding additional instructions on model set-up, restructuring, and updating the instructions, and turning the 'visualization type 'function into a button to make it easier to find. After the improvements, the updated usability test assessed usability again so that the impact of the modifications could be evaluated. Participants followed similar procedures in both the benchmark$(\mathrm{N}=12)$and updated usability$(\mathrm{N}=12)$tests. Participants completed a pre-test assessing their math abilities and confidence, watched a video on geometric transformations, and then were randomly assigned to interact with either the physical or virtual version of the app. While interacting with the app, participants were given tasks to complete while thinking out loud and provided an ease-of-use rating from$1=\text{very}$easy to$7=\text{very}$difficult (i.e., SEQ score). Once done interacting with the app, participants completed a post-test assessing their math abilities and confidence and provided feedback on their overall experience with the app (i.e., SUS). A thematic analysis was conducted after each test to identify and code themes in interaction and compare findings from the benchmark and updated tests. Results indicated that after changes were made to the app, the usability of both versions significantly improved: users were better able to set up the space shuttle model, effectively utilize the in-app instructions, and quickly access all of the app's features. Findings from the updated usability test contribute to enhancing the AR-Classroom app and further its use in higher education classrooms for learning matrix algebra.

Project AR-Classroom aims to enhance undergraduate students learning spatial transformations and their mathematical representations. Understanding closely allied spatial and mathematical concepts significantly contributes to STEM learning in fields of computer graphics, computer-aided design, computer vision, robotics, and many more. The technology and learning innovations of this research include novel AR features and their implications for learning. In AR-Classroom, a student can hold and manipulate a 3D physical model (a LEGO space shuttle as an example) while simultaneously interacting with AR visualization of 3D rotations. Two usability tests with 24 participants total have been conducted for AR-Classroom leading to promising results and recommendations for improvements. The project contributes to advancing our knowledge in (1) the role of interplay between physical and virtual manipulatives to engage students in embodied learning and (2) the features of AR to make difficult, invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation.