This research full paper describes the AR-Classroom application that utilizes augmented reality (AR) and physical and virtual manipulatives to enable undergraduate students to build intuition about the relation between spatial transformations and their mathematical representations. To further build on the app's usability and functionality, additional features are being prototyped to continue improving the user-app interaction with the AR-Classroom. Some of the challenges the students faced when using AR-Classroom were recalling basic matrix operations without geometric context, basic trigonometric functions and their applications in the two-dimensional space, loss of AR registration for not understanding the AR environment, and User Interface (UI) issues. To address these issues, a conversational Artificial Intelligence (AI)-based multi-sensory and interactive assistance has been added to the AR-Classroom. Integrating sophisticated language processing and response generation of AI with immersive three-dimensional capabilities of AR can create a more engaging learning experience than the previous versions of the app. This integration focuses on creating a symbiosis between AR and AI. It creates an elevated user experience by offering real-time, personalized assistance to students dealing with issues related to understanding mathematical concepts and functionalities of the app. A qualitative exploratory usability study was done to assess the user's interaction with the AI implemented in the AR-Classroom, aiming to explore the AI's ability to guide students in using AR technology and aid in introductory matrix algebra learning, to effectively serve the students' learning. Based on the thematic analysis of the user experiment we found four main themes related to users' perceptions of AR-Classroom AI features usability: (1) AI chatbot ease-of-use, (2) Need for answer elaboration from AI, (3) Desire for visual information, and (4) Increased understanding of the content area. The scores of ease of use indicate AI's ability to guide complex tasks in an AR environment using AI features with less concern for the cognitive load. The overall result suggests the need for further investigation on incorporating AI-guided visual cues in an AR environment.

The AR-Classroom application utilizes augmented reality technology (AR) to make the three-dimensional (3D) rotations underlying matrix algebra visible and interactive. The AR-Classroom has physical and virtual versions, where users can perform rotations using a physical LEGO model or by manipulating the application's x, y, and z axes sliders to rotate a virtual model. Both versions provide 3D matrices, color-coded axes lines, and a green wireframe superimposed onto a LEGO model to represent transformations. To ensure that the AR-Classroom makes learning 3D matrix algebra more engaging and accessible, two usability tests were used to evaluate the discoverability and usability of the app. The benchmark test assessed usability in the AR-Classroom's original format, and recommendations were made to improve the app., such as adding additional instructions on model set-up, restructuring, and updating the instructions, and turning the 'visualization type 'function into a button to make it easier to find. After the improvements, the updated usability test assessed usability again so that the impact of the modifications could be evaluated. Participants followed similar procedures in both the benchmark$(\mathrm{N}=12)$and updated usability$(\mathrm{N}=12)$tests. Participants completed a pre-test assessing their math abilities and confidence, watched a video on geometric transformations, and then were randomly assigned to interact with either the physical or virtual version of the app. While interacting with the app, participants were given tasks to complete while thinking out loud and provided an ease-of-use rating from$1=\text{very}$easy to$7=\text{very}$difficult (i.e., SEQ score). Once done interacting with the app, participants completed a post-test assessing their math abilities and confidence and provided feedback on their overall experience with the app (i.e., SUS). A thematic analysis was conducted after each test to identify and code themes in interaction and compare findings from the benchmark and updated tests. Results indicated that after changes were made to the app, the usability of both versions significantly improved: users were better able to set up the space shuttle model, effectively utilize the in-app instructions, and quickly access all of the app's features. Findings from the updated usability test contribute to enhancing the AR-Classroom app and further its use in higher education classrooms for learning matrix algebra.

Project AR-Classroom aims to enhance undergraduate students learning spatial transformations and their mathematical representations. Understanding closely allied spatial and mathematical concepts significantly contributes to STEM learning in fields of computer graphics, computer-aided design, computer vision, robotics, and many more. The technology and learning innovations of this research include novel AR features and their implications for learning. In AR-Classroom, a student can hold and manipulate a 3D physical model (a LEGO space shuttle as an example) while simultaneously interacting with AR visualization of 3D rotations. Two usability tests with 24 participants total have been conducted for AR-Classroom leading to promising results and recommendations for improvements. The project contributes to advancing our knowledge in (1) the role of interplay between physical and virtual manipulatives to engage students in embodied learning and (2) the features of AR to make difficult, invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation.

Robust estimation of camera motion under the presence of outlier noisevision. Despite existing efforts that focus on detecting motion and scene degeneracies, the best existing approach that builds on Random Consensus Sampling (RANSAC) still has non-negligible failure rate. Since a single failure can lead to the failure of the entire visual simultaneous localization and mapping, it is important to further improve the robust estimation algorithm. We propose a new robust camera motion estimator (RCME) by incorporating two main changes: a model-sample consistency test at the model instantiation step and an inlier set quality test that verifies model-inlier consistency using differential entropy. We have implemented our RCME algorithm and tested it under many public datasets. The results have shown a consistent reduction in failure rate when comparing to the RANSAC-based Gold Standard approach and two recent variations of RANSAC methods.

To develop a multi-modal in-traffic bridge deck scanning device, we need to estimate the relative pose between a ground penetrating radar (GPR) and a camera. Unlike camera images, GPR output is in a non-Euclidean coordinate system because it only detects underground objects relative to road surface. When road surface is non-planar, its output cannot be trivially mapped to a 3D Cartesian system which is necessary for sensor fusion. Since there is no joint coverage between two sensors due to mounting requirements, we design an artificial planar bridge assisted by a planar mirror as the calibration rig. We combine the pinhole camera model with mirror reflection transformation and model the GPR imaging process. We estimate the camera and mirror poses and extract readings from hyperbolas generated from metal balls. We employ the maximum likelihood estimator to estimate the rigid body transformation between the two sensors and provide the closed form error analysis. We have conducted physical experiments to validate our calibration process and shown the average error of 6.67 mm for our calibration model. The result is satisfying considering the GPR signal wave length is 18.75 cm.