Mixed reality (MR) and AI-powered pedagogical agents have been increasingly explored in STEM labs context as a means of providing hands-on practice with virtual teaching assistants. However, teaching microelectronics presents unique challenges because learners must integrate conceptual knowledge, procedural skills, and visual–spatial understanding to reason about multilayer chip structures and how fabrication processes optimize performance. We introduce ChipXR, a mixed-reality learning environment developed through a design-science approach and multistakeholder participatory design involving instructors, teaching assistants, students, education researchers, and HCI researchers to systematically refine interaction design and reduce potential cognitive load. The final prototype integrates interactive lecture modules for conceptual understanding, embodied MR activities for procedural skill development, and a LEGO-based building task to foster visual–spatial reasoning, all supported by a GPT-powered agent that explains the principles behind each activity. In a controlled study comparing MR+AI agent, MR-only, and traditional video-based lecture, we found no significant differences in conceptual quiz scores, but the MR+AI condition showed trends associated with visual–spatial understanding and perceived knowledge gains. A deployment in an introductory advanced semiconductor packaging course, supplemented by think-aloud sessions and focus groups, further contextualizes these findings. Together, these results suggest how MR+AI systems can be purposefully orchestrated to target complementary knowledge types and inform the design of future microelectronics curricula and other complex STEM domains requiring conceptual, procedural, and spatial reasoning.

Shadows of Tomorrow is an innovative public art installation utilizing LiDAR body tracking and augmented reality to promote climate change awareness. The installation projects depictions of real-world climate change scenarios from Australia, Kuwait, the United States, and Greenland on a large display. Shadows utilizes the Azure Kinect to capture and integrate audience member silhouettes as a simple user interface for the display. As audiences move in front of the display, their silhouette reveals the impact of climate change on the projected environment. By bringing global climate change stories to local audiences, we emphasize the universal, yet highly localized impacts of climate crisis. These interactive visualizations encourage audiences to engage with and understand the stark realities of climate change in regions far removed from their own. Created for display in high-transit public areas like museums, airports, and city centers, Shadows of Tomorrow aims to create a global dialogue around our shared responsibility for climate action.

Augmented reality in public health communications is an under-explored field. Researchers forward Covid Reflections, a public health communications installation which employs augmented reality enhanced with AI LiDAR body tracking to engage public audiences in short duration health-oriented experiences. Covid Reflections helps audiences to visualize potential health outcomes of Covid-19 through depicting the process of disease contraction, sickness, and potential hospitalization on a virtual avatar which mirrors the user’s physical body in real-time. The user is immersed in a “virtual first-hand experience” of Covid-19, and is thus supported in drawing concrete conclusions about the potential personal implications of contracting Covid-19.

The rise of online learning poses unique challenges in music education, where live demonstration and musical synchronization are critical for student success. We present HoloKeys, a music education interface which allows instructors to play remotely located pianos using an augmented reality headset and wifi-enabled microcontrollers. This approach allows students to receive distance education which is more direct, immersive, and comprehensive than conventional video conferencing allows for. HoloKeys enables remote students to observe live instructional demonstration on a physical keyboard in their immediate environment just as they would in traditional settings. HoloKeys consists of two separate components: an augmented reality user interface and a piano playing apparatus. Our system aims to extend online music education beyond desktop platforms into the physical world, thereby addressing crucial obstacles encountered by educators and students transitioning into online education.