Extended Reality (XR) systems for physical skill training have largely emphasized simulation rather than real-time in-situ instruction. We present WeldAR, an Augmented Reality (AR) system with five learning modules that overlays real-time guidance during live welding using a headset integrated into a welding helmet and a torch attachment. We conducted an in-situ within-subjects study with 24 novices, comparing AR guidance to video instruction for live welding across practice and unassisted tests. AR improved performance in both assisted practice and unassisted tests, primarily driven by gains in travel speed and work angle. By offering real-time feedback on four performance measures, AR supported novices in carrying embodied knowledge into independent tasks. Our contributions include: (1) WeldAR for in-situ physical skill training; (2) empirical evidence that AR enhances composite welding performance and key physical skills; and (3) implications for the development of AR systems that support in-situ, embodied skill training in welding and related trades.

Developing fine motor skills requires complex coordination between physical and mental processes, which can be challenging for novices as they manage multiple sensory inputs while executing precise movements. To address this, we developed a proof-of-concept hybrid wearable system that guides mindful breathing through different combinations of AR feedback and air-induced haptic cues. Fifteen participants took part in a mixed-method exploratory study involving paper-based precision tasks inspired by welding. The outcomes indicate that the breathing guide with haptic cues can positively influence task performance and highlight a connection among improved performance, enhanced body awareness, and attention regulation with mindfulness exercises. While initial findings are promising, larger validation studies are needed. This study examines the potential for integrating mindfulness into fine motor skill development through multimodal interaction, suggesting applicability in welding and other skills as a complement to traditional training.

This paper introduces and investigates the use of subtle visual cues in Mixed Reality (MR) interfaces to blend digital content with the physical world, enhancing user experience. With Extended Reality (XR) technologies becoming more common, creating immersive environments that effectively combine digital and physical elements is crucial. Traditional MR methods often inadequately integrate 2D and 3D digital content into physical settings.

Metal welding is a craft manufacturing skill that can be unusually difficult to externalize and represent to novices. Building competency requires an apprentice to iteratively practice embodied skills and sensitize themselves to a sensorially complex practice. To explore these challenges, we organized a series of co-design workshops with a youth program in welding and fabrication. Working with eight instructors and four students, we identified opportunities for mixed reality, sensing, and tinyML processes to augment welding training and practice. This resulted in an extended reality (XR) welding helmet and torch that enhances the embodied learning of welding in three key ways: biometric sensing enhances mindfulness and stress management in sensorially challenging environments; acoustic sensing focuses learner attention on non-visual cues of weld performance; and combined motion-sensing and visual XR feedback helps improve proprioceptive and embodied learning. These features are assessed and we offer design implications for augmenting novice learning of craft-practice with XR approaches.

Hybrid Environments – spaces where physical and digital elements are blended in-situ – have been growing and expanding over the past few decades. As these technologies become woven into our everyday lives, new challenges and opportunities emerge as they transform our experiences in our homes, workplaces, museums, and cities. Designing Hybrid Environments requires training in both the design of physical environments and interactive technologies. Cross-trained students in higher education need to be able to creatively respond and adapt to these emerging landscapes. Students must bridge fundamental knowledge in design, architecture, art, interactivity, and computation. This is a complex pedagogical practice. How are educators meeting this emergent need? The workshop’s primary goals are to (i) form a collective understanding of the current technological and pedagogical challenges to teaching the design and understanding of Hybrid Environments; (ii) bring together educators and practitioners interested in the future of Hybrid Environments, (iii) discuss creativity support tools as well as frameworks, methods, and approaches for teaching emerging technologies, (iv) form a multidisciplinary community to shape the future of pedagogical inquiry and implementation of Hybrid Environments.

Voice assistants have become a mainstay of our increasingly smart domestic spaces. Within this design-through-research inquiry, how new forms of intelligence might creatively respond to everyday interactions. In this, we are particularly interested in non-verbal communication and expression as a design material for interaction design. In this demonstration, we present a prototype illustrating this opportunity: a tinyML-enabled helper that augments the Google Home and enables it to recognize and respond to everyday phenomenon such as a sneeze, a door opening, or rapping on a table. The helper offers a creative toolkit for designers and end-users to quickly explore, understand, and imagine the affordances of everyday sounds.

In this paper, we investigated the potential of using a tangible interaction interface for creative activities as a preliminary step to achieve human-AI co-creation. We conducted a preliminary study to explore sand as a medium for co-creation and identify potential opportunities of AI mediating the process. We created Sand Playground, a co-creation interface that uses sand as an interaction medium. We describe details for building the interface. We selected two opportunities from the study and developed them as two modes, Artistic Mimicry and Zen Garden. We also conducted a workshop study using Sand Playground to evaluate collaboration and the characteristics of the tangible interface. We identified findings responding to drawing strategies, multisensorial experience, and performative and ephemeral quality of creation. We discuss next steps of achieving our ultimate goal: an AI-mediated interface that collaborates with users physically in real-time on sand.