Extended Reality (XR) systems for physical skill training have largely emphasized simulation rather than real-time in-situ instruction. We present WeldAR, an Augmented Reality (AR) system with five learning modules that overlays real-time guidance during live welding using a headset integrated into a welding helmet and a torch attachment. We conducted an in-situ within-subjects study with 24 novices, comparing AR guidance to video instruction for live welding across practice and unassisted tests. AR improved performance in both assisted practice and unassisted tests, primarily driven by gains in travel speed and work angle. By offering real-time feedback on four performance measures, AR supported novices in carrying embodied knowledge into independent tasks. Our contributions include: (1) WeldAR for in-situ physical skill training; (2) empirical evidence that AR enhances composite welding performance and key physical skills; and (3) implications for the development of AR systems that support in-situ, embodied skill training in welding and related trades.

The rapid pace of technological development over the past several decades has led to an exponential rise in electronic waste, or “e-waste.” As devices become obsolete through cycles of innovation and planned obsolescence, the data once contained on these devices fades into obscurity. This project situates itself within ongoing discourses of material circularity and archival media, asking how obsolete technologies might experience renewal in new and creative forms. Specifically, this research explores the intersection of electronic waste, data sonification, and reuse through the design and development of Waste Speaker, a hand-held audio device built from discarded hard disk drives (HDDs). By translating latent data stored within these HDDs into generative sound compositions, Waste Speaker reconfigures the hard drive as both a medium and message of revival. In doing so, this project contributes to broader conversations around defunct media and sustainable design, positioning e-waste as a site of resurgence: no longer “waste,” and instead a medium for sensory engagement and critical reflection.

Developing fine motor skills requires complex coordination between physical and mental processes, which can be challenging for novices as they manage multiple sensory inputs while executing precise movements. To address this, we developed a proof-of-concept hybrid wearable system that guides mindful breathing through different combinations of AR feedback and air-induced haptic cues. Fifteen participants took part in a mixed-method exploratory study involving paper-based precision tasks inspired by welding. The outcomes indicate that the breathing guide with haptic cues can positively influence task performance and highlight a connection among improved performance, enhanced body awareness, and attention regulation with mindfulness exercises. While initial findings are promising, larger validation studies are needed. This study examines the potential for integrating mindfulness into fine motor skill development through multimodal interaction, suggesting applicability in welding and other skills as a complement to traditional training.

Reflecting on a one-year design-research project, we examine design drifts by documenting and analyzing three distinct conceptual and technical shifts that took place. We extend the notion of drifting by framing through stages of unmaking and remaking —where unmaking involves reversals of progress and remaking incorporates these reversals into new research framings. While remaking has been discussed as emergence and reframing, we argue that unmaking during design research is often hidden yet equally significant. This pictorial presents a visual reflection on our artifact's development. We also characterize how a drift may form as a six-step process. We emphasize unmaking's contribution to drifting: it embodies reflection-in-action with materials and helps generate designerly knowledge for future research.

Figure 1: System Workflow: (1) Record Module captures dialog and object interaction while the user is using Rhino3D (2) Recall Module allows user to review all object interactions, generate dialog-based annotations, and manage causal reasoning.

Sidewalk robots are increasingly common across the globe. Yet, their operation on public paths poses challenges for people with mobility disabilities (PwMD) who face barriers to accessibility, such as insufficient curb cuts. We interviewed 15 PwMD to understand how they perceive sidewalk robots. Findings indicated that PwMD feel they have to compete for space on the sidewalk when robots are introduced. We next interviewed eight robotics practitioners to learn about their attitudes towards accessibility. Practitioners described how issues often stem from robotic companies addressing accessibility only after problems arise. Both interview groups underscored the importance of integrating accessibility from the outset. Building on this finding, we held four co-design workshops with PwMD and practitioners in pairs. These convenings brought to bear accessibility needs around robots operating in public spaces and in the public interest. Our study aims to set the stage for a more inclusive future around public service robots.

Metal welding is a craft manufacturing skill that can be unusually difficult to externalize and represent to novices. Building competency requires an apprentice to iteratively practice embodied skills and sensitize themselves to a sensorially complex practice. To explore these challenges, we organized a series of co-design workshops with a youth program in welding and fabrication. Working with eight instructors and four students, we identified opportunities for mixed reality, sensing, and tinyML processes to augment welding training and practice. This resulted in an extended reality (XR) welding helmet and torch that enhances the embodied learning of welding in three key ways: biometric sensing enhances mindfulness and stress management in sensorially challenging environments; acoustic sensing focuses learner attention on non-visual cues of weld performance; and combined motion-sensing and visual XR feedback helps improve proprioceptive and embodied learning. These features are assessed and we offer design implications for augmenting novice learning of craft-practice with XR approaches.

Sidewalk robots are becoming increasingly common worldwide, yet their operation on public walkways presents challenges for pedestrians. This is especially true for people with motor disabilities (PWMD) who already manage obstacles such as inadequate ramps and public incivility. The addition of sidewalk robots could further intensify these difficulties, which poses an urgent need to examine how the design of sidewalk robots may influence the daily navigation experiences of PWMD. This poster illustrates findings from semi-structured interviews with ten PWMD, providing insights into their perspectives on the presence of sidewalk robots. The study uncovers potential conflicts in shared sidewalk use and the adaptive actions PWMD described needing to undertake in response. Interviewees raised concerns about whether the robots could accommodate the needs of PWMD, as compared to people walking on foot, and the repercussions of any shortcomings in this regard. Our research also examines tensions stemming from different robotic design choices, indicating the necessity for more accessible public robot designs. We further delve into PWMD’s interaction needs and modalities for routine operation and in the event of robot malfunction. As cities increasingly allow for the deployment of robots in public spaces, this work seeks to inform equitable design and deployment guidelines for sidewalk robots and calls for further research into the implications of the rise of public robots for the diverse populations that make up any given municipality.

We introduce an interactive tool called "Browsing the Latent Space" (BLS), which explores the potential of latent space visualization and interpolation in 3D generative systems for design space exploration. By visualizing and interpreting the abstract space in which the outputs produced by generative systems operate (the latent space), users gain a better understanding of how the model generates its outputs, and can explore underlying patterns and structures in the data. Using the GET3D generative model, this demonstration showcases how latent space manipulation can enhance the creative process and provides an intuitive and interactive way to explore, remix, and create new chair designs. Allowing interaction with and manipulation of the generative system’s outputs in real-time, this tool supports increased awareness of the latent space as a resource for a designer’s understanding of generative systems and creative exploration.

Generative image AI has sparked heated discussion among creative professionals. However, how it might be a tool for design practice is understudied, and it is not yet understood how designers may find image AI helpful across the stages of design practice. To address this, our preliminary study explores how designers use generative image AI accompanied by design sketches to inform early-stage 3D design. Further, we also examine the perceived limitations of text-to-image models. To do this, we recruited 11 Architecture graduate students with a median work experience of 2 years. Participants completed a design task using generative image AI packages and incorporated design sketches as inputs. The study findings provide insights into how image AI can or can not be a valuable resource for architectural design practitioners. Further, findings suggest possible directions for future image AI-assisted design tools and workflows.

Our everyday technologies could have appeared terrifying to our ancestors: instantaneous disembodied communication, access to knowledge, objects with ‘intelligence’ that talk to us (and each other). Black boxes and intangible entities are omnipresent in our homes and lives without our necessarily understanding the hidden flows of data, unknown agendas, imaginary clouds, and mysterious rules that govern them. Have humanity's ways of relating to the unknown throughout history gone away, or have they perhaps transmuted into new forms? In an ongoing project, we have inventoried examples, encounters and reflections on contemporary technology, framed through the perspective of the haunted, spectral and otherworldly. In this paper, we excerpt this collection to illustrate the value and opportunity of an unfamiliar, disquieting perspective in helping to frame the frictions, beliefs and myths that are emerging around interactions with everyday technologies. We posit and demonstrate ‘spooky technology’ as an accessible framework to reflect and respond to our increasingly entangled relationships with technology.

Hand-based gestural interaction in augmented reality (AR) is an increasingly popular mechanism for spatial interactions. However, it presents many challenges. For example, most hand gesture interactions work well for interactions with virtual content and interfaces, but seldom work with physical devices and users’ environment. To explore this, and rather than inventing new paradigms for AR interactions, this paper revisits Zigelbaum, Kumpf, Vazquez, and Ishii’s 2008 project ‘Slurp’ [72] - a physical eyedropper to interact with digital content from IoT devices. We revive this historical work in a new modality of AR through a five step process: re-presecencing, design experimentation, scenario making, expansion through generative engagements with designers, and reflection. For the designers we engaged, looking back and designing with a restored prototype helped increased understanding of interactive strategies, intentions and rationales of original work. By revisiting Slurp, we also found many new potentials of its metaphorical interactions that could be applied in the context of emerging spatial computing platforms (e.g., smart home devices). In doing so, we discuss the value of mining past works in new domains and demonstrate a new way of thinking about designing interactions in emerging platforms.

Hybrid Environments – spaces where physical and digital elements are blended in-situ – have been growing and expanding over the past few decades. As these technologies become woven into our everyday lives, new challenges and opportunities emerge as they transform our experiences in our homes, workplaces, museums, and cities. Designing Hybrid Environments requires training in both the design of physical environments and interactive technologies. Cross-trained students in higher education need to be able to creatively respond and adapt to these emerging landscapes. Students must bridge fundamental knowledge in design, architecture, art, interactivity, and computation. This is a complex pedagogical practice. How are educators meeting this emergent need? The workshop’s primary goals are to (i) form a collective understanding of the current technological and pedagogical challenges to teaching the design and understanding of Hybrid Environments; (ii) bring together educators and practitioners interested in the future of Hybrid Environments, (iii) discuss creativity support tools as well as frameworks, methods, and approaches for teaching emerging technologies, (iv) form a multidisciplinary community to shape the future of pedagogical inquiry and implementation of Hybrid Environments.

Voice assistants have become a mainstay of our increasingly smart domestic spaces. Within this design-through-research inquiry, how new forms of intelligence might creatively respond to everyday interactions. In this, we are particularly interested in non-verbal communication and expression as a design material for interaction design. In this demonstration, we present a prototype illustrating this opportunity: a tinyML-enabled helper that augments the Google Home and enables it to recognize and respond to everyday phenomenon such as a sneeze, a door opening, or rapping on a table. The helper offers a creative toolkit for designers and end-users to quickly explore, understand, and imagine the affordances of everyday sounds.

Devices for remote communication and sending digital gestures have been widely explored in the field of ubiquitous computing. With the lack of social contact during the COVID-19 pandemic and renewed interest in these devices in mind, we suggest the affordances of a culturally-informed design approach. We present SneezeLove, a device that uses sneezes to communicate that someone is missing you. Users tickle the device’s nose to send an electromechanical ‘sneeze’ to another device. The concept plays on a superstition in Chinese culture that attributes an additional layer of meaning to sneezes, and we found similar beliefs in other cultures that signify that someone out there is thinking of you: hiccups in India, “burning” ears in Ireland. The device, and its design approach, suggests how low-resolution messages can be enlivened and made more personally meaningful by tapping into expressive and specific cultural superstitions.

Traditionally, the setting for maker education has been a physical space where students can engage in hands-on learning and often work collaboratively. However, as many schools adopt a remote learning model due to COVID pandemic conditions, a pressing need has arisen for instructional practices and tools that facilitate project-based distance learning. This work in progress presents the design process of a QR-code enabled learning tool for instructors and students working with electronics in introductory physical computing courses. Through iterative learning design research and deployment in two 3-week micro courses, we uncovered preliminary evidence for the versatility of tactile card decks in supporting disciplinary based learning behaviors, debugging practices, visual and remote communication, as well as the opportunities for enhancing knowledge transfer enabled through digital augmentation.

Spectrality describes the spatiotemporal relationships between landscape and memories where the past ‘haunt’ the present. We explore how spatialized soundscapes may evoke the spectral memories of physical landscape. To do this, we curated a site-specific mixed reality audio walk at Carrie Furnace, a national historic landmark in Pittsburgh’s ‘Rivers of Steel’ National Heritage area. This audio walk leverages virtual soundscapes, mobile mixed reality, and open-ended narratives. Each visitor takes a unique path through the spectral memories of the site’s industrial past and post-industrial relics. Findings from an eleven-person user study assess our approach to spatialized spectral soundscapes and reveal the dimensions (content, context, experience) that resonate with audiences.

Inspired by the Bauhaus material sensory training, we examine the role that texture plays in contemporary design workflows. To do this, we prepare a prototype texture scanning device that allows near real-time sampling of physical textures at microscopic scale. Its implementation as a low-cost, portable, open-source device is first described. Next and through this tool, we examine the value and opportunities created by incorporating 3D textures into design workflows. To do this, we present a series of case studies of scanned materials, and discuss the fidelity of the textures produced, and its rendering of materiality. This explores both typical (wood, fibers) and atypical (bioplastics, state changes) materials. Additionally, three expert users examine these cases and reflect on potential opportunities for material inquiry enabled by this tool. Finally, we discuss how this tool might augment design processes and reintroduce material sensibility training, similar to that of the Bauhaus sensory training.

Language is ambiguous; many terms and expressions can convey the same idea. This is especially true in creative practice, where ideas and design intents are highly subjective. We present a dataset-Ambiguous Descriptions of Art Images (ADARI)-of contemporary workpieces, which aims to provide a foundational resource for subjective image description and multimodal word disambiguation in the context of creative practice. The dataset contains a total of 240k images labeled with 260k descriptive sentences. It is additionally organized into sub-domains of architecture, art, design, fashion, furniture, product design and technology. In subjective image description, labels do not necessarily correspond to well-defined entities i.e. cars, quantitative attributes such as the color red, or actions like playing. For example, the ambiguous label dynamic is a qualitative attribute of an extensive amount of objects and thus, the data's variance is high. To understand this complexity, we analyze the ambiguity and relevance of text with respect to images using the state-of-the-art pre-trained BERT model for sentence classification. We provide a baseline for multi-label classification tasks and demonstrate the potential of multimodal approaches for understanding ambiguity in design intentions. We hope that ADARI dataset and baselines constitute a first step towards subjective label classification.

In response to recent calls for HCI to address ongoing environmental crises and existential threats, this paper introduces the concept of collaborative survival and examines how it shapes the design of interactive artifacts. Collaborative survival describes how our (human) ability to persist as a species is deeply entangled with and dependent upon the health of a multitude of other species. We explore collaborative survival within the context of designing tools for mushroom foraging and reflect on how interactive products can open new pathways for noticing and joining-with these entanglements towards preferable futures. In addition to highlighting three tactics-engagement, attunement and expansion-that can guide designs towards multispecies flourishing, our prototypes illustrate the potential for wearable technology to extend the body into the environment.

Many teens struggle with school writing and particularly in identifying personally relevant topics, which motivate writing improvement. However, a wealth of potential topics is available through their social media data. We explore the design of Sparkfolio, a prewriting support tool that aims to help students successfully prepare meaningful writing topics from their own social media content. This web-delivered tool was evaluated with 46 teen users in the context of their high school English class. Findings demonstrate that the use of social media can enhance the quality of written outcomes, as Sparkfolio users had significantly greater gains than a control group.

The Arts and Digital Culture program has offered a high quality forum for the presentation of interactive and arts-based multimedia applications at the annual ACM Multimedia conference for over a decade. This tutorial will explore the evolution of this program as a guide to new authors considering future participation in this program. By surveying both past technical and past exhibited contributions, this tutorial will offer guidance to artists, researchers and practitioners on success at this multifaceted, interdisciplinary forum at ACM Multimedia.

The Arts and Digital Culture program has offered a high quality forum for the presentation of interactive and arts-based multimedia applications at the annual ACM Multimedia conference for over a decade. This tutorial will explore the evolution of this program as a guide to new authors considering future participation in this program. By surveying both past technical and past exhibited contributions, this tutorial will offer guidance to artists, researchers and practitioners on success at this multifaceted, interdisciplinary forum at ACM Multimedia.

Mobile exertion games offer potential to increase physical activity levels by combining exercise with play. Yet, studies indicate that engagement and use of these games wane over time, reducing their effectiveness for promoting healthy physical activity and long-term behavior change. In this paper, we discuss the importance of non-parallel and multiplayer play modes as mechanisms to increase teen motivation to engage in non-exercise forms of physical activity. We conclude with an introduction to Scatter, a new mobile exertion game emphasizing non-parallel gameplay.

Lifelog collections digitally capture and preserve personal experiences and can be mined to reveal insights and understandings of individual significance. These rich data sources also offer opportunities for learning and discovery by motivated third parties. We employ a custom-designed storytelling application in constructing meaningful lifelog summaries from third-party perspectives. This storytelling initiative was implemented as a core component in a university media-editing course. We present promising results from a preliminary study conducted to evaluate the utility and potential of our approach in creatively interpreting a unique experiential dataset.