Physical computing engages learners in computational thinking through building interest-driven physical interactive projects. However, traditional electronic approaches require deep understanding of circuits and embedded systems — a challenge for young learners before meaningful engagement with computational concepts. This pictorial demo- nstrates an alternative, computer vision (CV) approach using fiducial markers and craft materials, through a workshop model where learners design custom controllers for video games. With learner stories from workshops with learners aged 10-15, and emerging themes of computational, material, and social moments, we illustrate how CV-driven physical comp-uting can be an effective, low-barrier, introduction to computing that preserves the creative learning benefits of physical computing.

Tangible User Interfaces (TUIs) that integrate digital information with physical interaction require specialized hardware and complex calibration, limiting their adoption in portable or mobile display systems. This paper introduces ArUcoTUI, a computer vision (CV) toolkit for prototyping tangible interactions on portable screens, leveraging standard cameras and the OpenCV library. ArUcoTUI uses ArUco fiducial markers to detect physical inputs. The software toolkit offers streamlined calibration, a signal processing pipeline, and a client application that translates tangible input into structured events for use in HCI applications. Using a conventional camera in a top-down setting with a flat-panel display, we demonstrate how this toolkit supports the development of interactive surface TUIs with advanced features, including 3D spatial interaction, multi-device interaction, and actuated tangibles within applications. We describe the software implementation, which utilizes accessible hardware to support the development of these tangible interactions. We provide the results of a preliminary evaluation with users, including design implications and suggestions for future research and development.

Outdated Computer Vision (CV) toolkits for Tangible User Interfaces (TUI) have led to fragmented practices, diminished reproducibility, and reduced community support. This paper examines the past, present, and future trajectory of CV-TUI toolkits. Our scoping review of ACM literature (N=120) reveals a divergence between applications using the limited interactions of established toolkits like ReacTIVision and the fragmented, bespoke systems built for complex interactions, highlighting the need for advanced toolkits that enable accessible making. We found how ReacTIVision-based projects were primarily used in stationary tabletop settings with standard token events, while non-ReacTIVision projects explored unique interaction techniques in diverse configurations. Reflecting on these insights, we offer six suggestions for building the next-generation CV-TUI toolkits to support rapid prototyping of tangible interactions. This study provides the TUI community with an updated perspective to inform future research.

Driven by the vision of future responsive environments, where everyday surroundings can perceive human behaviors and respond through intelligent robotic actuation, we propose Wizard of Props (WoP): a human-centered design workflow for creating expressive, implicit, and meaningful interactions. This collaborative experience prototyping approach integrates full-scale physical props with Mixed Reality (MR) to support ideation, prototyping, and rapid testing of responsive environments. We present two design explorations that showcase our investigations of diverse design solutions based on varying technology resources, contextual considerations, and target audiences. Design Exploration One focuses on mixed environment building, where we observe fluid prototyping methods. In Design Exploration Two, we explore how novice designers approach WoP, and illustrate their design ideas and behaviors. Our findings reveal that WoP complements conventional design methods, enabling intuitive body-storming, supporting flexible prototyping fidelity, and fostering expressive environment-human interactions through in-situ improvisational performance.

Computational thinking has been identified as an important skill for children to learn in the 21st century, and many innovative kits and tools have been developed to integrate it into children’s learning. Yet, most solutions require the use of devices like computers or other expensive hardware, thus being inaccessible to low-income schools and communities. We present Cartoonimator, a low-cost, paper-based computational kit for children to create animations and engage with computational thinking. Cartoonimator requires only paper and a smartphone to use, offering an affordable learning experience. Children can draw the scenes and characters for their animation on the paper, which is printed with computer vision markers. We developed the mobile web app to provide an interface to capture keyframes and compile them into animations. In this paper, we describe the implementation and workflow of Cartoonimator, its deployment with children at a local STEAM event, and a planned evaluation for the kit.

The electronics-centered approach to physical computing presents challenges when designers build tangible interactive systems due to its inherent emphasis on circuitry and electronic components. To explore an alternative physical computing approach we have developed a computer vision (CV) based system that uses a webcam, computer, and printed fiducial markers to create functional tangible interfaces. Through a series of design studios, we probed how designers build tangible interfaces with this CV-driven approach. In this paper, we apply the annotated portfolio method to reflect on the fifteen outcomes from these studios. We observed that CV markers offer versatile materiality for tangible interactions, afford the use of democratic materials for interface construction, and engage designers in embodied debugging with their own vision as a proxy for CV. By sharing our insights, we inform other designers and educators who seek alternative ways to facilitate physical computing and tangible interaction design.

Fostering data visualization literacy (DVL) as part of childhood education could lead to a more data literate society. However, most work in DVL for children relies on a more formal educational context (i.e., a teacher-led approach) that limits children's engagement with data to classroom-based environments and, consequently, children's ability to ask questions about and explore data on topics they find personally meaningful. We explore how a curiosity-driven, child-led approach can provide more agency to children when they are authoring data visualizations. This paper explores how informal learning with crafting physicalizations through play and curiosity may foster increased literacy and engagement with data. Employing a constructionist approach, we designed a do-it-yourself toolkit made out of everyday materials (e.g., paper, cardboard, mirrors) that enables children to create, customize, and personalize three different interactive visualizations (bar, line, pie). We used the toolkit as a design probe in a series of in-person workshops with 5 children (6 to 11-year-olds) and interviews with 5 educators. Our observations reveal that the toolkit helped children creatively engage and interact with visualizations. Children with prior knowledge of data visualization reported the toolkit serving as more of an authoring tool that they envision using in their daily lives, while children with little to no experience found the toolkit as an engaging introduction to data visualization. Our study demonstrates the potential of using the constructionist approach to cultivate children's DVL through curiosity and play.

Alternative Controllers (Alt Controls) enable game designers to creatively explore how humans interact with games and challenge the status-quo of game interfaces. Alt Controls, however, require technical skills and fabrication infrastructure that often make them inaccessible to the average designer. Tangible User Interface researchers stand to benefit from the unique approach that Alt Controls promote. My research aims to bridge the gap between game developers and Alt Controls through the use of everyday materials and crafting techniques. In this paper, I discuss a framework for physical computing that uses computer vision (Beholder) and an example introductory platform for Alt Controller design (TinyCade). Further research will refine this framework and incorporate the perspective of other game designers.

Tinycade is a platform designed to help game designers build their own mini arcade games by hand. With this platform, one can craft functioning game controllers out of everyday materials such as cardboard and toothpicks. By utilizing computer vision markers, we can create a variety of inputs without a single wire. In this pictorial, we discuss the functionality of Tinycade and showcase three games that demonstrate the variety of controls possible with this platform.

Fostering data literacy has largely been the domain of formal educational systems and export-oriented tools. Informal educational approaches, such as games or family activities, may overcome barriers to engaging with data by fostering data literacy through casual engagement. This work in progress explores how informal learning through creation and play with interactive data representations (physicalizations) can foster increased literacy and engagement with data. We outline a series of DIY paper charts using AR markers and everyday materials to help children interact and explore data through the creative process of making.

This pictorial presents a design investigation at the intersection of paper and computer vision for tangible interfaces. Through this exploration, we uncovered various characteristics of paper that connect tangible interactions with concealing and revealing printed fiducial markers for detection-particularly through the affordances of paper craft and fiber. We illustrate a variety of paper structures that construct and deconstruct fiducial markers. We also demonstrate how these structures enable untethered functional physical inputs, such as push buttons and sliders. We showcase four proposals that extend these material insights into tangible interface applications, including interactive data physicalizations and functional paper prototypes. Furthermore, we continue the legacy of pictorials by exposing fabrication drawings for others to engage with this work at a more practical level.

The Telling Board uses the concept of storyboarding (organizing graphics in sequence) to enable children ages 6 and up to conceptualize and build their own narratives. By populating square grids on the board with their choice of either pre-illustrated or their own hand drawn "story cards," children practice crafting stories in sequence from beginning to end. They are able to tell those stories and share them via a built-in recording mechanism on the board.