Tangible User Interfaces (TUIs) that integrate digital information with physical interaction require specialized hardware and complex calibration, limiting their adoption in portable or mobile display systems. This paper introduces ArUcoTUI, a computer vision (CV) toolkit for prototyping tangible interactions on portable screens, leveraging standard cameras and the OpenCV library. ArUcoTUI uses ArUco fiducial markers to detect physical inputs. The software toolkit offers streamlined calibration, a signal processing pipeline, and a client application that translates tangible input into structured events for use in HCI applications. Using a conventional camera in a top-down setting with a flat-panel display, we demonstrate how this toolkit supports the development of interactive surface TUIs with advanced features, including 3D spatial interaction, multi-device interaction, and actuated tangibles within applications. We describe the software implementation, which utilizes accessible hardware to support the development of these tangible interactions. We provide the results of a preliminary evaluation with users, including design implications and suggestions for future research and development.

Outdated Computer Vision (CV) toolkits for Tangible User Interfaces (TUI) have led to fragmented practices, diminished reproducibility, and reduced community support. This paper examines the past, present, and future trajectory of CV-TUI toolkits. Our scoping review of ACM literature (N=120) reveals a divergence between applications using the limited interactions of established toolkits like ReacTIVision and the fragmented, bespoke systems built for complex interactions, highlighting the need for advanced toolkits that enable accessible making. We found how ReacTIVision-based projects were primarily used in stationary tabletop settings with standard token events, while non-ReacTIVision projects explored unique interaction techniques in diverse configurations. Reflecting on these insights, we offer six suggestions for building the next-generation CV-TUI toolkits to support rapid prototyping of tangible interactions. This study provides the TUI community with an updated perspective to inform future research.

Developing whole-body tactile skins for robots remains a challenging task, as existing solutions often prioritize modular, one-size-fits-all designs, which, while versatile, fail to account for the robot's specific shape and the unique demands of its operational context. In this work, we introduce GenTact Toolbox, a computational pipeline for creating versatile wholebody tactile skins tailored to both robot shape and application domain. Our method includes procedural mesh generation for conforming to a robot's topology, task-driven simulation to refine sensor distribution, and multi-material 3D printing for shape-agnostic fabrication. We validate our approach by creating and deploying six capacitive sensing skins on a Franka Research 3 robot arm in a human-robot interaction scenario. This work represents a shift from “one-size-fits-all” tactile sensors toward context-driven, highly adaptable designs that can be customized for a wide range of robotic systems and applications. The project website is available at https://hiro-group.ronc.one/gentacttoolbox

Using information and communication technology (ICT) while solo dining can alleviate loneliness but may also lead to unhealthy food choices and decreased satiety. Mindful eating practices are needed to raise awareness about the impact of ICT technology on eating behavior. Our work aims to use technology to augment physical practices and generate physical artifacts by physicalizing sensor data and letting users self-annotate and make sense of the sensor data. With a toolkit that supports constructive physicalization of inertia sensor data related to the users' eating behavior, users increase their awareness of their mundane routine of solo dining with ICT devices by building personalized data physicalization and adding annotations. Results of a five-day preliminary study indicate how annotating data physicalization could be a beneficial intervention for promoting mindful eating behavior in a personal context.

Network visualizations are understudied in graphical perception. As a result, most network visualization designs still largely rely on designer intuition and algorithm optimizations rather than being guided by knowledge of human perception. The lack of perceptual understanding of network visualizations also limits the generalizability of past empirical evaluations, given their focus on performance over causal interpretation. To bridge this gap between perception and network visualization, we introduce a framework highlighting five key perceptual mechanisms used in node-link diagrams and adjacency matrices: attention, visual search, perceptual organization, ensemble coding, and object recognition. Our framework describes the role these perceptual mechanisms play in common network analytical tasks. We use the framework to revisit four past empirical investigations and outline future design experiments that can help produce more perceptually effective network visualizations. We anticipate this connection will afford translational understanding to guide more effective network visualization design and offer hypotheses for perception-aware network visualizations.

Annotations are a critical component of visualizations, helping viewers interpret the visual representation and highlighting critical data insights. Despite their significant role, we lack an understanding of how annotations can be incorporated into other data representations, such as physicalizations and sonifications. Given the emergent nature of these representations, sonifications, and physicalizations lack formalized conventions (e.g., design space, vocabulary) that can introduce challenges for audiences to interpret the intended data encoding. To address this challenge, this work focuses on how annotations can be more tightly integrated into the design process of creating sonifications and physicalizations. In an exploratory study with 13 designers, we explore how visualization annotation techniques can be adapted to sonic and physical modalities. Our work highlights how annotations for sonification and physicalizations are inseparable from their data encodings.

Interaction is critical for data analysis and sensemaking. However, designing interactive physicalizations is challenging as it requires cross-disciplinary knowledge in visualization, fabrication, and electronics. Interactive physicalizations are typically produced in an unstructured manner, resulting in unique solutions for a specific dataset, problem, or interaction that cannot be easily extended or adapted to new scenarios or future physicalizations. To mitigate these challenges, we introduce a computational design pipeline to 3D print network physicalizations with integrated sensing capabilities. Networks are ubiquitous, yet their complex geometry also requires significant engineering considerations to provide intuitive, effective interactions for exploration. Using our pipeline, designers can readily produce network physicalizations supporting selection-the most critical atomic operation for interaction-by touch through capacitive sensing and computational inference. Our computational design pipeline introduces a new design paradigm by concurrently considering the form and interactivity of a physicalization into one cohesive fabrication workflow. We evaluate our approach using (i) computational evaluations, (ii) three usage scenarios focusing on general visualization tasks, and (iii) expert interviews. The design paradigm introduced by our pipeline can lower barriers to physicalization research, creation, and adoption.

The widespread adoption of 3D printers exacerbates existing environmental challenges as these machines increase energy consumption, waste output, and the use of plastics. Material choice for 3D printing is tightly connected to these challenges, and as such researchers and designers are exploring sustainable alternatives. Building on these efforts, this work explores using spent coffee grounds as a sustainable material for prototyping with 3D printing. This material, in addition to being compostable and recyclable, can be easily made and printed at home. We describe the material in detail, including the process of making it from readily available ingredients, its material characteristics and its printing parameters. We then explore how it can support sustainable prototyping practices as well as HCI applications. In reflecting on our design process, we discuss challenges and opportunities for the HCI community to support sustainable prototyping and personal fabrication. We conclude with a set of design considerations for others to weigh when exploring sustainable materials for 3D printing and prototyping.

Data visualization literacy (DVL) is increasingly important in navigating today’s world. Young students are now required to develop data visualization skills (e.g., identifying patterns, collecting, organizing, and analyzing data). Past work focuses on cultivating children’s DVL through playful and gamified approaches. However, these past solutions require hardware (e.g., smartphones) that is not readily replicable within standard classrooms. In addition, the extent of embodied interactions found in past solutions is fairly limited to large-scale interfaces. To address these challenges, we explore how to cultivate children’s DVL through embodied learning that is more apt for a classroom environment. Using a standard Google Chromebook, we designed an interface that utilizes the tracking of a fiducial marker on a wooden arrow to allow a student to author their own bar graphs. We conducted Wizard of Oz tests (n = 3) to gather feedback on the functionality of our design. This project aims to display the benefits of giving students a mode to develop their own visualizations and broaden how they can interact with data.

The electronics-centered approach to physical computing presents challenges when designers build tangible interactive systems due to its inherent emphasis on circuitry and electronic components. To explore an alternative physical computing approach we have developed a computer vision (CV) based system that uses a webcam, computer, and printed fiducial markers to create functional tangible interfaces. Through a series of design studios, we probed how designers build tangible interfaces with this CV-driven approach. In this paper, we apply the annotated portfolio method to reflect on the fifteen outcomes from these studios. We observed that CV markers offer versatile materiality for tangible interactions, afford the use of democratic materials for interface construction, and engage designers in embodied debugging with their own vision as a proxy for CV. By sharing our insights, we inform other designers and educators who seek alternative ways to facilitate physical computing and tangible interaction design.

Fostering data visualization literacy (DVL) as part of childhood education could lead to a more data literate society. However, most work in DVL for children relies on a more formal educational context (i.e., a teacher-led approach) that limits children's engagement with data to classroom-based environments and, consequently, children's ability to ask questions about and explore data on topics they find personally meaningful. We explore how a curiosity-driven, child-led approach can provide more agency to children when they are authoring data visualizations. This paper explores how informal learning with crafting physicalizations through play and curiosity may foster increased literacy and engagement with data. Employing a constructionist approach, we designed a do-it-yourself toolkit made out of everyday materials (e.g., paper, cardboard, mirrors) that enables children to create, customize, and personalize three different interactive visualizations (bar, line, pie). We used the toolkit as a design probe in a series of in-person workshops with 5 children (6 to 11-year-olds) and interviews with 5 educators. Our observations reveal that the toolkit helped children creatively engage and interact with visualizations. Children with prior knowledge of data visualization reported the toolkit serving as more of an authoring tool that they envision using in their daily lives, while children with little to no experience found the toolkit as an engaging introduction to data visualization. Our study demonstrates the potential of using the constructionist approach to cultivate children's DVL through curiosity and play.

Data physicalization enables people to represent and interact with data physically rather than digitally. Physical representations afford visual analysis in comparable ways to traditional, desktop-based visualization by introducing new capabilities, such as facilitating tactile manipulation, accessible interactions, and immersion, that are beyond traditional 2D visualizations. However, physicalization has historically been a niche aspect of visualization research due to its unique challenges. In this paper, I discuss the current challenges of data physicalization and address three areas where data physicalization can aid other research thrusts: broadening participation, supporting analytics, and promoting creative expression. This paper exemplifies each approach through the lens of my work.

Designing a data physicalization requires a myriad of different considerations. Despite the cross-disciplinary nature of these considerations, research currently lacks a synthesis across the different communities data physicalization sits upon, including their approaches, theories, and even terminologies. To bridge these communities synergistically, we present a design space that describes and analyzes physicalizations according to three facets: context (end-user considerations), structure (the physical structure of the artifact), and interactions (interactions with both the artifact and data). We construct this design space through a systematic review of 47 physicalizations and analyze the interrelationships of key factors when designing a physicalization. This design space cross-pollinates knowledge from relevant HCI communities, providing a cohesive overview of what designers should consider when creating a data physicalization while suggesting new design possibilities. We analyze the design decisions present in current physicalizations, discuss emerging trends, and identify underlying open challenges.

Fostering data literacy has largely been the domain of formal educational systems and export-oriented tools. Informal educational approaches, such as games or family activities, may overcome barriers to engaging with data by fostering data literacy through casual engagement. This work in progress explores how informal learning through creation and play with interactive data representations (physicalizations) can foster increased literacy and engagement with data. We outline a series of DIY paper charts using AR markers and everyday materials to help children interact and explore data through the creative process of making.

Traditional handcraft and modern cyborg culture share a common goal: democratize creation through demonstrations and education. Cyborg Crafts blends techniques from the fiber arts with cyborg-inspired technologies (e.g., open-source biosensing EEG headsets and RFID implants). Second SKIN (Soft Keen INteraction), intended to support this practice, is a handmade collection of four modular soft wearable sensors with a temperature-dependent dynamic display. Each sensor has unique sensor-specific outer shell textures based on non-woven textile techniques, and each supports a different sense: momentary switch, pressure sensor, pinch sensor, and a gesture-detecting, capacitive touch sensor. The interactions of pressing, pinching, and touching are encouraged by sensor-specific extruded designs lending to finger placement. The outer shell textures are made from a mixture of flaxseed mucilage and silicone rubber. Thermochromic pigment additives endow display functionality to these passive devices through the application of heat in excess of 86°F.

Mind mapping is a popular way to explore a design space in creative thinking exercises, allowing users to form associations between concepts. Yet, most existing digital tools for mind mapping focus on authoring and organization, with little support for addressing the challenges of mind mapping such as stagnation and design fixation. We present Spinneret, a functional approach to aid mind mapping by providing suggestions based on a knowledge graph. Spinneret uses biased random walks to explore the knowledge graph in the neighborhood of an existing concept node in the mind map, and provides "suggestions" for the user to add to the mind map. A comparative study with a baseline mind-mapping tool reveals that participants created more diverse and distinct concepts with Spinneret, and reported that the suggestions inspired them to think of ideas they would otherwise not have explored.