Translating physical games to digital tabletops is a promising way to engage museum visitors in open-ended museum experiences, but the process is poorly understood. How can the mechanics and social dynamics of physical games be translated into this new material form? This paper presents an empirical account of translating a physical, history-themed card game to a tangible tabletop for museum play. Through a year-long Research through Design study, we developed and deployed three comparative prototypes, each investigating a key dimension of translation: (1) the shift from turn-based to real-time event placement, (2) the introduction of time pressure, and (3) the adaptation of game difficulty. Drawing on in-situ playtests with families and workshops with expert designers, our findings reveal a central tension between fidelity to the source material’s social strengths and the transformative affordances of the digital medium. We contribute actionable design implications for translation and an open-source toolkit to support future work.

Cookie banners and privacy settings attempt to give users a sense of control over how their personal data is collected and used, but background tracking of personal information often continues unnoticed. To explore how such invisible data collection might be made more perceptible, we present DataCrumb, a physical probe that reacts in real-time to data tracking with visual and auditory feedback. Using a research-through-design approach, we deployed the artifact in three households and studied participants’ responses. Instead of providing details about what data was being tracked, the artifact introduced subtle disruptions that made background data flows harder to ignore. Participants described new forms of awareness, contradiction, and fatigue. Our findings show how sensory feedback can support reflection by drawing attention to tracking data flows that are usually hidden. We argue for designing systems that foster awareness and interpretation, especially when the users’ control and understanding are limited.

Smart home cameras (SHCs) offer convenience and security to users, but also cause greater privacy concerns than other sensors due to constant collection and processing of sensitive data. Moreover, privacy perceptions may differ between primary users and other users at home. To address these issues, we developed three physical cover prototypes for SHCs: Manual, Hybrid, and Automatic, based on design criteria of observability, understandability, and tangibility. With 90 SHC users, we ran an online survey using video vignettes of the prototypes. We evaluated how the physical covers alleviated privacy concerns by measuring perceived creepiness and trustworthiness. Our results show that the physical covers were well received, even though primary SHC users valued always-on surveillance. We advocate for the integration of physical covers into future SHC designs, emphasizing their potential to establish a shared understanding of surveillance status. Additionally, we provide design recommendations to support this proposition.

Cameras provide a vast amount of information at high rates and are part of many specialized or general-purpose devices. This versatility makes them suitable for many interaction scenarios, yet they are constrained by geometry and require objects to keep a minimum distance for focusing. We present the LensLeech, a soft silicone cylinder that can be placed directly on or above lenses. The clear body itself acts as a lens to focus a marker pattern from its surface into the camera it sits on. This allows us to detect rotation, translation, and deformation-based gestures such as pressing or squeezing the soft silicone. We discuss design requirements, describe fabrication processes, and report on the limitations of such on-lens widgets. To demonstrate the versatility of LensLeeches, we built prototypes to show application examples for wearable cameras, smartphones, and interchangeable-lens cameras, extending existing devices by providing both optical input and output for new functionality.

Remote video communication is now part of everyday life, also for families. At the same time, children encounter digital devices at an early age, but studies indicate that physical play is still vital for their development. To support physical play at a distance, we introduce two prototypes build around projection displays, WallWizard and TableCanvas. Both allow users to play together remotely by combining physical and digital elements on shared surfaces. We evaluated our prototypes through an expert review, and, based on this study, we elaborate further on TableCanvas. We conducted a second qualitative user study with an updated prototype, focused on evaluating the remote aspect. The overall feedback was positive and suggested that the concept could facilitate and promote open-ended play, as well as support a successful remote play experience. Users also indicated additional potential use cases for board gaming, education, and work-related tasks.

Real-time online interaction is the norm today. Tabletops and other dedicated interactive surface devices with direct input and tangible interaction can enhance remote collaboration, and open up new interaction scenarios based on mixed physical/virtual components. However, they are only available to a small subset of users, as they usually require identical bespoke hardware for every participant, are complex to setup, and need custom scenario-specific applications. We present SurfaceCast, a software toolkit designed to merge multiple distributed, heterogeneous end-user devices into a single, shared mixed-reality surface. Supported devices include regular desktop and laptop computers, tablets, and mixed-reality headsets, as well as projector-camera setups and dedicated interactive tabletop systems. This device-agnostic approach provides a fundamental building block for exploration of a far wider range of usage scenarios than previously feasible, including future clients using our provided API. In this paper, we discuss the software architecture of SurfaceCast, present a formative user study and a quantitative performance analysis of our framework, and introduce five example application scenarios which we enhance through the multi-user and multi-device features of the framework. Our results show that the hardware- and content-agnostic architecture of SurfaceCast can run on a wide variety of devices with sufficient performance and fidelity for real-time interaction.

Self-tracking technologies have long promised to enhance our well-being. However, our initial work and that of others show that most of these technologies focus on data, not the user. Based on interviews, development of mood boards, and the creation of a research product, we propose an alternative approach to self-tracking: re-humanising self-tracking technologies. Our work shows that feelings play an important role with data, that data are temporal, and associated with work and utility. We interpret four design criteria, which are applied in the creation of Frankie: a human-centred tracking device which records both quantitative (number of activities) and qualitative (perceived weight of the activity and spoken reflections) data to foster self-reflection. Through this design case we add to the discussion on re-imagining self-tracking technologies to go beyond data-centric artefacts.

Smartphone-using pedestrians are often distracted, leading to frequent accidents of varying severity with a risk of both awkwardness and injury. We introduce HeadsUp, a mobile app designed to warn the user of imminent collisions with solid obstacles. HeadsUp runs on unmodified commodity smartphones without additional hardware and uses active ultrasound sensing based on the Doppler effect. We contribute an analysis of the ultrasound audio characteristics of six different smartphone models to verify the feasibility of our approach across vendors and device classes, and a description of two implementation variants of our signal processing pipeline. We evaluate our system both in a lab environment and under real-world conditions, and we conclude that HeadsUp can effectively work at a range of up to 3 meters, even though overall performance is heavily dependent on both the individual user and the environment characteristics.

From small tangibles to large tabletops, mirrors with complex geometries can be invaluable tools to reflect or form light for cameras and projectors. While many fabrication techniques are available for prototyping physical or electronic components, creating mirrors requires manual computation and industrial manufacturing equipment and is therefore considerably slower and more expensive. We propose a technique for fabricating mirror surfaces based on thermoplastics sheets with a laminated metallization layer and 3D-printed fixtures for bending or vacuum forming. Given our toolchain, a simulation of mirrors for cameras and projectors by rendering the CAD model is possible and allows to fabricate and evaluate design iterations quickly, making the process reasonably accessible for research. Finally, we show two prototypes for tangible interfaces based on our mirrors for projection and camera-based interaction, discussing advantages and limitations.

Routines are an important element of day-to-day work life, supporting people in structuring their day around required tasks. Effectively managing these routines is, however, experienced as challenging by many – an issue further amplified by the current work from home lockdown measures. In this paper we present Rainmaker, a tangible device to support people in their working life in the context of their own homes. We evaluate and iterate on our prototype through two qualitative studies, spanning respectively three days (N = 11) and 15 days (N = 2). Our results highlight the perceived advantages of the use of a primarily physical rather than digital tool for work support, allowing users to stay focused on their tasks and reflect on their work achievements. We present lessons for future work in this area and publicly release the software and hardware used in the construction of Rainmaker.

We present Seedmarkers, shape-independent topological markers that can be embedded in physical objects manufactured with common rapid-prototyping techniques. Many markers are optimized for technical performance while visual appearance or the feasibility of permanently merging marker and physical object is not considered. We give an overview of the aesthetic properties of a wide range of existing markers and conducted a short online survey to assess the perception of popular marker designs. Based on our findings we introduce our generation algorithm making use of weighted Voronoi diagrams for topological optimization. With our generator, Seedmarkers can be created from technical drawings during the design process to fill arbitrary shapes on any surface. Given dimensions and manufacturing constraints, different configurations for 3 or 6 degrees of freedom tracking are possible. We propose a set of application examples for shape-independent markers, including 3D printed tangibles, laser cut plates and functional markers on printed circuit boards.

We present TempoWatch, a wearable smartwatch-based interface designed to allow dance instructors to control music playback and tempo directly on their wrist via touch gestures using a circular watch display. Dance instructors have unique requirements with respect to music playback in their classes, in particular the ability to stay in position while controlling the playback, and to change speed via time-stretching. However, common stereo decks and mobile music player apps do not support these requirements well. We present the design and architecture of our system, and a qualitative evaluation performed with 9 semi-professional instructors in their own dance classes. Dance instructors were involved in this project from the very beginning to match the system and interface design to its prospective use cases. Results show that instructors are able to use TempoWatch productively after only a short learning phase.

Several fields of science are experiencing a "replication crisis" that has negatively impacted their credibility. Assessing the validity of a contribution via replicability of its experimental evidence and reproducibility of its analyses requires access to relevant study materials, data, and code. Failing to share them limits the ability to scrutinize or build-upon the research, ultimately hindering scientific progress.

Abstract This paper presents a short‐contact multitouch vocabulary for interacting with scatterplot matrices (SPLOMs) on wall‐sized displays. Fling‐based gestures overcome central interaction challenges of such large displays by avoiding long swipes on the typically blunt surfaces, frequent physical navigation by walking for accessing screen areas beyond arm's reach in the horizontal direction and uncomfortable postures for accessing screen areas in the vertical direction. Furthermore, we make use of the display's high resolution and large size by supporting the efficient specification of two‐tiered focus + context regions which are consistently propagated across the SPLOM. These techniques are complemented by axis‐centered and lasso‐based selection techniques for specifying subsets of the data. An expert review as well as a user study confirmed the potential and general usability of our seamlessly integrated multitouch interaction techniques for SPLOMs on large vertical displays.

Since its introduction in 2005, the TUIO protocol has been widely employed within a multitude of usage contexts in tangible and multi-touch interaction. While its simple and versatile design still covers the core functionality of interactive tabletop systems, the conceptual and technical developments of the past decade also led to a variety of ad-hoc extensions and modifications for specific scenarios. In this paper, we present an analysis of the strengths and shortcomings of TUIO 1.1, leading to the constitution of an extended abstraction model for tangible interactive surfaces and the specification of the second-generation TUIO 2.0 protocol, along with several example encodings of existing tangible interaction concepts.

Although public displays are increasingly being deployed in everyday situations, they are still mostly used as auto-active information sources. Adding interactivity can help to attract and engage users. We report on the design and in-the-wild evaluation of an interactive advert for a public display in a tourist information center. We evaluate and compare 3 different variants - non-interactive, interaction using body tracking, and interaction using personal mobile devices - with respect to attracting the attention and interaction from passersby. We further compare these variants with an iterated version of the body tracking system with an extended tracking area. Our findings include an unexpected reluctance of passersby to use their mobile device in public, and the increased interactive area for body interaction resulting in increased engagement and spontaneous multi-user interaction, while removing the so-called 'landing effect'. Based on our findings, we suggest guidelines for interactive adverts on public displays.

We present TouchScope, a hybrid multitouch interface for common off-the-shelf oscilloscopes. Oscilloscopes are a valuable tool for analyzing and debugging electronic circuits, but are also complex scientific instruments. Novices are faced with a seemingly overwhelming array of knobs and buttons, and usually require lengthy training before being able to use these devices productively.

We present TouchScope, a hybrid multitouch interface for common off-the-shelf oscilloscopes. Oscilloscopes are a valuable tool for analyzing and debugging electronic circuits, but are also complex scientific instruments. Novices are faced with a seemingly overwhelming array of knobs and buttons, and usually require lengthy training before being able to use these devices productively.

We present ShakeCast, a system for automatic peer-to-peer exchange of contact information between two persons who just shook hands. The accelerometer in a smartwatch is used to detect the physical handshake and implicitly triggers a setup-free information transfer between the users' personal smartphones using Bluetooth LE broadcasts. An abstract representation of the handshake motion data is used to disambiguate between multiple simultaneous transmissions and to prevent accidental data leakage.

We introduce CalendarCast, a novel method to support the common task of finding a suitable time and date for a shared meeting among co-located participants using their personal mobile devices. In this paper, we describe the Bluetooth-based wireless protocol and interaction concept on which CalendarCast is based, present a prototypical implementation with Android smartphones and dedicated beacons, and report on results of a user study demonstrating improved task performance compared to unaugmented calendars. The motivating scenario for CalendarCast occurs quite often in a variety of contexts, for example at the end of a prior meeting or during ad-hoc conversations in the hallway. Despite a large variety of digital calendar tools, this situation still usually involves a lengthy manual comparison of free and busy time slots. CalendarCast utilizes Bluetooth Low Energy (BTLE) advertisement broadcasts to share the required free/busy information with a limited, localized audience, on demand only, and without revealing detailed personal information. No prior knowledge about the other participants, such as email addresses or account names, is required.

Optical sensing technologies are among the most versatile hardware solutions for interactive surfaces, as they are capable of recognizing touch as well as (limited) hover state in addition to printed tokens. One widely used system is the Pixelsense/SUR40, currently one of very few devices which provides these capabilities in the form factor of a regular table, thereby allowing working at the device in a sitting position. Unfortunately, the device has been discontinued by the manufacturer, provides only an unsupported SDK on an outdated operating system, and has a gathered a reputation for high latency as well as sensitivity to environment light.

We present Shoe me the Way, a novel tactile interface for eyes-free pedestrian navigation in urban environments. Our prototypical implementation can be fully integrated into users' own, regular shoes without permanent modifications. Interface use does not distract users from their surroundings. It thereby adds to users' safety and enables them to explore their environments more freely than is possible with prevailing mobile map-based pedestrian navigation systems. We evaluated our prototype using two different navigation modes in a study with 21 participants and report on significant differences in user performance and preferences between the modes. Study results also show that even our prototypical implementation is already stable, functional and has high usability.

We present GISpL, the Gestural Interface Specification Language. GISpL is a formal language which allows both researchers and developers to unambiguously describe the behavior of a wide range of gestural interfaces using a simple JSON-based syntax. GISpL supports a multitude of input modalities, including multi-touch, digital pens, multiple regular mice, tangible interfaces or mid-air gestures.

In recent years, a large number of multitouch sensor concepts have been presented. Particularly optical sensors are highly popular due to their versatility. However, especially camera-based systems often require a significant amount of space behind the screen and are not well suited to flatscreen-based setups. While integrated sensors for flatscreens have already been presented, they are mostly complex, expensive or both.

Many types of tangible interaction systems, such as interactive surfaces and gesture-based interfaces, are based on various kinds of optical tracking, using infrared illuminators and cameras. One drawback of these setups is that they suffer from problems common to optical trackers, such as sensitivity to stray environment light from artificial and natural sources. In this paper, we present a method to significantly enhance tracking robustness for those systems which employ active illumination. Through addition of a small electronic circuit which modulates the LEDs used to illuminate the scene, contrast can be significantly increased.

Multi-touch interfaces have been a focus of research in recent years, resulting in development of various innovative UI concepts. Support for existing WIMP interfaces, however, should not be overlooked. Although several approaches exist, there is still room for improvement, particularly regarding implementation of the "hover" state, commonly used in mouse-based interfaces.

Ubiquitous tracking setups, covering large tracking areas with many heterogeneous sensors of varying accuracy, require dedicated middleware to facilitate development of stationary and mobile applications by providing a simple interface and encapsulating the details of sensing, calibration and sensor fusion. In this paper we present a centrally coordinated peer-to-peer architecture for ubiquitous tracking, where a server computes optimal data flow configurations for sensor and application clients, which are directly exchanging tracking data with low latency using a light-weight data flow framework. The server's decisions are inferred from an actively maintained central spatial relationship graph (SRG) using spatial relationship patterns. The system is compared to a previous Ubitrack implementation using the highly distributed DWARF middleware. It exhibits significantly better performance in a reference scenario.