Tangible User Interfaces (TUIs) that integrate digital information with physical interaction require specialized hardware and complex calibration, limiting their adoption in portable or mobile display systems. This paper introduces ArUcoTUI, a computer vision (CV) toolkit for prototyping tangible interactions on portable screens, leveraging standard cameras and the OpenCV library. ArUcoTUI uses ArUco fiducial markers to detect physical inputs. The software toolkit offers streamlined calibration, a signal processing pipeline, and a client application that translates tangible input into structured events for use in HCI applications. Using a conventional camera in a top-down setting with a flat-panel display, we demonstrate how this toolkit supports the development of interactive surface TUIs with advanced features, including 3D spatial interaction, multi-device interaction, and actuated tangibles within applications. We describe the software implementation, which utilizes accessible hardware to support the development of these tangible interactions. We provide the results of a preliminary evaluation with users, including design implications and suggestions for future research and development.

Outdated Computer Vision (CV) toolkits for Tangible User Interfaces (TUI) have led to fragmented practices, diminished reproducibility, and reduced community support. This paper examines the past, present, and future trajectory of CV-TUI toolkits. Our scoping review of ACM literature (N=120) reveals a divergence between applications using the limited interactions of established toolkits like ReacTIVision and the fragmented, bespoke systems built for complex interactions, highlighting the need for advanced toolkits that enable accessible making. We found how ReacTIVision-based projects were primarily used in stationary tabletop settings with standard token events, while non-ReacTIVision projects explored unique interaction techniques in diverse configurations. Reflecting on these insights, we offer six suggestions for building the next-generation CV-TUI toolkits to support rapid prototyping of tangible interactions. This study provides the TUI community with an updated perspective to inform future research.

Crowd management is crucial for large-scale events, requiring a 90% upfront planning effort with inter-disciplinarity and collaboration. However, existing tools often fall short of facilitating dynamic interactions between stakeholders during the planning process. This paper presents CrowdCollab, an interactive system consisting of a portable tabletop display platform with tangibles and a handheld display that adds a situated experience, fostering hands-on exploration from multiple perspectives. A user study was conducted with 16 participants to evaluate the usability and gather user feedback. The findings demonstrate valuable insights into how the designed system facilitates shared understanding and an engaging collaboration experience in crowd management.

Tracking fluid intake can help reduce the risk of dehydration and, subsequently, the rehospitalization of older adults. Several technological solutions have been developed to track fluid intake, but older adults have not widely embraced them. We report on a study using an existing solution, MyBeaker, as a probe to explore the limitations when used in the daily lives of the expected user group. We underscore the insights gathered from the study and address them by suggesting a proof-of-concept redesign aimed at integrating elderly users' current fluid intake practices more seamlessly. We highlight insights gained from the study and demonstrate the potential benefits of using existing technology as probes to better understand the home context of older adults for the purpose of improving existing or informing the design of new assistive technologies.

Using information and communication technology (ICT) while solo dining can alleviate loneliness but may also lead to unhealthy food choices and decreased satiety. Mindful eating practices are needed to raise awareness about the impact of ICT technology on eating behavior. Our work aims to use technology to augment physical practices and generate physical artifacts by physicalizing sensor data and letting users self-annotate and make sense of the sensor data. With a toolkit that supports constructive physicalization of inertia sensor data related to the users' eating behavior, users increase their awareness of their mundane routine of solo dining with ICT devices by building personalized data physicalization and adding annotations. Results of a five-day preliminary study indicate how annotating data physicalization could be a beneficial intervention for promoting mindful eating behavior in a personal context.

Collective stress is the stress within a group or an organization. It affects individuals’ well-being and group productivity. HCI research has started exploring collective stress visualization to facilitate group awareness and collective coping via testing prototypes in controlled settings. However, an in-depth understanding of users’ needs and envisaged scenarios based on their authentic experiences are still lacking. In this study, we utilized a participatory approach called co-constructing stories to investigate how a collective stress visualization would be used in office workers’ authentic workday routines. We constructed use case stories with a group of office workers separately based on their personal lived experiences, using a design probe called AffectiveGarden. Our results categorized six clusters of benefits for collective coping through visualization and their implications for future design practice.

This paper presents TexonMask, a facial expression recognition system using lightweight electrode-augmented commodity facemasks. With a matrix of textile electrodes carefully deployed on a commodity mask, our edge computing system recognizes the wearer’s facial expressions with machine learning based on the capacitive sensor readings, provides a wearable affective display and communicates with external devices using low bandwidth. Results from user studies show that the system is effective and efficient at recognizing five or ten facial expressions with an accuracy of around , using a personalized classifier trained with only six data points per expression. The system’s performance is stable across the use sessions and further improves when more data points are collected. We further developed two LiveEmoji applications for facilitating online and face-to-face communication of facemask wearers, demonstrated them in user interviews, and obtained positive participant feedback. Based on the results and findings of the study, we discuss implications and future research directions for facilitating emotional communication between facemask wearers and others.

Video-based actigraphy is a non-contact technology that measures body movement through video recordings, commonly expressed as activity count. It has been widely used to assess infant sleep. One validated method for video-based actigraphy is the three-dimensional recursive search (3DRS). However, its sleep-wake classification performance in preterm infants is only moderately satisfactory, necessitating methods to improve it. This paper proposes an enhanced approach to compute activity count for video-based actigraphy in evaluating sleep patterns in preterm infants. The proposed technique involves applying exponentially weighted moving average (EWMA) to continuous activity count to reduce abrupt variations and false detection of wakefulness. Additionally, the 3DRS protocol we previously utilized is proprietary, limiting its accessibility to the public and its widespread adoption. To overcome this limitation, we have investigated the efficacy of an open-source algorithm known as background subtraction (BS), which, in our previous work, has demonstrated superior performance in body motion detection in preterm infants. To evaluate the performance of the proposed method, a dataset consisting of video recordings from five hospitalized preterm infants was used. The activity counts obtained from 3DRS, and BS were compared in terms of their ability to classify sleep and wake states using a linear discriminant classifier. The results obtained through leave-one-patient-out cross-validation revealed a significant improvement in the classification of sleep and wake states in preterm infants for both methods when the EWMA were applied to the activity counts. The mean Cohen's kappa coefficients were found to be 0.58 for 3DRS and 0.51 for BS. Despite BS exhibited comparatively lower performance than 3DRS, it can still be considered a viable alternative for acquiring video-based actigraphy data in the assessment of sleep of preterm infants.

The aim of this study is to develop an explainable late-onset sepsis (LOS) prediction algorithm using continuous multi-channel physiological signals that can be applied to a patient monitor for preterm infants in a neonatal intensive care unit (NICU). The algorithm uses features on heart rate variability (HRV), respiration, and motion, based on electrocardiogram (ECG) and chest impedance (CI). In this study, 127 preterm infants were included, of whom 59 were bloodculture-proven LOS patients and 68 were control patients. Features in 24 hours before the onset of sepsis (LOS group), and an age-matched onset time point (control group) were extracted and fed into machine learning classifiers with gestational age and birth weight. We compared the prediction performance of several well-known classifiers using features from different signal channels (HRV, respiration, and motion) individually as well as their combinations. The prediction performance was evaluated using the area under the receiver-operating-characteristics curve (AUC). The best performance was achieved by an extreme gradient boosting classifier combining features from all signal channels, with an AUC of 0.88, a positive predictive value of 0.80, and a negative predictive value of 0.83 during the 6 hours preceding LOS onset. This feasibility study demonstrates the complementary predictive value of motion information in addition to cardiorespiratory information for LOS prediction. Furthermore, visualization of how each feature in the individual patient impacts the algorithm decision strengthen its interpretability. In clinical practice, it is important to motivate clinical interventions and this visualization method can help to support the clinical decision.

NFC (Near-Field Communication) has been widely applied for human-computer interaction (HCI). However, the short sensing distance of NFC requires the users to initiate the tasks with extra effort mostly using their hands, so it is inconvenient to use NFC in hands-busy scenarios. This paper presents an investigation of body-centric interactions between the NFC device users and their surroundings. The exploration is based on the recent development of near-field enabled clothing, which can passively extend an NFC-enabled device’s reading distance to users’ body landmarks. We present an accessible method for fabricating flexible, extensible, and scalable NFC extenders on clothing pieces, and an easy-to-use toolkit for facilitating designers to realize the interactive experiences. The method and toolkit were tested in technical experiments and in a co-creation workshop. The elicited design outcomes and the further exploratory makings generated knowledge for future research and embodied interaction design opportunities.

Remote collaboration is a challenge for physical product designers, and especially limits the mutual communication of the “thinking through prototyping” approach. Inspired by the NURBSforms and self-shape sensing technologies, we investigate how to support remote co-design with a conceptual modular shape-changing interface toolkit. By using stop-motion videos and low-fi non-electronic prototypes as probes, the concept was evaluated through on-line questionnaires and workshops. The result shows its potential especially in the exploration phase of remote co-design.

In this paper, we propose NFCStack, which is a physical building block system that supports stacking and frictionless interaction and is based on near-field communication (NFC). This system consists of a portable station that can support and resolve the order of three types of passive identifiable stackable: bricks, boxes, and adapters. The bricks support stable and sturdy physical construction, whereas the boxes support frictionless tangible interactions. The adapters provide an interface between the aforementioned two types of stackable and convert the top of a stack into a terminal for detecting interactions between NFC-tagged objects. In contrast to existing systems based on NFC or radio-frequency identification technologies, NFCStack is portable, supports simultaneous interactions, and resolves stacking and interaction events responsively, even when objects are not strictly aligned. Evaluation results indicate that the proposed system effectively supports 12 layers of rich-ID stacking with the three types of building block, even if every box is stacked with a 6-mm offset. The results also indicate possible generalized applications of the proposed system, including 2.5-dimensional construction. The interaction styles are described using several educational application examples, and the design implications of this research are explained.

This paper presents NFCSense, a data-defined rich-ID motion sensing technique for fluent tangible interaction design by using commodity near-field communication (NFC) tags and a single NFC tag reader. An NFC reader can reliably recognize the presence of an NFC tag at a high read rate (∼ 300 reads/s) with low latency, but such high-speed reading has rarely been exploited because the reader may not effectively resolve collisions of multiple tags. Therefore, its human–computer interface applications have been typically limited to a discrete, hands-on interaction style using one tag at a time. In this work, we realized fluent, hands-off, and multi-tag tangible interactions by leveraging gravity and anti-collision physical constraints, which support effortless user input and maximize throughput. Furthermore, our system provides hot-swappable interactivity that enables smooth transitions throughout extended use. Based on the design parameters explored through a series of studies, we present a design space with proof-of-concept implementations in various applications.

This research investigates the design space of combining touchscreens with passive rich-ID building block systems to support the physical construction of contexts in touchscreen interactions. With two proof-of-concept systems, RFIPillars and RFITiles, we explore various schemes for using tangible inputs for context enrichment in touchscreen interactions. Instead of incorporating an electronic touchscreen module that requires per-module maintenance, this work intentionally makes each tangible object passive. We explore rear-projection solutions to integrate touchscreen interactions into these passive building blocks with capacitive touch sensing techniques and deliberate physical forgiving to retain the merits of being both batteryless and wireless. The presented research artifacts embody the interaction designs and elucidate scalability challenges in integrating touchscreen interactions into this emerging tangible user interface.

Co-dining benefits in closeness and connectedness between all family members. However, it is hard for those in different locations to enjoy a meal together due to the geographical barriers and inconsistent life schedule. The widely applied video technology caused excessive visual interference when solving problems, while the effect of sound in a meal on intimacy has not been studied. In this paper, we explored how can recorded audio create an asynchronous co-dining experience and support intimacy. We conducted a field study with 24 participants, and the result shows the system potentially enhance the intimacy in remote coding experience. In the follow-up workshop with 6 participants, we further discussed the future landscape of designing a sound interface for remote meals.

This issue of the Proceedings of the ACM on Human-Computer Interaction features contributions in the intersection of human-computer interaction and software engineering, with further disciplines blending into a rich set of scientific works. 2021 is the first time the annual conference on Engineering Interactive Computing Systems (EICS) is hosted in the Netherlands and in the context of an Industrial Design department. We take this opportunity to focus on the relations and influence of the design discipline on the work of the EICS community. This resulted in a new set of topics for EICS, which were already partly reflected in the many submissions we received in three extensive review rounds throughout 2020 and the beginning of 2021. In this editorial we offer a perspective on what EICS is not yet, looking at the inclusion of and interplay with design as a related discipline.

Mechanical pushbuttons, which provide physical landmarks and clear tactile feedback, are easily accessible and highly reliable in eyes-free use. Potentially, their merits can improve the experiences of on-body or wearable HCI. However, they are not commonly adopted as a user interface of smart textiles because the physical mechanism of conventional pushbutton hardware requires further integration, which should be seamless enough to be comfortably worn. In this pictorial, we present a design exploration of the methodologies for interweaving mechanical pushbuttons into fabrics. The exploration used a frame system, which unifies the workflow of digital embroidery and 3D printing and enables the exploration of the physical design. Through the process, we investigated methods of integration and fabrication through making and presented our findings with proof-of-concept implementations. We also discussed the alternative designs and interaction methods as well as their implications to enlighten future research directions and opportunities.

This paper presents a crowd-sourcing video-based investigation on the users' perception of proactiveness towards an object with artificial eyes through winking. Based on both the session initiation protocol (SIP) and the perceptual crossing paradigm, we synthesized a set of minimally-designed video clips and tested them in Amazon Mechanical Turk with 240 participants. The results show that, in both single- or multi-user scenarios, winking can be a useful expression that makes the user view the object as being proactive and encourages reciprocal input. We also discuss how the object-initiated interaction extends the perceptual crossing paradigm in human-object communication.

Tangibles can model abstract structures. One educational subject where this can be utilized is instruction on data visualization inter- pretation. Data physicalizations, tangible representations of data, offer graspable handles for the users to manipulate data visualiza- tions directly so that they can better understand what information they hold. However, investigations on the applicability of interac- tive data physicalizations in educational settings are still sparse. In this paper, we explore how students reason with an interactive tangible scatterplot through a collaborative data interpretation tool, CoDa. We report the design, development, and the user experiences in an exploratory study where 11 students, in groups of 2 to 4, completed a data analysis task with CoDa. The qualitative results show insights in the process of data interpretation, how interaction with the tangibles influenced these data interpretations, how the system aided collaboration and, overall user experience. We believe the results and implications offer a step towards nurturing future educational applications on interactive data physicalizations.

This work explores and evaluates the designs of finger-worn radio-frequency identification (RFID) motion tracking for activity recognition on tagged objects. We propose an index-finger-worn device that consists of a short-range (~2cm) RFID reader and a pair of two inertial measurement units (IMUs), which are mounted at the locations where an artificial nail and a ring are worn. The short-range RFID reader recognizes the tagged object on finger touch, and then the IMU data are used for activity recognition. Data collected from the user of this device allows for a post-hoc analysis, which informs the activity recognition performance in various RFID+IMU and IMU-only configurations on the same task. The results of a ten-participant user study show that when the objects have similar physical form factors, the hybrid RFID motion tracking significantly outperforms the IMU-only tracking, especially in a larger-number set of objects. In our test, three IMU configurations (i.e., NailOnly, RingOnly, and Nail+Ring) achieved comparable action recognition performances, i.e., ≥90% accuracy, with 500 ms recognition time, though the NailOnly RFID+IMU configuration provided the highest wearability. The practical challenges toward a real-world deployment of a finger-worn RFID motion tracking system are also discussed.

This paper presents RFTouchPads, a system of batteryless and wireless modular hardware designs of two-dimensional (2D) touch sensor pads based on the ultra-high frequency (UHF) radio-frequency identification (RFID) technology. In this system, multiple RFID IC chips are connected to an antenna in parallel. Each chip connects only one of its endpoints to the antenna; hence, the module normally turns off when it gets insufficient energy to operate. When a finger touches the circuit trace attached to another endpoint of the chip, the finger functions as part of the antenna that turns the connected chip on, while the finger touch location is determined according to the chip's ID. Based on this principle, we propose two hardware designs, namely, StickerPad and TilePad. StickerPad is a flexible 3×3 touch-sensing pad suitable for applications on curved surfaces such as the human body. TilePad is a modular 3×3 touch-sensing pad that supports the modular area expansion by tiling and provides a more flexible deployment because its antenna is folded. Our implementation allows 2D touch inputs to be reliability detected 2 m away from a remote antenna of an RFID reader. The proposed batteryless, wireless, and modular hardware design enables fine-grained and less-constrained 2D touch inputs in various ubiquitous computing applications.

We present RFIBricks, an interactive building block system based on ultrahigh frequency radio-frequency identification (RFID) sensing. The system enables geometry resolution based on a simple yet highly generalizable mechanism: an RFID contact switch, which is made by cutting each RFID tag into two parts, namely antenna and chip. A magnetic connector is then coupled with each part. When the antenna and chip connect, an interaction event with an ID is transmitted to the reader. On the basis of our design of RFID contact switch patterns, we present a system of interactive physical building blocks that resolves the stacking order and orientation when one block is stacked upon another, determines a three-dimensional (3D) geometry built on a two-dimensional base plate, and detects user inputs by incorporating electromechanical sensors. Because it is calibration-free and does not require batteries in each block, it facilitates straightforward maintenance when deployed at scale. Compared with other approaches, this RFID-based system resolves several critical challenges in human-computer interaction, such as 1) determining the identity and the built 3D geometry of passive building blocks, 2) enabling stackable token+constraint interaction on a tabletop, and 3) tracking in-hand assembly.

This paper presents BioFidget, a biofeedback system that integrates physiological sensing and display into a smart fidget spinner for respiration training. We present a simple yet novel hardware design that transforms a fidget spinner into 1) a nonintrusive heart rate variability (HRV) sensor, 2) an electromechanical respiration sensor, and 3) an information display. The combination of these features enables users to engage in respiration training through designed tangible and embodied interactions, without requiring them to wear additional physiological sensors. The results of this empirical user study prove that the respiration training method reduces stress, and the proposed system meets the requirements of sensing validity and engagement with 32 participants in a practical setting.

This paper presents a technique enabling distributed batteryless near-field identification (ID) between two passive radio frequency ID (RFID) tags. Each conventional ultra-high-frequency (UHF) RFID tag is modified by connecting its antenna and chip to a reed switch and then attaching a magnet to one of the reed switch's terminals, thus transforming it into an always-on switch. When the two modules approach each other, the magnets counteract each other and turn off both switches at the same time. The coabsence of IDs thus indicates a unique interaction event. In addition to sensing, the module also provides native haptic feedback through magnetic repulsion force, enabling users to perceive the system's state eyes-free, without physical constraints. Additional visual feedback can be provided through an energy-harvesting module and a light emitting diode. This specific hardware design supports contactless, orientation-invariant sensing, with a form factor compact enough for embedded and wearable use in ubiquitous computing applications.

Thumb-to-fingers interfaces augment touch widgets on fingers, which are manipulated by the thumb. Such interfaces are ideal for one-handed eyes-free input since touch widgets on the fingers enable easy access by the stylus thumb. This study presents DigitSpace, a thumb-to-fingers interface that addresses two ergonomic factors: hand anatomy and touch precision. Hand anatomy restricts possible movements of a thumb, which further influences the physical comfort during the interactions. Touch precision is a human factor that determines how precisely users can manipulate touch widgets set on fingers, which determines effective layouts of the widgets. Buttons and touchpads were considered in our studies to enable discrete and continuous input in an eyes-free manner. The first study explores the regions of fingers where the interactions can be comfortably performed. According to the comfort regions, the second and third studies explore effective layouts for button and touchpad widgets. The experimental results indicate that participants could discriminate at least 16 buttons on their fingers. For touchpad, participants were asked to perform unistrokes. Our results revealed that since individual participant performed a coherent writing behavior, personalized $1 recognizers could offer 92% accuracy on a cross-finger touchpad. A series of design guidelines are proposed for designers, and a DigitSpace prototype that uses magnetic-tracking methods is demonstrated.

This work develops a system of spherical magnetic tangibles, GaussMarbles, that exploits the unique affordances of spherical tangibles for interacting with portable physical constraints. The proposed design of each magnetic sphere includes a magnetic polyhedron in the center. The magnetic polyhedron provides bi-polar magnetic fields, which are expanded in equal dihedral angles as robust features for tracking, allowing an analog Hall-sensor grid to resolve the near-surface 3D position accurately in real-time. Possible interactions between the magnetic spheres and portable physical constraints in various levels of embodiment were explored using several example applications.

We present GaussRFID, a hybrid RFID and magnetic-field tag sensing system that supports interactivity when embedded in retrofitted or new physical objects. The system consists of two major components - GaussTag, a magnetic-RFID tag that is combined with a magnetic unit and an RFID tag, and GaussStage, which is a tag reader that is combined with an analog Hall-sensor grid and an RFID reader. A GaussStage recognizes the ID, 3D position, and partial 3D orientation of a GaussTag near the sensing platform, and provides simple interfaces for involving physical constraints, displays and actuators in tangible interaction designs. The results of a two-day toy-hacking workshop reveal that all six groups of 31 participants successfully modified physical toys to interact with computers using the GaussRFID system.

This paper investigates the feasibility of using a nail-mounted array of tactors, NailTactors, as an eyes-free output device. By rim-attached eccentric-rotating-mass (ERM) vibrators to artificial nails, miniature high-resolution tactile displays were realized as an eyes-free output device. To understand how to deliver rich signals to users for valid signal perception, three user studies were conducted. The results suggest that users can not only recognized absolute and relative directional cues, but also recognized numerical characters in EdgeWrite format with an overall 89% recognition rate. Experiments also identified the optimal placement of ERM actuators for maximizing information transfer.

The analog Hall-sensor grid, GaussSense, is a thin-form magnetic-field camera technology for designing expressive occlusion-free, near-surface tangible interactions on conventional portable displays. The studio will provide hands-on experiences that combine physical designs and the GaussSense technology. Through a series of brainstorming and making exercises, participants will learn how to exploit natural hand and micro interactions through designing the expressions and affordances of physical objects, and know how to utilize physical constraints to provide additional kinesthetic awareness and haptic feedback. The exercises will be including form-giving, electronic prototyping, and hacking physical toys that are prepared by either the organizers or participants.

This paper presents EdgeVib, a system of spatiotemporal vibration patterns for delivering alphanumeric characters on wrist-worn vibrotactile displays. We first investigated spatiotemporal pattern delivery through a watch-back tactile display by performing a series of user studies. The results reveal that employing a 2×2 vibrotactile array is more effective than employing a 3×3 one, because the lower-resolution array creates clearer tactile sensations in less time consumption. We then deployed EdgeWrite patterns on a 2×2 vibrotactile array to determine any difficulties of delivering alphanumerical characters, and then modified the unistroke patterns into multistroke EdgeVib ones on the basis of the findings. The results of a 24-participant user study reveal that the recognition rates of the modified multistroke patterns were significantly higher than the original unistroke ones in both alphabet (85.9% vs. 70.7%) and digits (88.6% vs. 78.5%) delivery, and a further study indicated that the techniques can be generalized to deliver two-character compound messages with recognition rates higher than 83.3%. The guidelines derived from our study can be used for designing watch-back tactile displays for alphanumeric character output.

For makers and developers, circuit prototyping is an integral part of building electronic projects. Currently, it is common to build circuits based on breadboard schematics that are available on various maker and DIY websites. Some breadboard schematics are used as is without modification, and some are modified and extended to fit specific needs. In such cases, diagrams and schematics merely serve as blueprints and visual instructions, but users still must physically wire the breadboard connections, which can be time-consuming and error-prone. We present CircuitStack, a system that combines the flexibility of breadboarding with the correctness of printed circuits, for enabling rapid and extensible circuit construction. This hybrid system enables circuit reconfigurability, component reusability, and high efficiency at the early stage of prototyping development.

This paper presents Cyclops, a single-piece wearable device that sees its user's whole body postures through an ego-centric view of the user that is obtained through a fisheye lens at the center of the user's body, allowing it to see only the user's limbs and interpret body postures effectively. Unlike currently available body gesture input systems that depend on external cameras or distributed motion sensors across the user's body, Cyclops is a single-piece wearable device that is worn as a pendant or a badge. The main idea proposed in this paper is the observation of limbs from a central location of the body. Owing to the ego-centric view, Cyclops turns posture recognition into a highly controllable computer vision problem. This paper demonstrates a proof-of-concept device, and an algorithm for recognizing static and moving bodily gestures based on motion history images (MHI) and a random decision forest (RDF). Four example applications of interactive bodily workout, a mobile racing game that involves hands and feet, a full-body virtual reality system, and interaction with a tangible toy are presented. The experiment on the bodily workout demonstrates that, from a database of 20 body workout gestures that were collected from 20 participants, Cyclops achieved a recognition rate of 79% using MHI and simple template matching, which increased to 92% with the more advanced machine learning approach of RDF.

This work presents WonderLens, a system of optical lenses and mirrors for enabling tangible interactions on printed paper. When users perform spatial operations on the optical components, they deform the visual content that is printed on paper, and thereby provide dynamic visual feedback on user interactions without any display devices. The magnetic unit that is embedded in each lens and mirror allows the unit to be identified and tracked using an analog Hall-sensor grid that is placed behind the paper, so the system provides additional auditory and visual feedback through different levels of embodiment, further enhancing the interactivity with the printed content on the physical paper.

This work presents LEaD, a helmet-based visual guidance system utilizing light movement in scooter drivers' peripheral vision for turn-by-turn navigation. A linear light strip mounted on a helmet navigates for scooter drivers using simple 1D light movement, which can be easily acquired and identified by peripheral vision with the on-going foveal vision task. User studies suggest that this novel system can effectively direct scooter drivers without introducing visual distractions in route-guided experiences.

This paper presents CyclopsRing, a ring-style fisheye imaging wearable device that can be worn on hand webbings to en- able whole-hand and context-aware interactions. Observing from a central position of the hand through a fisheye perspective, CyclopsRing sees not only the operating hand, but also the environmental contexts that involve with the hand-based interactions. Since CyclopsRing is a finger-worn device, it also allows users to fully preserve skin feedback of the hands. This paper demonstrates a proof-of-concept device, reports the performance in hand-gesture recognition using random decision forest (RDF) method, and, upon the gesture recognizer, presents a set of interaction techniques including on-finger pinch-and-slide input, in-air pinch-and-motion input, palm-writing input, and their interactions with the environ- mental contexts. The experiment obtained an 84.75% recognition rate of hand gesture input from a database of seven hand gestures collected from 15 participants. To our knowledge, CyclopsRing is the first ring-wearable device that supports whole-hand and context-aware interactions.

This paper presents FlexiBend, an easily installable shape-sensing strip that enables interactivity of multi-part, deformable fabrications. The flexible sensor strip is composed of a dense linear array of strain gauges, therefore it has shape sensing capability. After installation, FlexiBend can simultaneously sense user inputs in different parts of a fabrication or even capture the geometry of a deformable fabrication.

This work describes a novel building block system for tangible interaction design, GaussBricks, which enables real-time constructive tangible interactions on portable displays. Given its simplicity, the mechanical design of the magnetic building blocks facilitates the construction of configurable forms. The form constructed by the magnetic building blocks, which are connected by the magnetic joints, allows users to stably manipulate with various elastic force feedback mechanisms. With an analog Hall-sensor grid mounted to its back, a portable display determines the geometrical configuration and detects various user interactions in real time. This work also introduce several methods to enable shape changing, multi-touch input, and display capabilities in the construction. The proposed building block system enriches how individuals interact with the portable displays physically.

This work presents GaussStones, a system of shielded magnetic tangibles design for supporting multi-token interactions on portable displays. Unlike prior works in sensing magnetic tangibles on portable displays, the proposed tangible design applies magnetic shielding by using an inexpensive galvanized steel case, which eliminates interference between magnetic tangibles. An analog Hall-sensor grid can recognize the identity of each shielded magnetic unit since each unit generates a magnetic field with a specific intensity distribution and/or polarization. Combining multiple units as a knob further allows for resolving additional identities and their orientations. Enabling these features improves support for applications involving multiple tokens. Thus, using prevalent portable displays provides generic platforms for tangible interaction design.

We present GaussBits, which is a system of the passive magnetic tangible designs that enables 3D tangible interactions in the near-surface space of portable displays. When a thin magnetic sensor grid is attached to the back of the display, the 3D position and partial 3D orientation of the GaussBits can be resolved by the proposed bi-polar magnetic field tracking technique. This portable platform can therefore enrich tangible interactions by extending the design space to the near-surface space. Since non-ferrous materials, such as the user's hand, do not occlude the magnetic field, interaction designers can freely incorporate a magnetic unit into an appropriately shaped non-ferrous object to exploit the metaphors of the real-world tasks, and users can freely manipulate the GaussBits by hands or using other non-ferrous tools without causing interference. The presented example applications and the collected feedback from an explorative workshop revealed that this new approach is widely applicable.

This work presents a novel and always-available nail mounted display known as NailDisplay. The proposed display augments the use of a finger by allowing for always-available visual feedback owing to its fast accessibility and binding user controls with the display, i.e. what you control is what you see (through the display). Potential benefits of NailDisplay are demonstrated in three applications: from displaying to combining it with user controls. In the first application, NailDisplay can reveal what is occluded under a finger touch, making it a solution to operate small UI elements. In the second application, NailDisplay is complementary to an imaginary interface, helping users to learn an imaginary interface (e.g., on the users' arms) and allowing them to reassure the interface when their memory of it becomes unclear. In the third application, NailDisplay is integrated with rich finger interactions, such as swiping in the air. We also report users' feedbacks gathered from an explorative user study.

We present FingerPad, a nail-mounted device that turns the tip of the index finger into a touchpad, allowing private and subtle interaction while on the move. FingerPad enables touch input using magnetic tracking, by adding a Hall sensor grid on the index fingernail, and a magnet on the thumbnail. Since it permits input through the pinch gesture, FingerPad is suitable for private use because the movements of the fingers in a pinch are subtle and are naturally hidden by the hand. Functionally, FingerPad resembles a touchpad, and also allows for eyes-free use. Additionally, since the necessary devices are attached to the nails, FingerPad preserves natural haptic feedback without affecting the native function of the fingertips. Through user study, we analyze the three design factors, namely posture, commitment method and target size, to assess the design of the FingerPad. Though the results show some trade-off among the factors, generally participants achieve 93% accuracy for very small targets (1.2mm-width) in the seated condition, and 92% accuracy for 2.5mm-width targets in the walking condition.

This work presents GaussSense, which is a back-of-device sensing technique for enabling input on an arbitrary surface using stylus by exploiting magnetism. A 2mm-thick Hall sensor grid is developed to sense magnets that are embedded in the stylus. Our system can sense the magnetic field that is emitted from the stylus when it is within 2cm of any non-ferromagnetic surface. Attaching the sensor behind an arbitrary thin surface enables the stylus input to be recognized by analyzing the distribution of the applied magnetic field. Attaching the sensor grid to the back of a touchscreen device and incorporating magnets into the corresponding stylus enable the system 1) to distinguish touch events that are caused by a finger from those caused by the stylus, 2) to sense the tilt angle of the stylus and the pressure with which it is applied, and 3) to detect where the stylus hovers over the screen. A pilot study reveals that people were satisfied with the novel sketching experiences based on this system.

This paper presents a novel interaction system, PUB (Point Upon Body), to explore eyes-free interaction in a personal space by allowing users tapping on their own arms to be provided with haptic feedback from their skin. Two user studies determine how users can interact precisely with their forearms and how users behave when operating in their arm space. According to those results, normal users can divide their arm space at most into 6 points between their wrists and elbows with iterative practice. Experimental results also indicate that the divided pattern of each user is unique from that of other ones. Based on the design principles from the observations, an interaction system, PUB, is designed to demonstrate how interaction design benefits from those findings. Two scenarios, remote display control and mobile device control, are demonstrated through the UltraSonic device attached on the users' wrists to detect their tapped positions.

This work describes a novel approach to utilizing everyday objects of users as additional, auxiliary, and instant tabletop controllers. Based on this approach, a prototype platform, called iCon, is developed to explore the possible design. Field studies and user studies reveal that utilizing everyday objects such as auxiliary input devices might be appropriate under a multi-task scenario. User studies further demonstrate that daily objects can generally be applied in low precision circumstances, low engagement with selected objects, and medium-to-high frequency of use. The proposed approach allows users to interact with computers while not altering their original work environments.