6-3-5 brainwriting is a structured, silent ideation method where 6 participants each generate 3 ideas in 5 minutes across 6 rounds, yielding up to 108 ideas and enabling equal participation, rapid ideation and cumulative development. Many ideas are fundamentally visual, making generative image models especially useful for accelerating visual exploration and making abstract ideas quickly tangible, but existing tools cannot fully support visual composition, where ideas are built from atomic elements and iteratively remixed. A formative study (N=18) showed that even with GenAI embedded in a 6-3-5 structure, participants invested effort in crafting monolithic prompts yet struggled to obtain intended results, build on others’ ideas, or work modularly, leading to redundancy and limited divergence. To address these issues, we present AtoMix, a collaborative visual interface for remote 6-3-5 brainwriting that supports granularity and cross-pollination through atomic prompting, canvas composition, and reuse of image fragments. In a comparative study (N=24), AtoMix afforded more fine-grained interaction than the baseline and better supported collaborative visual ideation.

Embodiment can enhance conversational agents, such as increasing their perceived presence. This is typically achieved through visual representations of a virtual body; however, visual modalities are not always available, such as when users interact with agents using headphones or display-less glasses. In this work, we explore auditory embodiment. By introducing auditory cues of bodily presence - through spatially localized voice and situated Foley audio from environmental interactions - we investigate how audio alone can convey embodiment and influence perceptions of a conversational agent. We conducted a 2 (spatialization: monaural vs. spatialized) x 2 (Foley: none vs. Foley) within-subjects study, where participants (n=24) engaged in conversations with agents. Our results show that spatialization and Foley increase co-presence, but reduce users' perceptions of the agent's attention and other social attributes.

Multimodal interaction has long promised to make interfaces more intuitive and effective by combining complementary inputs. Among these, gaze and speech form a compelling pairing: gaze provides rapid spatial grounding, while speech conveys rich semantic information. Together, they offer rich cues for understanding user behaviour and intent. Yet despite decades of exploration, the research remains fragmented, making this synthesis timely as these inputs mature and are integrated into consumer-ready devices. This scoping review examined 103 studies published between 1991 and 2025, organised into explicit, where users intentionally provide gaze and speech, and implicit, where systems leverage users' natural behaviours to support interaction. Across both, we identified recurring ways for combining gaze and speech to resolve ambiguity, ground references, and support adaptivity. We contribute a synthesis of research on their combined use while highlighting challenges of temporal alignment, fusion and privacy, offering guidance for future research toward richer multimodal human-computer interaction.

The increasing popularity of microcontroller platforms like Arduino enables diverse end-user developers to participate in circuit prototyping. Traditionally, follow-along tutorials serve as an essential learning method for makers, and in fact, several prior toolkits leveraged this format as a way to engage new makers. However, literature and our formative study (N=12) show that makers have unique preferences regarding the construction of their circuits and idiosyncratic ways to assess and debug problems, which contrasts with the step-by-step instructional nature of tutorials and those systems leveraging this method. To address this mismatch, we present a prototyping platform that supports personalized circuit construction and debugging. Our system utilizes an augmented breadboard, which is circuit-aware and supports on-the-fly hardware reconfiguration via contextualized guidance and in-situ circuit validation through interactive tests. Through a usability study (N=12), we demonstrate how makers leverage circuit-aware guidance and debugging to support individual building patterns.

Figure 1: We present a bimanual motion-coupled vibration algorithm capable of creating connected material experiences between two hands.By modulating the parameters of vibrations between the hands, our algorithm can induce material properties of elasticity while stretching A ○, flexibility while bending B○ and torsion while twisting C ○.

Remote VR teleoperation with supernumerary robotic limbs enables distant users to operate in another’s local space. While a shared first-person view aids hand-eye coordination, locking the guest’s camera to the host’s head can degrade comfort, embodiment, and coordination. Based on a formative study (N=10) using a virtual supernumerary robotic limbs configuration to stress-test coordination, we propose guest-driven perspective switching from a shared first-person baseline (Shared Embodied View) to two alternatives: (a) a stabilized view with guest-controlled rotation (Embedded Anchored View), and (b) a fully decoupled third-person view (Out-of-body View). We ran a user study with 24 pairs (N=48), who switched between the baseline and proposed views as task demands changed. We measured performance, embodiment, fatigue, physiological arousal, and switching behaviors. Our results reveal role-dependent trade-offs: Out-of-body View improves navigation efficiency and reduces errors, while Embedded Anchored View supports embodiment. We conclude with guidelines: use Embedded Anchored View for hand-centric adjustments, Out-of-body View for navigation and object placement, and ensure smooth transitions.

Generative AI technologies are reshaping everyday environments by enabling multimodal interaction. As their ubiquity and agentic capacities grow, there is a pressing need to understand how these systems reshape human–computer interaction in relational, social, and systemic terms. We introduce a scenario-based design pack for investigating Human–GenAI relations. Grounded in assemblage theory and structured around a three-stage process—Prepare, Make, Reflect—the pack supports the prototyping, analysis, and critical reflection of emergent sociotechnical configurations. We evaluated the pack across three deployments: an ACM workshop (n=22), a multidisciplinary design session (n=20), and a university HCI class (n=260). Participants generated scenarios that surfaced relational issues of power, agency, visibility, and care. We contribute the design pack alongside an exploratory framework to advance relational enquiry into multimodal Human–GenAI relations, support more inclusive and socially responsive GenAI practices, and complement FATE approaches by grounding fairness, accountability, and transparency in lived, multimodal configurations.

Haptic perception of physical sizes increases the realism and immersion in Virtual Reality (VR). Prior work rendered sizes by exerting pressure on the user’s fingertips or employing tangible, shape-changing devices. These interfaces are constrained by the physical shapes they can assume, making it challenging to simulate objects growing larger or smaller than the perceived size of the interface. Motivated by literature on pseudo-haptics describing the strong influence of visuals over haptic perception, this work investigates modulating the perception of size beyond this range. We developed a fixed-sized VR controller leveraging finger-repositioning to create a visuo-haptic illusion of dynamic size-change of handheld virtual objects. Through two user studies, we found that with an accompanying size-changing visual context, users can perceive virtual object sizes up to 44.2% smaller to 160.4% larger than the perceived size of the device. Without the accompanying visuals, a constant size (141.4% of device size) was perceived.

Reconfigurable physical interfaces empower users to swiftly adapt to tailored design requirements or preferences. Shape-changing interfaces enable such reconfigurability, avoiding the cost of refabrication or part replacements. Nonetheless, reconfigurable interfaces are often bulky, expensive, or inaccessible. We propose a reversible shape-changing mechanism that enables reconfigurable 3D printed structures via translations and rotations of parts. We investigate fabrication techniques that enable reconfiguration using magnets and the thermoplasticity of heated polymer. Proposed interfaces achieve tunable haptic feedback and adjustment of different user affordances by reconfiguring input motions. The design space is demonstrated through applications in rehabilitation, embodied communication, accessibility, safety, and gaming.

3D printed displays promise to create unique visual interfaces for physical objects. However, current methods for creating 3D printed displays either require specialized post-fabrication processes (e.g., electroluminescence spray and silicon casting) or function as passive elements that simply react to environmental factors (e.g., body and air temperature). These passive displays offer limited control over when, where, and how the colors change. In this paper, we introduce ThermoPixels, a method for designing and 3D printing actively controlled and visually rich thermochromic displays that can be embedded in arbitrary geometries. We investigate the color-changing and thermal properties of thermochromic and conductive filaments. Based on these insights, we designed ThermoPixels and an accompanying software tool that allows embedding ThermoPixels in arbitrary 3D geometries, creating displays of various shapes and sizes (flat, curved, or matrix displays) or displays that embed textures, multiple colors, or that are flexible.

The advent of user-friendly embedded prototyping systems, exemplified by platforms like Arduino, has significantly democratized the creation of interactive devices that combine software programs with electronic hardware. This interconnection between hardware and software, however, makes the identification of bugs very difficult, as problems could be rooted in the program, in the circuit, or at their intersection. While there are tools to assist in identifying and resolving bugs, they typically require hardware instrumentation or visualizing logs in serial monitors. Based on the findings of a formative study, we designed Inline a programming tool that simplifies debugging of embedded systems by making explicit the internal state of the hardware and the program's execution flow using visualizations of the hardware logs directly within the user's code. The system's key characteristics are 1) an inline presentation of logs within the code, 2) real-time tracking of the execution flow, and 3) an expression language to manipulate and filter the logs. The paper presents the detailed implementation of the system and a study with twelve users, which demonstrates what features were adopted and how they were leveraged to complete debugging tasks.

Nowadays, physicians have at their hands a huge amount of data produced by a large set of diagnostic and instrumental tests integrated with data obtained by high-throughput technologies. If such data were opportunely linked and analysed, they might be used to strengthen predictions, so that to improve the prevention and the time-to-diagnosis, reduce the costs of the health system, and bring out hidden knowledge. Machine learning is the principal technique used nowadays to leverage data and gain useful information. However, it has led to various challenges, such as improving the interpretability and explainability of the employed predictive models and integrating expert knowledge into the final system. Solving those challenges is of paramount importance to enhance the trust of both clinicians and patients in the system predictions. To solve the aforementioned issues, in this paper we propose a software workflow able to cope with the trustworthiness aspects of machine learning models and considering a multitude of heterogeneous data and models.

One of the first step in RNA-Sequencing (RNA-Seq) data analysis consists of aligning (Next Generation Sequencing) reads to a reference genome. In literature, there are several tools implemented by practitioners and researchers for the alignment step. However, two tools are the de-facto-standard used by bioinformatics researchers in their pipelines: HISAT (version 2) and STAR (version 2). The aim of this study is to determine the impact of the alignment tool on the RNA -Seq analysis in terms of biological relevance of the results and computational time. The two implemented pipelines return different results on the biological side. This is due to assumptions the used tools made and to the specific characteristics of the underlying (statistical) models. The study provides valuable insights for researchers interested in optimizing their RNA-Seq pipelines and making informed decisions about which pipeline to use. As lesson learned, we suggest bioinformatics researchers to use more pipelines when make experiments to reduce the prediction errors induced by assumption of a specific tool or method.

Online fitness video tutorials are an increasingly popular way to stay fit at home without a personal trainer. However, to keep the screen playing the video in view, users typically disrupt their balance and break the motion flow — two main pillars for the correct execution of yoga poses. While past research partially addressed this problem, these approaches supported only a limited view of the instructor and simple movements. To enable the fluid execution of complex full-body yoga exercises, we propose FlowAR, an augmented reality system for home workouts that shows training video tutorials as always-present virtual static and dynamic overlays around the user. We tested different overlay layouts in a study with 16 participants, using motion capture equipment for baseline performance. Then, we iterated the prototype and tested it in a furnished lab simulating home settings with 12 users. Our results highlight the advantages of different visualizations and the system’s general applicability.

This paper’s goal is to understand the haptic-visual congruency perception of skin-slip on the fingertips given visual cues in Virtual Reality (VR). We developed SpinOcchio (Spin for the spinning mechanism used, Occhio for the Italian word “eye”), a handheld haptic controller capable of rendering the thickness and slipping of a virtual object pinched between two fingers. This is achieved using a mechanism with spinning and pivoting disks that apply a tangential skin-slip movement to the fingertips. With SpinOcchio, we determined the baseline haptic discrimination threshold for skin-slip, and, using these results, we tested how haptic realism of motion and thickness is perceived with varying visual cues in VR. Surprisingly, the results show that in all cases, visual cues dominate over haptic perception. Based on these results, we suggest applications that leverage skin-slip and grip interaction, contributing further to realistic experiences in VR.

Various automated eating detection wearables have been proposed to monitor food intakes. While these systems overcome the forgetfulness of manual user journaling, they typically show low accuracy at outside-the-lab environments or have intrusive form-factors (e.g., headgear). Eyeglasses are emerging as a socially-acceptable eating detection wearable, but existing approaches require custom-built frames and consume large power. We propose MyDJ, an eating detection system that could be attached to any eyeglass frame. MyDJ achieves accurate and energy-efficient eating detection by capturing complementary chewing signals on a piezoelectric sensor and an accelerometer. We evaluated the accuracy and wearability of MyDJ with 30 subjects in uncontrolled environments, where six subjects attached MyDJ on their own eyeglasses for a week. Our study shows that MyDJ achieves 0.919 F1-score in eating episode coverage, with 4.03 × battery time over the state-of-the-art systems. In addition, participants reported wearing MyDJ was almost as comfortable (94.95%) as wearing regular eyeglasses.

The unique behaviors of thermoplastic polymers enable shape-changing interfaces made of 3D printed objects that do not require complex electronics integration. While existing techniques rely on external trigger, such as heat, applied globally on a 3D printed object initiating all at once the shape-changing response (e.g., hot water, heat gun, oven), independent control of multiple parts of the object becomes nearly impossible. We introduce ShrinkCells, a set of shape-changing actuators that enables localized heat to shrink or bend, through combining the properties of two materials — conductive PLA is used to generate localized heat which selectively triggers the shrinking of a Shape Memory Polymer. The unique benefit of ShrinkCells is their capability of triggering simultaneous or sequential shape transformations for different geometries using a single power supply. This results in 3D printed rigid structures that actuate in sequence, avoiding self-collisions when unfolding. We contribute to the body of literature on 4D fabrication by a systematic investigation of selective heating with two different materials, the design and evaluation of the ShrinkCells shape-changing primitives, and applications demonstrating the usage of these actuators.

Current head-mounted displays enable users to explore virtual worlds by simply walking through them (i.e., real-walking VR). This led researchers to create haptic displays that can also simulate different types of elevation shapes. However, existing shape-changing floors are limited by their tabletop scale or the coarse resolution of the terrains they can display due to the limited number of actuators and low vertical resolution. To tackle this challenge, we introduce Elevate, a dynamic and walkable pin-array floor on which users can experience not only large variations in shapes but also the details of the underlying terrain. Our system achieves this by packing 1200 pins arranged on a 1.80 × 0.60m platform, in which each pin can be actuated to one of ten height levels (resolution: 15mm/level). To demonstrate its applicability, we present our haptic floor combined with four walkable applications and a user study that reported increased realism and enjoyment.

Computational notebooks help data analysts analyze and visualize datasets, and share analysis procedures and outputs. However, notebooks typically combine code (e.g., Python scripts), notes, and outputs (e.g., tables, graphs). The combination of disparate materials is known to hinder the comprehension of notebooks, making it difficult for analysts to collaborate with other analysts unfamiliar with the dataset. To mitigate this problem, we introduce ToonNote, a JupyterLab extension that enables the conversion of notebooks into “data comics.” ToonNote provides a simplified view of a Jupyter notebook, highlighting the most important results while supporting interactive and free exploration of the dataset. This paper presents the results of a formative study that motivated the system, its implementation, and an evaluation with 12 users, demonstrating the effectiveness of the produced comics. We discuss how our findings inform the future design of interfaces for computational notebooks and features to support diverse collaborators.

Virtual Reality experiences, such as games and simulations, typically support the usage of bimanual controllers to interact with virtual objects. To recreate the haptic sensation of holding objects of various shapes and behaviors with both hands, previous researchers have used mechanical linkages between the controllers that render adjustable stiffness. However, the linkage cannot quickly adapt to simulate dynamic objects, nor it can be removed to support free movements. This paper introduces GamesBond, a pair of 4-DoF controllers without physical linkage but capable to create the illusion of being connected as a single device, forming a virtual bond. The two controllers work together by dynamically displaying and physically rendering deformations of hand grips, and so allowing users to perceive a single connected object between the hands, such as a jumping rope. With a user study and various applications we show that GamesBond increases the realism, immersion, and enjoyment of bimanual interaction.

Assembling circuits is a challenging and time consuming activity for novice makers, frequently resulting in incorrect placements of wires and components into breadboards. This results in errors that are difficult to identify and debug, and delays that hinder creating, exploring or reconfiguring circuit layouts. This paper presents VirtualWire, a tool that allows users to rapidly design and modify circuits in software and have these changes instantiated in real-time as electrical connections on a physical breadboard. To achieve this, VirtualWire dynamically translates circuit design files into physical connections inside a hardware switching matrix, which handles wiring across breadboard rows and to/from an embedded Arduino. The user can interactively test, tune, and share different circuit layouts for an Arduino shield, and once satisfied, can fabricate the circuit on a permanent substrate. Quantitative and qualitative user studies demonstrate that VirtualWire significantly reduces the time taken for (by 37%), and the number of errors made during (by 53%) circuit assembly, while also supporting users in creating readable, space-efficient and flexible layouts.

On-body electronics and sensors offer the opportunity to seamlessly augment the human with computing power. Accordingly, numerous previous work investigated methods that exploit conductive materials and flexible substrates to fabricate circuits in the form of wearable devices, stretchable patches, and stickers that can be attached to the skin. For all these methods, the fabrication process involves several manual steps, such as designing the circuit in software, constructing conductive patches, and manually placing these physical patches on the body. In contrast, in this work, we propose to fabricate electronics directly on the skin. We present BodyPrinter, a wearable conductive-ink deposition machine, that prints flexible electronics directly on the body using skin-safe conductive ink. The paper describes our system in detail and, through a series of examples and a technical evaluation, we show how direct on-body fabrication of electronic circuits and sensors can further enhance the human body.

Assembling circuits on breadboards using reference designs is a common activity among makers. While tools like Fritzing offer a simplified visualization of how components and wires are connected, such pictorial depictions of circuits are rare in formal educational materials and the vast bulk of online technical documentation. Electronic schematics are more common but are perceived as challenging and confusing by novice makers. To improve access to schematics, we propose SchemaBoard, a system for assisting makers in assembling and inspecting circuits on breadboards from schematic source materials. SchemaBoard uses an LED matrix integrated underneath a working breadboard to visualize via light patterns where and how components should be placed, or to highlight elements of circuit topology such as electrical nets and connected pins. This paper presents a formative study with 16 makers, the SchemaBoard system, and a summative evaluation with an additional 16 users. Results indicate that SchemaBoard is effective in reducing both the time and the number of errors associated with building a circuit based on a reference schematic, and for inspecting the circuit for correctness after its assembly.

Haptic controllers have an important role in providing rich and immersive Virtual Reality (VR) experiences. While previous works have succeeded in creating handheld devices that simulate dynamic properties of rigid objects, such as weight, shape, and movement, recreating the behavior of flexible objects with different stiffness using ungrounded controllers remains an open challenge. In this paper we present ElaStick, a variable-stiffness controller that simulates the dynamic response resulting from shaking or swinging flexible virtual objects. This is achieved by dynamically changing the stiffness of four custom elastic tendons along a joint that effectively increase and reduce the overall stiffness of a perceived object in 2-DoF. We show that with the proposed mechanism, we can render stiffness with high precision and granularity in a continuous range between 10.8 and 71.5Nmm/degree. We estimate the threshold of the human perception of stiffness with a just-noticeable difference (JND) study and investigate the levels of immersion, realism and enjoyment using a VR application.

Prototyping electronic circuits is an increasingly popular activity, supported by researchers, who develop toolkits to improve the design, debugging, and fabrication of electronics. Although past work mainly dealt with circuit topology, in this paper we propose a system for determining or tuning the values of the circuit components. Based on the results of a formative study with seventeen makers, we designed VirtualComponent, a mixed-reality tool that allows users to digitally place electronic components on a real breadboard, tune their values in software, and see these changes applied to the physical circuit in real-time. VirtualComponent is composed of a set of plug-and-play modules containing banks of components, and a custom breadboard managing the connections and components' values. Through demonstrations and the results of an informal study with twelve makers, we show that VirtualComponent is easy to use and allows users to test components' value configurations with little effort.

To better explore the incorporation of pointing and gesturing into ubiquitous computing, we introduce WRIST, an interaction and sensing technique that leverages the dexterity of human wrist motion. WRIST employs a sensor fusion approach which combines inertial measurement unit (IMU) data from a smartwatch and a smart ring. The relative orientation difference of the two devices is measured as the wrist rotation that is independent from arm rotation, which is also position and orientation invariant. Employing our test hardware, we demonstrate that WRIST affords and enables a number of novel yet simplistic interaction techniques, such as (i) macro-micro pointing without explicit mode switching and (ii) wrist gesture recognition when the hand is held in different orientations (e.g., raised or lowered). We report on two studies to evaluate the proposed techniques and we present a set of applications that demonstrate the benefits of WRIST. We conclude with a discussion of the limitations and highlight possible future pathways for research in pointing and gesturing with wearable devices.

Force feedback is said to be the next frontier in virtual reality (VR). Recently, with consumers pushing forward with untethered VR, researchers turned away from solutions based on bulky hardware (e.g., exoskeletons and robotic arms) and started exploring smaller portable or wearable devices. However, when it comes to rendering inertial forces, such as when moving a heavy object around or when interacting with objects with unique mass properties, current ungrounded force feedback devices are unable to provide quick weight shifting sensations that can realistically simulate weight changes over 2D surfaces. In this paper we introduce Aero-plane, a force-feedback handheld controller based on two miniature jet propellers that can render shifting weights of up to 14 N within 0.3 seconds. Through two user studies we: (1) characterize the users' ability to perceive and correctly recognize different motion paths on a virtual plane while using our device; and, (2) tested the level of realism and immersion of the controller when used in two VR applications (a rolling ball on a plane, and using kitchen tools of different shapes and sizes). Lastly, we present a set of applications that further explore different usage cases and alternative form-factors for our device.

This work presents HapticSphere, a wearable spherical surface enabled by bridging a finger and the head-mounted display (HMD) with a passive string. Users perceive a physical support on a finger attached to a string, when extending their arm and reaching out to the string's maximum extension. This physical support assists users in precise touch interaction in the context of stationary and walking virtual or mixed-reality experiences. We propose three methods of attachment of the haptic string (directly on the head or on the body), and illustrate a novel single-step calibration algorithm that supports these configurations by estimating a grand haptic sphere, once a head-coordinated touch interaction is established. Two user studies were conducted to validate our approach and to compare the touch performance with physical support in sitting and walking conditions in the context of mobile mixed-reality scenarios. The results show that, in the walking condition, touch interaction with physical support significantly outperformed the visual-only condition.

Abstract Creating objects with threads is commonly referred to as string art. It is typically a manual, tedious work reserved for skilled artists. In this paper, we investigate how to automatically fabricate string art pieces from one single continuous thread in such a way that it looks like an input image. The proposed system consists of a thread connection optimization algorithm and a custom‐made fabrication machine. It allows casual users to create their own personalized string art pieces in a fully automatic manner. Quantitative and qualitative evaluations demonstrated our system can create visually appealing results.

Smart-rings are ideal for subtle and always-available haptic notifications due to their direct contact with the skin. Previous researchers have highlighted the feasibility of haptic technology in smart-rings and their promise in delivering noticeable stimulations by poking a limited set of planar locations on the finger. However, the full potential of poking as a mechanism to deliver richer and more expressive information on the finger is overlooked. With three studies and a total of 76 participants, we informed the design of PokeRing, a smart-ring capable of delivering information via stimulating eight different locations around the index finger's proximal phalanx. We report our evaluation of the performance of PokeRing in semi-realistic wearable conditions, (standing and walking), and its effective usage for information transfer with twenty-one spatio-temporal patterns designed by six interaction designers in a workshop. Finally, we present three applications that exploit PokeRing's notification usages.

As robotic telepresence is becoming more common and receives academic and industrial attention, the issue of how to increase the expressiveness of the telecommunication is becoming also important. In this paper, we explored the usage of 2D visual patterns displayed on the robot's body as a subsidiary visual channel to enhance communication between two remote users with a robot. Through a series of workshops with seventeen designers, we designed ten user-scenarios and corresponding 2D visual patterns to be used in four different environments: home, office, hospital, and department store. We then presented these patterns on the body of the M4K telepresence robot, and investigate with an online survey the expressiveness of the patterns in each of the proposed environments. Our results show the possibility of complex 2D visual expressions on the body of telepresence robots as a supplementary communication method and introduce a possible strategy for design those. Moreover, our results also show the overall perception and acceptability of these patterns split into three categories (progress indication, attention requests, and emotions) and across different environments.

Smart rings are an emerging wearable technology particularly suitable for discrete notifications based on haptic cues. Previous work mostly focused on tactile actuators that stimulate only specific skin receptors on the finger, resulting in limited information expressiveness. We propose tactoRing, a novel tactile display that, by dragging a small tactor on the skin around the finger, excites multiple skin areas resulting in more accurate cue recognition. In this paper, we present the hardware and a perception study to understand the ability of users to recognize eight distinct points around the finger. Moreover, we show two different techniques to encode information through skin-dragging motion with accuracy up to 94%. We finally showcase a set of applications that, by combining sequences of tactile stimuli, achieve higher expressiveness than prior methods.

SpeCam is a lightweight surface color and material sensing approach for mobile devices which only uses the front-facing camera and the display as a multi-spectral light source. We leverage the natural use of mobile devices (placing it face-down) to detect the material underneath and therefore infer the location or placement of the device. SpeCam can then be used to support discreet micro-interactions to avoid the numerous distractions that users daily face with today's mobile devices. Our two-parts study shows that SpeCam can i) recognize colors in the HSB space with 10 degrees apart near the 3 dominant colors and 4 degrees otherwise and ii) 30 types of surface materials with 99% accuracy. These findings are further supported by a spectroscopy study. Finally, we suggest a series of applications based on simple mobile micro-interactions suitable for using the phone when placed face-down.

Playful learning is a powerful method to enhance children's engagement with teaching material, often resulting in better learning. Several prior works demonstrated the existence of a relation between tangibles and engagement, but they did not characterize it with specific measures. Therefore, this paper aims to describe the degree of engagement with tangible blocks by quantifying children's proactive and passive actions during a learning session. Based on a user study with 36 kindergarten children, our findings show that tangibles support a higher degree of engagement, fosters attention and collaboration, and possibly lead to more active learning.

Shape-changing interfaces are an emerging topic in HCI research: they merge the simplicity of tangible interfaces with the expressiveness of dynamic physical affordances. However, while prior work largely focused on technical aspects and proposed classifications of shape-changing interfaces based on the physical properties of the actuators and the user's levels of control, this work presents a classification of shape-changing interfaces based on the context and identity of the users. After introducing a new prototype for a shape-changing pushbutton, we conducted a series of workshop studies with designers and engineers to explore the design space and potential applications for this interface. We used the result of our workshops to propose a generalized taxonomy of interactions, and built two applications that reflect the proposed model. The paper concludes by highlighting future possible research directions for context and user aware shape-changing interfaces.

The screen size of a smartwatch provides limited space to enable expressive multi-touch input, resulting in a markedly difficult and limited experience. We present WatchMI: Watch Movement Input that enhances touch interaction on a smartwatch to support continuous pressure touch, twist, pan gestures and their combinations. Our novel approach relies on software that analyzes, in real-time, the data from a built-in Inertial Measurement Unit (IMU) in order to determine with great accuracy and different levels of granularity the actions performed by the user, without requiring additional hardware or modification of the watch. We report the results of an evaluation with the system, and demonstrate that the three proposed input interfaces are accurate, noise-resistant, easy to use and can be deployed on a variety of smartwatches. We then showcase the potential of this work with seven different applications including, map navigation, an alarm clock, a music player, pan gesture recognition, text entry, file explorer and controlling remote devices or a game character.

We present RealPen, an augmented stylus for capacitive tablet screens that recreates the physical sensation of writing on paper with a pencil, ball-point pen or marker pen. The aim is to create a more engaging experience when writing on touch surfaces, such as screens of tablet computers. This is achieved by regenerating the friction-induced oscillation and sound of a real writing tool in contact with paper. To generate realistic tactile feedback, our algorithm analyzes the frequency spectrum of the friction oscillation generated when writing with traditional tools, extracts principal frequencies, and uses the actuator's frequency response profile for an adjustment weighting function. We enhance the realism by providing the sound feedback aligned with the writing pressure and speed. Furthermore, we investigated the effects of superposition and fluctuation of several frequencies on human tactile perception, evaluated the performance of RealPen, and characterized users' perception and preference of each feedback type.

PassBYOP is a new graphical password scheme for public terminals that replaces the static digital images typically used in graphical password systems with personalized physical tokens, herein in the form of digital pictures displayed on a physical user-owned device such as a mobile phone. Users present these images to a system camera and then enter their password as a sequence of selections on live video of the token. Highly distinctive optical features are extracted from these selections and used as the password. We present three feasibility studies of PassBYOP examining its reliability, usability, and security against observation. The reliability study shows that image-feature based passwords are viable and suggests appropriate system thresholds - password items should contain a minimum of seven features, 40% of which must geometrically match originals stored on an authentication server in order to be judged equivalent. The usability study measures task completion times and error rates, revealing these to be 7.5 s and 9%, broadly comparable with prior graphical password systems that use static digital images. Finally, the security study highlights PassBYOP's resistance to observation attack - three attackers are unable to compromise a password using shoulder surfing, camera-based observation, or malware. These results indicate that PassBYOP shows promise for security while maintaining the usability of current graphical password schemes.

Tablet computers and portable eReaders are gradually becoming the preferred platform for the consumption of textual materials. However, although these technologies are powerful, it is widely acknowledged that print documents better support the advanced active reading tasks necessary to gain a deep understanding of a text. While prior work to address this issue has aimed improve digital eReaders by either leveraging familiar physical affordances or by extending paper's capabilities with digital tools, in this paper we propose a juncture of these two approaches. We first present a formative study that captures the needs and requirements of users during active reading tasks with tablets. We instantiate the findings in the design of a simple physical aid to support active reading: a smart bookmark. We then define an interaction space for this device, describe a set of interfaces designed to facilitate active reading and close with a user study that assesses the potential of the bookmark device and interaction techniques.

Electronic door locks must be carefully designed to allow valid users to open (or close) a door and prevent unauthorized people from opening (or closing) the door. However, lock manufacturers have often ignored the fact that door locks can be modified by attackers in the real world. In this paper, we demonstrate that the most popular electronic door locks can easily be compromised by inserting a malicious hardware backdoor to perform unauthorized operations on the door locks. Attackers can replay a valid DC voltage pulse to open (or close) the door in an unauthorized manner or capture the user's personal identification number (PIN) used for the door lock.

Tangible systems present compelling interaction opportunities but are typically enabled by complex, bulky, awkward or expensive sensing infrastructures. This hinders their adoption in many application areas. In order to address this issue, this paper explores the use of simple active magnetic tokens that create carefully controlled patterns of varying magnetic flux as the building blocks of tangible systems. We describe the construction of these tokens and a software system capable of detecting their presence and inferring their location based on data sampled from a single triaxial magnetometer a standard component of most current mobile devices. The system can recognize token positions from a set of six pre-calibrated locations with an accuracy of 99%. We describe the hardware and software components of this system and five demonstration applications that illustrate its functionality.

Incorrect postures and long sitting sessions are among the main causes of back pain and discomfort, a problem affecting a growing portion of office workers. In the past, researchers have suggested possible ways to notify with alerts those users who sit improperly, but often such notifications are perceived as intrusive and tend to be ignored. In this paper, we present a flower-shaped physical avatar that subtly and pleasantly (through motion, colors and sounds) provides an ambient feedback to users about their postures. Compared with similar previous work, the flower-avatar described in this paper presents, through better sensing and fine-grain output capabilities, a more expressive ambient media platform that showcases the potential of media ambient avatars in health-care scenarios.

This paper introduces MagPen, a magnetically driven pen interface that works both on and around mobile devices. The proposed device is accompanied by a new vocabulary of gestures and techniques that increase the expressiveness of the standard capacitive stylus. These techniques are: 1) detecting the orientation that the stylus is pointing to, 2) selecting colors using locations beyond screen boundaries, 3) recognizing different spinning gestures associated with different actions, 4) inferring the pressure being applied to the pen, and 5) identifying various pens associated with different operational modes. These techniques are achieved using commonly available smartphones that sense and analyze the magnetic field produced by a permanent magnet embedded in a standard capacitive stylus. This paper explores how magnets can be used to expand the design space of current pen interaction, and proposes a new technology to achieve such results.

This paper introduces VibPress, a software technique that enables pressure input interaction on mobile devices by measuring the level of vibration absorption with the built-in accelerometer when the device is in contact with a damping surface (e.g., user's hands). This is achieved using a real-time estimation algorithm running on the device. Through a user evaluation, we provide evidence that this system is faster than previous software-based approaches, and accurate as hardware-augmented approaches (up to 99.7% accuracy). With this work, we also provide an insight about the maximum number of pressure levels that users can reliably distinguish, reporting usability metrics (time, errors and cognitive load) for different pressure levels and types of gripping gestures (press and squeeze).

Tangible interaction allows the control of digital information through physical artifacts - virtual data is tied to real-world objects. Sensing and display technologies that enable this kind of functionality are typically complex. This represents a barrier to entry for researchers and also restricts where these interaction techniques can be deployed. Addressing these limitations, recent work has explored how the touch screens on mobile devices can be used as sensing and display platforms for tangible interfaces. This paper extends this work by exploring how magnets can be employed to achieve similar ends. To achieve this, it describes the design and construction of eight magnetic appcessories. These are cheap, robust physical interfaces that leverage magnets (and the magnetic sensing built into mobile devices) to support reliable and expressive tangible interactions with digital content.

Draw-a-Secret password schemes, like the Google Android Pattern Lock, entail stroking out a shape on a touch screen. This paper explores techniques for expanding the richness of this input modality (multitouch input, off-target interaction) in order to increase password entropy and resistance to observation. A formative user study highlights user perceptions and usability issues relating to this design space and suggests directions for future development of this concept.

Haptic and audio cues now appear commonly in computer interfaces, partially due to inherent advantages such as their support for eyes-free interaction. Their invisible, unobservable nature also makes them ideal candidates for security interfaces in which users have to enter secret information such as passwords. In particular, researchers have explored this idea through the design of PIN entry authentication systems based on multi-modal combinations of visual and non-visual content or on the recognition of small sets of unimodal haptic or audio stimuli. This paper highlights the benefits and performance limitations of these approaches and introduces an alternative based on unimodal audio or haptic temporal numerosity - the ability to accurately and rapidly determine the number of cues presented in rapid temporal succession. In essence, in a numerosity interface, rather than recognizing distinct cues, users must count the number of times that a single cue occurs. In an iterative process of design and evaluation, three prototypes implementing this concept are presented and studies of their use reported. The results show the fastest PIN entry times and lowest error rates to be 8 s and 2%, figures that improve substantially on previous research. These results are attained while maintaining low levels of workload and substantial resistance to observation attack (as determined via camera attack security studies). In sum, this paper argues that unimodal audio and haptic numerosity is a valuable and relatively unexplored metaphor for non-visual input and demonstrates the validity of this claim in the demanding task of unobservable authentication systems.

Authentication in public spaces, such as ATM PIN entry, is inherently susceptible to security attacks based on observation in person or via cameras. This paper briefly introduces the idea of decoupling the authentication process in two separate sub-tasks (the interaction needed for PIN input and its transmission to the terminal), each with different usability and security goals. In order to support this idea, we present two research projects based on multimodal feedback and physical proximity and explain how they fit into this model.

Tangible user interfaces are portals to digital information. In the future, securing access to such material will be an important concern. This paper describes the design, implementation and evaluation of a PIN entry system based on audio or haptic cues that is suitable for integration into such physical systems. The current implementation links movements on a mobile phone touch screen with the display of non-visual cues; selection of a sequence of these cues composes a password. Studies reveal the validity of this approach in terms of task times and error rates that improve over prior art. In sum, this paper demonstrates the potential of non-visual PINs as a mechanism for securing access to a range of systems, ultimately incorporating mobile, ubiquitous or tangible interfaces.

Research on tangible interaction and digital haptics has rarely intertwined, despite the natural relationship between physicality and touch. This paper addresses this relatively unexplored domain by presenting the Haptic Wheel, a freestanding single-axis rotational controller incorporating vibro-tactile cues. In addition to describing the hardware and implementation, the paper discusses the potential application of the system for eyes-free interaction, password entry and as an active puck on a tabletop system. The paper suggests that systems with active haptic feedback have unexploited potential as tools for tangible interaction.

Authentication in public spaces poses significant security risks. Most significantly, passwords can be stolen, potentially leading to fraud. A common method to steal a PIN is through an observation attack, either using a camera or through direct observation (e.g. shoulder-surfing). This paper addresses this problem by presenting the design and implementation of a novel input keypad which uses tactile cues as means to compose a password. In this system, passwords are encoded as a sequence of randomized vibration patterns, making it visually impossible for an observer to detect which items are selected. An evaluation of this system shows it outperforms previous interfaces which have used tactile feedback to obfuscate passwords.