The haptic properties of textile interfaces support eyes-free use well. However, user interfaces often require composing many controls, which complicates orientation by palpation. We investigated three composition concepts that combine textile components in different ways, using a sample smart home scenario: One uses the select-and-control flow of universal remotes, one resembles the users’ environment, and one divides the interface into application sections. Of the latter, we also created a spacious variant to observe the effect of blank space between interface regions. We first explored how well users understand those composition concepts when first using the interface eyes-free. After familiarization, we measured input performance and preferences. We found task completion times much faster than expected from previous recognition studies. While performance was similar for most interfaces, the select-and-control flow was slower, but participants preferred it after familiarization. From our findings, we derive design recommendations for compound textile UIs and individual components.

Figure 1: In our study, children (top) and adolescents (bottom) redesigned fair versions of three diferent dialogs that originally asked users to accept notifcations (left), to accept cookies (center), or to watch an ad to continue playing (right).We asked participants to nudge users toward acceptance.Redesigns contained no option to decline (C19), visual nudging (A20), colorful designs (C25, A32), increased wait times (C10), and emotional manipulation (A11).English translations are provided in Figure 5.

Smart homes are evolving into complex ecosystems, but their inner workings often remain opaque to users, limiting their ability to troubleshoot and customise the technology. The distributed and hidden components make it challenging for users to understand how these systems work. To design interfaces that enable intuitive troubleshooting for non-experts, we first need a clear understanding of how they form mental models and instinctively diagnose issues. We conducted three iterative vignette studies with realistic smart home scenarios to explore how users perceive interconnectivity in smart home ecosystems and how system complexity impacts their troubleshooting approaches. We identified common smart home topologies envisioned by users and categorised their diagnostic strategies. Our findings show that while users view their smart homes as hierarchical, their mental models are often incomplete, and current interfaces favour functional models, hiding the connections in the system. We provide seven actionable takeaways for designing interfaces that improve user understanding and troubleshooting capabilities.

Dark patterns are malicious UI design strategies that nudge users towards decisions going against their best interests. To create technical countermeasures against them, dark patterns must be automatically detectable. While researchers have devised algorithms to detect some patterns automatically, there has only been little work to use obtained results to technically counter the effects of dark patterns when users face them on their devices.

Machine embroidery is a versatile technique for creating custom and entirely fabric-based patterns on thin and conformable textile surfaces. However, existing machine-embroidered surfaces remain static, limiting the interactions they can support. We introduce Embrogami, an approach for fabricating textile structures with versatile shape-changing behaviors. Inspired by origami, we leverage machine embroidery to form finger-tip-scale mountain-and-valley structures on textiles with customized shapes, bistable or elastic behaviors, and modular composition. The structures can be actuated by the user or the system to modify the local textile surface topology, creating interactive elements like toggles and sliders or textile shape displays with an ultra-thin, flexible, and integrated form factor. We provide a dedicated software tool and report results of technical experiments to allow users to flexibly design, fabricate, and deploy customized Embrogami structures. With four application cases, we showcase Embrogami’s potential to create functional and flexible shape-changing textiles with diverse visuo-tactile feedback.

Personal fabrication has mostly focused on handheld tools as embodied extensions of the user, and machines like laser cutters and 3D printers automating parts of the process without intervention. Although interactive digital fabrication has been explored as a middle ground, existing systems have a fixed allocation of user intervention vs. machine autonomy, limiting flexibility, creativity, and improvisation. We explore a new class of devices that combine the desirable properties of a handheld tool and an autonomous fabrication robot, offering a continuum from manual and assisted to autonomous fabrication, with seamless mode transitions. We exemplify the concept of mixed-initiative physical sketching with a working robotic printer that can be handheld for free-hand sketching, can provide interactive assistance during sketching, or move about for computer-generated sketches. We present interaction techniques to seamlessly transition between modes, and sketching techniques benefitting from these transitions to, e.g., extend (upscale, repeat) or revisit (refine, color) sketches. Our evaluation with seven sketchers illustrates that RoboSketch successfully leverages each mode’s strengths, and that mixed-initiative physical sketching makes computer-supported sketching more flexible.

Textile surfaces, such as on sofas, cushions, and clothes, offer promising alternative locations to place controls for digital devices. Textiles are a natural, even abundant part of living spaces, and support unobtrusive input. While there is solid work on technical implementations of textile interfaces, there is little guidance regarding their design—especially their haptic cues, which are essential for eyes-free use. In particular, icons easily communicate information visually in a compact fashion, but it is unclear how to adapt them to the haptics-centric textile interface experience. Therefore, we investigated the recognizability of 84 haptic icons on fabrics. Each combines a shape, height profile (raised, recessed, or flat), and affected area (filled or outline). Our participants clearly preferred raised icons, and identified them with the highest accuracy and at competitive speeds. We also provide insights into icons that look very different, but are hard to distinguish via touch alone.

In handheld AR, users have only a small screen to see the augmented scene, making decisions about scene layout and rendering techniques crucial. Traditional device-perspective rendering (DPR) uses the device camera's full field of view, enabling fast scene exploration, but ignoring what the user sees around the device screen. In contrast, user-perspective rendering (UPR) emulates the feeling of looking through the device like a glass pane, which enhances depth perception, but severely limits the field of view in which virtual objects are displayed, impeding scene exploration and search. We introduce the notion of User-Aware Rendering. By following the principles of UPR, but pretending the device is larger than it actually is, it combines the strengths of UPR and DPR. We present two studies showing that User-Aware AR imitating a 50% larger device successfully achieves both enhanced depth perception and fast scene exploration in typical search and selection tasks.

Toolkits like the Arduino system have brought embedded programming to STEM education. However, learning embedded programming is still hard, requiring an understanding of coding, electronics, and how both sides interact. To investigate the opportunities of using a different programming paradigm than the imperative approach to learning embedded coding, we developed Flowboard . Students code in a visual iPad editor using flow-based programming , which is conceptually closer to circuit diagrams than imperative code. Two breadboards with I/O pins mirrored on the iPad connect electronics and program graph more seamlessly than existing IDEs. Program changes take effect immediately. This liveness reflects circuit behavior better than edit-compile-run loops. A first study confirmed that students can solve basic embedded programming tasks with Flowboard while highlighting important differences to a typical imperative IDE, Ardublock. A second, in-depth study provided qualitative insights into Flowboard’s impact on students’ conceptual models of electronics and embedded programming and exploring.

Textile interfaces enable designers to integrate unobtrusive media and smart home controls into furniture such as sofas. While the technical aspects of such controllers have been the subject of numerous research projects, the physical form factor of these controls has received little attention so far. This work investigates how general design properties, such as overall slider shape, raised vs. recessed sliders, and number and layout of tick marks, affect users’ preferences and performance. Our first user study identified a preference for certain design combinations, such as recessed, closed-shaped sliders. Our second user study included performance measurements on variations of the preferred designs from study 1, and took a closer look at tick marks. Tick marks supported orientation better than slider shape. Sliders with at least three tick marks were preferred, and performed well. Non-uniform, equally distributed tick marks reduced the movements users needed to orient themselves on the slider.

We present ANISMA, a software and hardware toolkit to prototype on-skin haptic devices that generate skin deformation stimuli like pressure, stretch, and motion using shape-memory alloys (SMAs). Our toolkit embeds expert knowledge that makes SMA spring actuators more accessible to human–computer interaction (HCI) researchers. Using our software tool, users can design different actuator layouts, program their spatio-temporal actuation and preview the resulting deformation behavior to verify a design at an early stage. Our toolkit allows exporting the actuator layout and 3D printing it directly on skin adhesive. To test different actuation sequences on the skin, a user can connect the SMA actuators to our customized driver board and reprogram them using our visual programming interface. We report a technical analysis, verify the perceptibility of essential ANISMA skin deformation devices with 8 participants, and evaluate ANISMA regarding its usability and supported creativity with 12 HCI researchers in a creative design task.

Audiovisual recordings of user studies and interviews provide important data in qualitative HCI research. Even when a textual transcription is available, researchers frequently turn to these recordings due to their rich information content. However, the temporal, unstructured nature of audiovisual recordings makes them less efficient to work with than text. Through interviews and a survey, we explored how HCI researchers work with audiovisual recordings. We investigated researchers’ transcription and annotation practice, their overall analysis workflow, and the prevalence of direct analysis of audiovisual recordings. We found that a key task was locating and analyzing inspectables, interesting segments in recordings. Since locating inspectables can be time consuming, participants look for detectables, visual or auditory cues that indicate the presence of an inspectable. Based on our findings, we discuss the potential for automation in locating detectables in qualitative audiovisual analysis.

Data scientists experiment heavily with their code, compromising code quality to obtain insights faster. We observed ten data scientists perform hypothesis-driven data science tasks, and analyzed their coding, commenting, and analysis practice. We found that they have difficulty keeping track of their code experiments. When revisiting exploratory code to write production code later, they struggle to retrace their steps and capture the decisions made and insights obtained, and have to rerun code frequently. To address these issues, we designed TRACTUS, a system extending the popular RStudio IDE, that detects, tracks, and visualizes code experiments in hypothesis-driven data science tasks. TRACTUS helps recall decisions and insights by grouping code experiments into hypotheses, and structuring information like code execution output and documentation. Our user studies show how TRACTUS improves data scientists' workflows, and suggest additional opportunities for improvement. TRACTUS is available as an open source RStudio IDE addin at http://hci.rwth-aachen.de/tractus.

In Handheld Augmented Reality, users look at AR scenes through the smartphone held in their hand. In this setting, having a mid-air pointing device like a pen in the other hand greatly expands the interaction possibilities. For example, it lets users create 3D sketches and models while on the go. However, perceptual issues in Handheld AR make it difficult to judge the distance of a virtual object, making it hard to align a pen to it. To address this, we designed and compared different visualizations of the pen's position in its virtual environment, measuring pointing precision, task time, activation patterns, and subjective ratings of helpfulness, confidence, and comprehensibility of each visualization. While all visualizations resulted in only minor differences in precision and task time, subjective ratings of perceived helpfulness and confidence favor a 'heatmap' technique that colors the objects in the scene based on their distance to the pen.

We present Headbang, an interaction technique that enriches touch input on handheld devices through slight head movement gestures. This way, users can easily execute shortcuts, like Copy, Paste, or Share, to on-screen targets while touching them. Headbang utilizes the capabilities of commodity smartphones to track the user’s head with their front facing cameras. We evaluated Headbang in two studies and show that the system can be reliably used while sitting and walking and offers a similar accuracy and speed as touch interaction.

Adding a mid-air pen to Handheld Augmented Reality creates a new kind of bimanual interaction for which many fundamental interaction design questions have not been answered yet. In particular, menus are an essential component in most visual interfaces, but it is unclear how to best interact with them in this setting: using the pen in mid-air or on a surface, using the touchscreen, or by moving the smartphone itself. We compared basic menus for these methods by analyzing success rates, selection times, device movement, and subjective ratings. Our results indicate that interacting with a mid-air menu using the pen, and operating a menu with the hand holding the smartphone, are sufficiently competitive to the current standard of two-handed touchscreen interaction, so that interaction designers can freely choose among them based on the interaction context of their application.

Data workers are non-professional data scientists who often use scripting languages like R, Python, or MATLAB, and employ an exploratory programming workflow. Current IDEs offer them two main programming modalities: script files and computational notebooks. To understand how these modalities impact work practice, we conducted a study with 21 data workers, and a subsequent larger survey with 62 respondents. Through interviews, walkthroughs, and screen recordings, we collected information about their workflows. Our analysis shows a tension between scripts and computational notebooks. Scripts are more common, better support storage and execution of previous analyses, but hinder experimentation. Notebooks better suit the actual data science workflow, but can become easily unorganized. We discuss how this dual nature of modality usage leads to several issues that affect data workers' workflows, and discuss implications for the design of programming IDEs.

Smartphones are used predominantly one-handed, using the thumb for input. Many smartphones, however, have grown beyond 5". Users cannot tap everywhere on these screens without destabilizing their grip. ForceRay (FR) lets users aim at an out-of-reach target by applying a force touch at a comfortable thumb location, casting a virtual ray towards the target. Varying pressure moves a cursor along the ray. When reaching the target, quickly lifting the thumb selects it. In a first study, FR was 195 ms slower and had a 3% higher selection error than the best existing technique, BezelCursor (BC), but FR caused significantly less device movement than all other techniques, letting users maintain a steady grip and removing their concerns about device drops. A second study showed that an hour of training speeds up both BC and FR, and that both are equally fast for targets at the screen border.

We introduce Springlets, expressive, non-vibrating mechanotactile interfaces on the skin. Embedded with shape memory alloy springs, we implement Springlets as thin and flexible stickers to be worn on various body locations, thanks to their silent operation even on the neck and head. We present a technically simple and rapid technique for fabricating a wide range of Springlet interfaces and computer-generated tactile patterns. We developed Springlets for six tactile primitives: pinching, directional stretching, pressing, pulling, dragging, and expanding. A study placing Springlets on the arm and near the head demonstrates Springlets' effectiveness and wearability in both stationary and mobile situations. We explore new interactive experiences in tactile social communication, physical guidance, health interfaces, navigation, and virtual reality gaming, enabled by Springlets' unique and scalable form factor.

Modeling in Augmented Reality (AR) lets users create and manipulate virtual objects in mid-air that are aligned to their real environment. We present ARPen, a bimanual input technique for AR modeling that combines a standard smartphone with a 3D-printed pen. Users sketch with the pen in mid-air, while holding their smartphone in the other hand to see the virtual pen traces in the live camera image. ARPen combines the pen's higher 3D input precision with the rich interactive capabilities of the smartphone touchscreen. We studied subjective preferences for this bimanual input technique, such as how people hold the smartphone while drawing, and analyzed the performance of different bimanual techniques for selecting and moving virtual objects. Users preferred a bimanual technique casting a ray through the pen tip for both selection and translation. We provide initial design guidelines for this new class of bimanual AR modeling systems.

Laser cutters are 2D tools, but their speed and compatibility with a variety of affordable materials also makes them a frequent choice to create 3D objects. We propose CutCAD, a tool to easily construct simple 3D objects from 2D faces, inspired by the process of paper modeling and magnetic construction kits. The user creates her 3D model by drawing or loading existing 2D shapes, and connecting their edges in the software. CutCAD then automatically resolves the resulting constraints, and folds the faces up into a 3D model that is previewed live. CutCAD also automatically creates the required finger joints based on thickness of the material and dihedral angles, for smooth assembly. Cutouts are easy to add by importing their outlines as vector drawings, and placing them onto faces. After the faces have been cut, CutCAD provides assembly instructions. Observations and feedback from using CutCAD show the resulting process to be easier to understand than traditional 3D modelling. CutCAD is open-source, and has been downloaded over 2,000 times.

Tangibles on multitouch tabletops increase speed, accuracy, and eyes-free operability for individual users, and verbal and behavioral social interaction among multiple users around smaller tables with a shared focus of attention. Modern multitouch tables, however, provide sizes and resolutions that let groups work alongside each other in separate workspaces. But how aware do these users remain of each other's actions, and what impact can tangibles have on their awareness? In our study, groups of 2--4 users around the table played an individual game grabbing their attention as primary task, while they also had to occasionally become aware of other players'actions and react as secondary task. We found that players were significantly more aware of other players'actions using tangibles than those using pure multitouch interaction, indicated by faster reaction times. This effect was especially strong with more players. We close with qualitative user feedback and design recommendations. We found that players were significantly more aware of other players'actions using tangibles than those using pure multitouch interaction, indicated by faster reaction times. This effect was especially strong with more players. We close with qualitative user feedback and design recommendations.

Picking values from long ordered lists, such as when setting a date or time, is a common task on smartphones. However, the system pickers and tables used for this require significant screen space for spinning and dragging, covering other information or pushing it off-screen. The Force Picker reduces this footprint by letting users increase and decrease values over a wide range using force touch for rate-based control. However, changing input direction this way is difficult. We propose three techniques to address this. With our best candidate, Thumb-Roll, the Force Picker lets untrained users achieve similar accuracy as a standard picker, albeit less quickly. Shrinking it to a single table row, 20% of the iOS picker height, slightly affects completion time, but not accuracy. Intriguingly, after 70 minutes of training, users were significantly faster with this minimized Thumb-Roll Picker compared to the standard picker, at the same accuracy and only 6% of the gesture footprint. We close with application examples.

E-Textiles are fabrics that integrate electronic circuits and components. Makers use them to create interactive clothing, furniture, and toys. However, this requires significant manual labor and skills, and using technology-centric design tools. We introduce Sketch&Stitch, an interactive embroidery system to create e-textiles using a traditional crafting approach: Users draw their art and circuit directly on fabric using colored pens. The system takes a picture of the sketch, converts it to embroidery patterns, and sends them to an embroidery machine. Alternating between sketching and stitching, users build and test their design incrementally. Sketch&Stitch features Circuitry Stickers representing circuit boards, components, and custom stitch patterns for wire crossings to insulate, and various textile touch sensors such as pushbuttons, sliders, and 2D touchpads. Circuitry Stickers serve as placeholders during design. Using computer vision, they are recognized and replaced later in the appropriate embroidery phases. We close with technical considerations and application examples.

Wearable toolkits simplify the integration of micro-electronics into fabric. They require basic knowledge about electronics for part interconnections. This technical aspect might be perceived as a barrier. We propose YAWN, a bus-based, modular wearable toolkit that simplifies the interconnection by relying on a pre-fabricated three-wire fabric band. This allows quick reconfiguration, ensures washability, and reduces the number of connection problems.

When people hold their smartphone in landscape orientation, they use their thumbs for input on the frontal touchscreen, while their remaining fingers rest on the back of the device (BoD) to stabilize the grip. We present BackXPress, a new interaction technique that lets users create BoD pressure input with these remaining fingers to augment their interaction with the touchscreen on the front: Users can apply various pressure levels with each of these fingers to enter different temporary "quasi-modes" that are only active as long as that pressure is applied. Both thumbs can then interact with the frontal screen in that mode. We illustrate the practicality of BackXPress with several sample applications, and report our results from three user studies: Study 1 investigated which fingers can be used to exert BoD pressure and found index, middle, and ring finger from both hands to be practical. Study 2 revealed how pressure touches from these six fingers are distributed across the BoD. Study 3 examined user performance for applying BoD pressure (a) during single touches at the front and (b) for 20 seconds while touching multiple consecutive frontal targets. Participants achieved up to 92% pressure accuracy for three separate pressure levels above normal resting pressure, with the middle fingers providing the highest accuracy. BoD pressure did not affect frontal touch accuracy. We conclude with design guidelines for BoD pressure input.

Ubiquitous computing strives to reach the calm computing state where sensors and actuators disappear from the foreground of our surroundings into the fabric of everyday objects. Despite the great progress in embedded technology, artificial interfaces, such as remote controls and touch screens, remain the dominant media for interacting with smart everyday objects. Motivated by recent advancements in smart textile technologies, we investigate the usability and acceptance of fabric-based controllers in the smart home environment. In this article we describe the development and evaluation of three textile interfaces for controlling a motorized recliner armchair in a living room setting. The core of this contribution is the empirical study with twenty participants that contrasted the user experience of three textile-based interaction techniques to a standard remote control. Despite the slightly lower reliability of the textile interfaces, their overall acceptance was higher. The study shows that the hedonic quality and attractiveness of textile interfaces have higher impact on user acceptance compared to pragmatic qualities, such as efficiency, fluidity of interaction, and reliability. Attractiveness profits from the direct and nearly invisible integration of the interaction device into textile objects such as furniture.

Modern smartphones, like iPhone 7, feature touchscreens with co-located force sensing. This makes touch input more expressive, e.g., by enabling single-finger continuous zooming when coupling zoom levels to force intensity. Often, however, the user wants to select and confirm a particular force value, say, to lock a certain zoom level. The most common confirmation techniques are Dwell Time (DT) and Quick Release (QR). While DT has shown to be reliable, it slows the interaction, as the user must typically wait for 1 s before her selection is confirmed. Conversely, QR is fast but reported to be less reliable, although no reference reports how to actually detect and implement it. In this paper, we set out to challenge the low reliability of QR: We collected user data to (1) report how it can be implemented and (2) show that it is as reliable as DT (97.6% vs. 97.2% success). Since QR was also the faster technique and more preferred by users, we recommend it over DT for force confirmation on modern smartphones.

Devices like smartphones, smartwatches, and fitness trackers enable runners to control music, query fitness parameters such as heart rate and speed, or be guided by coaching apps. But while these devices are portable, interacting with them during running is difficult: they usually have small buttons or touchscreens which force the user to slow down to interact with them properly. On-body tapping is an interaction technique that allows users to trigger actions by tapping at different body locations eyes-free. This paper investigates on-body tapping as a potential input technique for runners. We conducted a user study to evaluate where and how accurately runners can tap on their body. We motion-captured participants while tapping locations on their body and running on a treadmill at different speeds. Results show that a uniform layout of five targets per arm and two targets on the abdomen achieved 96% accuracy rate. We present a set of design implications to inform the design of on-body interfaces for runners.

Documentation and unit tests increase software maintainability, but real world software projects rarely have adequate coverage. We hypothesize that, in part, this is because existing authoring tools require developers to adjust their workflows significantly. To study whether improved interaction design could affect unit testing and documentation practice, we created an authoring support tool called Vesta. The main insight guiding Vesta's interaction design is that developers frequently manually test the software they are building. We propose leveraging runtime information from these manual executions. Because developers naturally exercise the part of the code on which they are currently working, this information will be highly relevant to appropriate documentation and testing tasks. In a complex coding task, nearly all documentation created using Vesta was accurate, compared to only 60% of documentation created without Vesta, and Vesta was able to generate significant portions of all tests, even those written manually by developers without Vesta.

Traditional audio guides in museums and similar spaces typically require the visitor to locate a track number at each exhibit and enter it on a keypad. These guides, however, provide no information on the amount of content available. Current mobile devices provide rich output capabilities, and indoor location tracking technology can simplify the selection of content in modern audio guides. In this paper, we compare the keypad-based interface to a map-based interface with and without automatic localization. Through a field study in a local museum with 84 participants, we found that the usability of all versions is rated high, with the keypad interface coming out ahead. Nevertheless, visitors favored the overview of the map and thumbnails to find the right exhibit, while numbers were considered helpful indicators in the real world. Those who used the self-localizing guide preferred it over manually adjusting the map.

Exploring source code history is an important task for software maintenance. Traditionally, source code history is navigated on the granularity of individual files. This is not fine-grained enough to support users in exploring the evolution of individual code elements. We suggest to consider the history of individual elements within the tree structure inherent to source code. A history graph created from these trees then enables new ways to explore events of interest defined by structural changes in the source code. We present Tree Flow, a visualization of these structural changes designed to enable users to choose the appropriate level of detail for the task at hand. In a user study, we show that both Chronicler and the history aware timeline, two prototype systems combining history graph navigation with a traditional source code view, outperform the more traditional history navigation on a file basis and users strongly prefer Chronicler for the exploration of source code.

Using a smartphone touchscreen to control apps mirrored to a distant display is hard, since the user cannot see where she is touching while looking at the distant screen. Tactile landmarks at the back of the phone can mitigate this problem, especially in landscape mode [3]: By moving a finger across these landmarks, the user can haptically estimate the finger position in proportion to the touchscreen. Upon pinching the thumb resting above the touchscreen towards that finger at the back, the finger position is transferred to the front and registered as a touch. However, despite proprioception, this technique leads to a shift between back and front position, denoted as pinch error. We investigated this error using different target locations, device thicknesses, and tilt angles to derive target sizes that can be acquired at a 96% success rate.

Mobile audio augmented reality systems (MAARS) simulate virtual audio sources in a physical space via headphones. While 20 years ago, these required expensive sensing and rendering equipment, the necessary technology has become widely available. Smartphones have become capable to run high-fidelity spatial audio rendering algorithms, and modern sensors can provide rich data to the rendering process. Combined, these constitute an inexpensive, powerful platform for audio augmented reality.

Using a smartphone for touch input to control apps and games mirrored to a distant screen is difficult, as the user cannot see where she is touching while looking at the distant display. We present HaptiCase, an interaction technique that provides back-of-device tactile landmarks that the user senses with her fingers to estimate the location of her finger in relation to the touchscreen. By pinching the thumb resting above the touch- screen to a finger at the back, the finger position is transferred to the front as the thumb touches the screen. In a study, we compared touch performance of different landmark layouts with a regular landmark-free mobile device. Using a land- mark design of dots on a 3x5 grid significantly improves eyes-free tapping accuracy and allows targets to be as small as 17.5 mm---a 14% reduction in target size---to cover 99% of all touches. When users can look at the touchscreen, land- marks have no significant effect on performance. HaptiCase is low-cost, requires no electronics, and works with unmodified software.

Mobile audio augmented reality systems (MAARS) provide a new and engaging modality to present information or to create playful experiences. Using special filters, spatial audio rendering creates the impression that the sound of a virtual source emanates from a certain position in the physical space. So far, most of the implementations of such systems rely on head tracking to create a realistic effect, which requires additional hardware. Recent results indicate that the built-in sensors of a smartphone can be used as source for orientation measurement, reducing deployment to a simple app download. AudioScope presents an alternative interaction technique to create such an experience, using the metaphor of pointing a directional microphone at the environment. In an experiment with 20 users, we compared the time to locate a proximate audio source and the perceived presence in the virtual environment. Results show that there is no significant difference between head-orientation measurement and AudioScope regarding accuracy and perceived presence. This means that MAARS, such as audio guides for museums, do not require special hardware but can run on the visitor's smartphones with standard headphones.

Each step of statistical analysis requires researchers to make decisions based on both statistical knowledge and the knowledge of their own data. For novice analysts, this is cognitively demanding and can lead to mistakes and misinterpretations of the results. We present Statsplorer, a software that helps novices learn and perform inferential statistical tests. It lets the user kick-start data analysis from their research questions. Statsplorer automatically tests necessary statistical assumptions and uses visualizations to guide the user in both selecting statistical tests and interpreting the results. We compared Statsplorer with a statistics lecture and investigated how Statsplorer prepares novices for learning statistics in an AB/BA crossover experiment. The results indicates that using Statsplorer prior to the lecture leads to significantly better test scores in understanding statistical assumptions and choosing appropriate statistical tests. Statsplorer is open-source and is available online at: http://hci.rwth-aachen.de/statsplorer.

Tangible objects on capacitive multi-touch surfaces are usually only detected while the user is touching them. When the user lets go of such a tangible, the system cannot distinguish whether the user just released the tangible, or picked it up and removed it from the surface. We introduce PERCs, persistent capacitive tangibles that "know" whether they are currently on a capacitive touch surface or not. This is achieved by adding a small field sensor to the tangible to detect the touch screen's own, weak electromagnetic touch detection probing signal. Thus, unlike previous designs, PERCs do not get filtered out over time by the adaptive signal filters of the touch screen. We provide a technical overview of the theory be- hind PERCs and our prototype construction, and we evaluate detection rates, timing performance, and positional and angular accuracy for PERCs on a variety of unmodified, commercially available multi-touch devices.Through their affordable circuitry and high accuracy, PERCs open up the potential for a variety of new applications that use tangibles on today's ubiquitous multi-touch devices.

Audio augmented reality systems overlay the physical world with a virtual audio space. Today's smartphones provide enough processing power to create the impression of virtual sound sources being located in the real world. To achieve this, information about the user's location and orientation is necessary which requires additional hardware. In a real-world installation, however, we observed that instead of turning their head to localize sounds, users tend to turn their entire body. Therefore, we suggest to simply measure orientation of the user's body - or even just the mobile device she is holding - to generate the spatial audio.

Using the space above desktop input devices adds a rich new input channel to desktop interaction. Input in this elevated layer has been previously used to modify the granularity of a 2D slider, navigate layers of a 3D body scan above a multitouch table and access vertically stacked menus. However, designing these interactions is challenging because the lack of haptic and direct visual feedback easily leads to input errors. For bare finger input, the user's fingers needs to reliably enter and stay inside the interactive layer, and engagement techniques such as midair clicking have to be disambiguated from leaving the layer. These issues have been addressed for interactions in which users operate other devices in midair, but there is little guidance for the design of bare finger input in this space.

Mobile audio augmented reality systems can be used in a series of applications, e.g., as a navigational aid for visually impaired or as audio guide in museums. The implementation of such systems usually relies on head orientation data, requiring additional hardware in form of a digital compass in the headphones. As an alternative we propose AudioTorch, a system that turns a smartphone into a virtual directional microphone. This metaphor, where users move the device to detect virtual sound sources, allows quick orientation and easy discrimination between proximate sources, even with simplified rendering algorithms. We compare the navigation performance of head orientation measurement to AudioTorch. A lab study with 18 users showed the rate of correctly recognized sources to be significantly higher with AudioTorch than with head-tracking, while task completion times did not differ significantly. The presence in the virtual environment received similar ratings for both conditions.

We report on the behavior of developers working with a live coding environment, which provides information about a program's execution immediately after each change to the source code. The live coding environment we used shows information about each individual source code line, e.g., changed variable values or truth values of conditions. In comparison to developers working in a non-live environment, those working live found and fixed bugs they introduced significantly faster. Further, working live encouraged developers to switch between editing and debugging phases more frequently.

Current IDEs excel at text manipulation, but offer little support for sketching and capturing informal visual artifacts that developers create during their work on the code base. Such artifacts promise to help the examination of existing source bases and the orientation therein when linked up to corresponding code fragments. In this paper, we present a design and prototype how to use linked sketches to assist the the developer in orientating in the code base. Our evaluation with 32 users shows that testers adopt the navigation through linked sketches and refer to the spatial documentation significantly more.

Understanding source code is crucial for successful software maintenance, and navigating the call graph is especially helpful to understand source code [12]. We compared maintenance performance across four different development environments: an IDE without any call graph exploration tool, a Call Hierarchy tool as found in Eclipse, and the tools Stacksplorer [7]and Blaze [11]. Using any of the call graph exploration tools more developers could solve certain maintenance tasks correctly. Only Stacksplorer and Blaze, however, were also able to decrease task completion times, although the Call Hierarchy offers access to a larger part of the call graph. To investigate if this result was caused by a change in navigation behavior between the tools, we used a set of predictive models to create formally comparable descriptions of programmer navigation. The results suggest that the decrease in task completion times has been caused by Stacksplorer and Blaze promoting call graph navigation more than the Call Hierarchy tool.

Touch interfaces provide great flexibility in designing an UI. However, the actual experience is often frustrating due to bad touch recognition. On small systems, we can analyze yaw, roll, and pitch of the finger to increase touch accuracy for a single touch. On larger systems, we need to take additional factors into account as users have more flexibility for their limb posture and need to aim over larger distances. Thus, we investigated how people perform touch sequences on those large touch surfaces. We show that the relative location of the predecessor of a touch has a significant impact on the orientation and position of the touch ellipsis.

Indirect touch systems combine a horizontal touch input surface with a vertical display for output. While this division is ergonomically superior to simple direct-touch displays for many tasks, users are no longer looking at their hands when touching. This requires the system to support an intermediate \'1ctracking\'1d state that lets users aim at objects without triggering a selection, similar to the hover state in mouse-based UIs. We present an empirical analysis of several interaction techniques for indirect touch systems to switch to this intermediate state, and derive design recommendations for incorporating it into such systems.

Although digital media playback and storage have several advantages, many DJs still prefer using vinyl records on turntables due to their direct manipulation and haptic qualities. The physical structure of a traditional vinyl record provides important cues for in-track navigation, such as track length or location of loud and soft passages. Digital vinyl systems use a timecode record to combine the advantages of digital playback with the handling DJs are used to. These records contain a special audio signal that is processed by a computer and mapped to information such as playback speed, direction, and absolute position in a track. However, due to their generic nature, timecode records cannot provide visual information to navigate inside individual tracks. Using top-projection, DiskPlay augments a white timecode record with individual visual cues of the medium, such as cue points or track start and end. In our observational study with four professional DJs, participants valued the co-location of visual feedback with the control vinyl on the turntable.

Direct manipulation video navigation (DMVN) systems allow to navigate inside video scenes by spatially manipulating objects in the video. Problems arise when dealing with temporal ambiguities where a time span is projected onto a single point in image space, e.g., when objects stop moving. Existing DMVN systems deal with these cases by either disabling navigation on the paused object or by allowing jumps in the timeline. Both of these workarounds are undesirable as they introduce inconsistency or provoke loss of context. We analyze current practices regarding temporal ambiguities and introduce two new methods to visualize and navigate object pauses. User tests show that the new approaches are better suited for navigation in scenes containing temporal ambiguities and are rated higher in terms of user satisfaction.

Most presentation software uses the slide deck metaphor to create visual presentation support. Recently, canvas presentation tools such as Fly or Prezi have begun to use a zoomable free-form canvas to arrange information instead. While their effect on authoring presentations has been evaluated previously, we studied how they impact the audience. In a quantitative study, we compared audience retention and macrostructure understanding of slide deck vs. canvas presentations. We found both approaches to be equally capable of communicating information to the audience. Canvas presentations, however, were rated by participants to better aid them in staying oriented during a talk. This makes canvas presentation tools a promising slideware alternative.

Flicking is a common interaction technique to move objects across large interactive surfaces, but little is known about its suitability for use on non-planar, curved surfaces. Flicking consists of two stages: First, visually determining the direction in which to flick the object, then planning and executing the corresponding gesture. Errors in both stages could influence flicking accuracy. We investigated flicking interactions on curved interactive surface to evaluate which type of error influences accuracy. Therefore, we carried out three user studies to analyze how each stage of flicking on a curved surface is influenced. Our main findings are: 1) Flicking gestures are more accurate if horizontal and vertical surface are joined by a continuous curve than if they are separated by an edge or gap. 2) Flicking gestures on curved surfaces are mostly influenced by the motor execution stage of the gesture rather than the visual perception stage. 3) Flicking accuracy decreases as the starting point of the gesture is moved closer to the curve. 4) We conclude with a first mathematical model to estimate the error users will make when flicking across a curve.

We present Pinstripe, a textile user interface element for eyes-free, continuous value input on smart garments that uses pinching and rolling a piece of cloth between your fingers. The input granularity can be controlled in a natural way by varying the amount of cloth pinched. Pinstripe input elements physically consist of fields of parallel conductive lines sewn onto the fabric. This way, they can be invisible, and can be included across large areas of a garment. Pinstripe also addresses several problems previously identified in the placement and operation of textile UI elements on smart clothing. Two user studies evaluate ideal placement and orientation of Pinstripe elements on the users' garments as well as acceptance and perceived ease of use of this novel textile input technique.

Elderly users suffering from hand tremor have difficulties interacting with touchscreens because of finger oscillation. It has been previously observed that sliding one's finger across the screen may help reduce this oscillation. In this work, we empirically confirm this advantage by (1) measuring finger oscillation during different actions and (2) comparing error rate and user satisfaction between traditional tapping and swabbing in which the user slides his finger towards a target on a screen edge to select it. We found that oscillation is generally reduced during sliding. Also, compared to tapping, swabbing resulted in improved error rates and user satisfaction. We believe that swabbing will make touchscreens more accessible to senior users with tremor.

We introduce dynamic physical properties as an additional degree of freedom for passive tabletop controls. Using electromagnetic actuation, we manipulate attributes of tangibles on the fly, such as perceived weight, spring resistance, friction, and latching. We describe our actuation concepts, prototypes, and measurements showing that magnetic fields can change physical effects in a linear way. Controlled experiments reveal that participants can tactually distinguish four rendered resistance levels of a button prototype and easily detect dynamic detents in a continuous slider. Finally, we describe how adjustable physical properties in tangibles can enhance tabletop interaction.

We present Stacksplorer, a new tool to support source code navigation and comprehension. Stacksplorer computes the call graph of a given piece of code, visualizes relevant parts of it, and allows developers to interactively traverse it. This augments the traditional code editor by offering an additional layer of navigation. Stacksplorer is particularly useful to understand and edit unknown source code because branches of the call graph can be explored and backtracked easily. Visualizing the callers of a method reduces the risk of introducing unintended side effects. In a quantitative study, programmers using Stacksplorer performed three of four software maintenance tasks significantly faster and with higher success rates, and Stacksplorer received a System Usability Scale rating of 85.4 from participants.

We introduce FingerFlux, an output technique to generate near-surface haptic feedback on interactive tabletops. Our system combines electromagnetic actuation with permanent magnets attached to the user's hand. FingerFlux lets users feel the interface before touching, and can create both attracting and repelling forces. This enables applications such as reducing drifting, adding physical constraints to virtual controls, and guiding the user without visual output. We show that users can feel vibration patterns up to 35 mm above our table, and that FingerFlux can significantly reduce drifting when operating on-screen buttons without looking.

We present a system for the actuation of tangible magnetic widgets (Madgets) on interactive tabletops. Our system combines electromagnetic actuation with fiber optic tracking to move and operate physical controls. The presented mechanism supports actuating complex tangibles that consist of multiple parts. A grid of optical fibers transmits marker positions past our actuation hardware to cameras below the table. We introduce a visual tracking algorithm that is able to detect objects and touches from the strongly sub-sampled video input of that grid. Six sample Madgets illustrate the capabilities of our approach, ranging from tangential movement and height actuation to inductive power transfer. Madgets combine the benefits of passive, untethered, and translucent tangibles with the ability to actuate them with multiple degrees of freedom.

TypeRight is a new tactile input device for text entry. It combines the advantages of tactile feedback with error prevention methods of word processors. TypeRight extends the standard keyboard so that the resistance to press each key becomes dynamically adjustable through software. Before each keystroke, the resistance of keys that would lead to a typing error according to dictionary and grammar rules is increased momentarily to make them harder to press, thus avoiding typing errors rather than indicating them after the fact. Two user studies showed that TypeRight decreases error correction rates by an average of 46%.

Modern presentation software is still built around interaction metaphors adapted from traditional slide projectors. We provide an analysis of the problems in this application genre that presentation authors face and present Fly, a presentation tool that is based on the idea of planar information structures. Inspired by the natural human thought processes of data chunking, association, and spatial memory, Fly explores authoring of presentation documents.

While learning new motor skills, we often rely on feedback from a trainer. Auditive feedback and demonstrations are used most frequently, but in many domains they are inappropriate or impractical. We introduce tactile instructions as an alternative to assist in correcting wrong posture during physical activities, and present a set of full-body vibrotactile patterns. An initial study informed the design of our tactile patterns, and determined appropriate locations for feedback on the body. A second experiment showed that users perceived and correctly classified our tactile instruction patterns in a relaxed setting and during a cognitively and physically demanding task. In a final experiment, snowboarders on the slope compared their perception of tactile instructions with audio instructions under real-world conditions. Tactile instructions achieved overall high recognition accuracy similar to audio instructions. Moreover, participants responded quicker to instructions delivered over the tactile channel than to instructions presented over the audio channel. Our findings suggest that these full-body tactile feedback patterns can replace audio instructions during physical activities.

We present Silicone iLluminated Active Peripherals (SLAP), a system of tangible, translucent widgets for use on multitouch tabletops. SLAP Widgets are cast from silicone or made of acrylic, and include sliders, knobs, keyboards, and buttons. They add tactile feedback to multi-touch tables, improving input accuracy. Using rear projection, SLAP Widgets can be relabeled dynamically, providing inexpensive, battery-free, and untethered augmentations. Furthermore, SLAP combines the flexibility of virtual objects with physical affordances. We evaluate how SLAP Widgets influence the user experience on tabletops compared to virtual controls. Empirical studies show that SLAPWidgets are easy to use and outperform virtual controls significantly in terms of accuracy and overall interaction time.

We present PocketDRAGON, a demonstrator prototype that allows direct manipulation video navigation on mobile touchscreen devices. In contrast to traditional video navigation techniques, PocketDRAGON does not require any overlay UI elements that occupy valuable screen real estate and obstruct the users' view on the video. Also, direct manipulation video navigation techniques have been shown to compare favorably to the established timeline slider interfaces in terms of performance times, intuitiveness, precision, and perceived ease of use. Our demonstrator system still uses a backend server for the computationally expensive parts of the algorithms but delivers the full-fledged user experience on the mobile device.

Tactile motion instructions are vibrotactile feedback patterns delivered across the entire body that indicate how to move during physical activities. This work investigates the perception and identification of such patterns, based on two different metaphors, under stationary and active situations. We further combine and sequentially trigger different patterns to explore whether tactile motion instructions are understandable as a simple language. A tactile language could represent motion sequences to guide students during demanding exercises. Finally, the presented studies provide insights into perception and interpretation of tactile feedback and help to inform a design space for full-body vibrotactile cues.

We present SLAPbook, an application using SLAP, translucent and tangible widgets for use on vision-based multi-touch tabletops in Single Display Groupware (SDG) environments. SLAP stands for Silicone ILluminated Active Peripherals and includes widgets such as sliders, knobs, keyboards, and buttons. The widgets and tactile feedback to multi-touch tables while simultaneously providing dynamic relabeling to tangible objects using the table's rear projection. SLAPbook provides multiple users the ability to add and edit content to a guestbook, browse other peoples' entries, and access personal data using a token-based personalization system. Interaction with the table takes place in the personal and public space so that users can make use of personal and shared controls to perform separate and coordinative actions.

We present DRAGON, a direct manipulation interaction technique for frame-accurate navigation in video scenes. This technique benefits tasks such as professional and amateur video editing, review of sports footage, and forensic analysis of video scenes. By directly dragging objects in the scene along their movement trajectory, DRAGON enables users to quickly and precisely navigate to a specific point in the video timeline where an object of interest is in a desired location. Examples include the specific frame where a sprinter crosses the finish line, or where a car passes a traffic light. Through a user study, we show that DRAGON significantly reduces task completion time for in-scene navigation tasks by an average of 19-42% compared to a standard timeline slider. Qualitative feedback from users is also positive, with multiple users indicating that the DRAGON interaction felt more natural than the traditional timeline slider for in-scene navigation.

iStuff Mobile is the first rapid prototyping framework that helps explore new sensor-based interfaces with existing mobile phones. It focuses on sensor-enhanced physical interfaces for ubiquitous computing scenarios. The framework includes sensor network platforms, mobile phone software, and a proven rapid prototyping framework. Interaction designers can use iStuff Mobile to quickly create and test functional prototypes of novel interfaces without making internal hardware or software modifications to the handset. A visual programming paradigm provides a low threshold for prototyping activities: the system is not difficult to learn. At the same time, the range of examples built using the toolkit demonstrates a high ceiling for prototyping activities: the toolkit places few limits on prototype complexity. A user study shows that the visual programming metaphor enables prototypes to be built faster and encourages more iterations than a previous approach.

Designing interactive conducting exhibits for public spaces poses unique challenges, primarily because the conceptual model of conducting music varies amongst users. In a user study, we compared how conductors and non-conductors place their beats when conducting to a fixed orchestral recording of Radetzky March, and found significant differences between these two groups. Conductors lead the actual music beat with their gestures by an average of 150 ms, compared to 50 ms for non-conductors; non-conductors also vary their placement of the beat 50% more than conductors. Furthermore, we found differences in how users conceptually mapped their gestures to the music, such as conducting to the musical rhythm rather than to the beat. We are incorporating these results into an upcoming conducting system for public spaces to increase its usability; we believe they also apply to a more general class of musical gestures such as dance.

The iStuff toolkit of physical devices, and the flexible software infrastructure to support it, were designed to simplify the exploration of novel interaction techniques in the post-desktop era of multiple users, devices, systems and applications collaborating in an interactive environment. The toolkit leverages an existing interactive workspace in-frastructure, making it lightweight and platform independent. The supporting software framework includes a dynamically configurable intermediary to simplify the mapping of devices to applications. We describe the iStuff architecture and provide several examples of iStuff, organized into a design space of ubiquitous computing interaction components. The main contribution is a physical toolkit for distributed, heterogeneous environments with run-time retargetable device data flow. We conclude with some insights and experiences derived from using this toolkit and framework to prototype experimental interaction techniques for ubiquitous computing environments.

Personal Orchestra is the first system to let users conduct\nan actual audio and video recording of an orchestra, using\nan infrared baton to control tempo, volume, and instrument\nsections. A gesture recognition algorithm interprets user input,\nand a novel high-fidelity playback algorithm renders audio\nand video data at variable speed without time-stretching\nartifacts. The system is installed as a public exhibit in the\nHOUSE OF MUSIC VIENNA.

Article Free Access Share on WorldBeat: designing a baton-based interface for an interactive music exhibit Author: Jan O. Borchers Telecooperation Research Group, Department of Computer Science, Linz University, 4040 Linz, Austria Telecooperation Research Group, Department of Computer Science, Linz University, 4040 Linz, AustriaView Profile Authors Info & Claims CHI '97: Proceedings of the ACM SIGCHI Conference on Human factors in computing systemsMarch 1997 Pages 131–138https://doi.org/10.1145/258549.258632Published:27 March 1997Publication History 24citation829DownloadsMetricsTotal Citations24Total Downloads829Last 12 Months40Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF