Uncertainty caused by instrument tracking errors affects critical tasks such as surgery assisted by Augmented Reality (AR) guidance. This work investigates whether visualizing such uncertainty can improve task performance, trust, and confidence. We present four visualization techniques: Cone, Circle, Gauge, and Color. A two-part study evaluated these techniques on a surgical drilling task, first with 24 non-professional participants and then with 4 professional surgeons. Results indicate that uncertainty visualization improved drilling accuracy by 24% but increased task time by 76%. It also enhanced user confidence and trust in the system, with Cone and Circle as the most preferred visualizations. Based on our findings, we discuss design recommendations for integrating uncertainty visualization into AR-based surgical systems. This work paves the way for a higher success rate in surgical procedures.

Word suggestion is a common feature of typing interfaces, but previous studies have found unclear or negative impacts. We report on three studies controlling for word suggestion accuracy and typing efficiency. Our accuracy factor uses a new methodology based on common word suggestion metrics. Typing efficiency is controlled by device type in the first study, and by artificial impairments in the following two. Results show that suggestions are used less as typing efficiency increases, and only improve speed when highly accurate, even with low typing efficiency. Inline suggestions save about 4% more keystrokes and increase typing speed by 2 words per minute compared to a bar suggestions, though they are more distracting. Based on our findings, we propose a model linking suggestion usage to accuracy and typing speed, and discuss implications for designing automation features in typing systems.

International audience

We investigate performance characteristics when switching between four pointing methods: absolute touch, absolute pen, relative mouse, and relative trackpad. The established “subtraction method” protocol used in mode-switching studies is extended to test pairs of methods and accommodate switch direction, multiple baselines, and controlling relative cursor position. A first experiment examines method switching on and around the horizontal surface of a tablet. Results find switching between pen and touch is fastest, and switching between relative and absolute methods incurs additional time penalty. A second experiment expands the investigation to an emerging foldable all-screen laptop form factor where switching also occurs on an angled surface and along a smoothly curved hinge. Results find switching between trackpad and touch is fastest, with all switching times generally higher. Our work contributes missing empirical evidence for switching performance using modern input methods, and our results can inform interaction design for current and emerging device form factors.

Suggesting words to complete a given sequence of characters is a common feature of typing interfaces. Yet, previous studies have not found a clear benefit, some even finding it detrimental. We report on the first study to control for two important factors, word suggestion accuracy and typing efficiency. Our accuracy factor is enabled by a new methodology that builds on standard metrics of word suggestions. Typing efficiency is based on device type. Results show word suggestions are used less often in a desktop condition, with little difference between tablet and phone conditions. Very accurate suggestions do not improve entry speed on desktop, but do on tablet and phone. Based on our findings, we discuss implications for the design of automation features in typing systems.

Conversational Agents (CA) have become one of the common user interfaces in many online domains. In this paper, we ask whether users have a preference about the personality of CAs, and whether this preference changes depending on the length and type of the tasks CAs are used for. In an online study (N = 410), we investigated three different CA personalities (introvert, extrovert, and non-personified) in four different tasks with different natures and lengths (teaching, booking, todo, and weather). Most of the participants preferred to interact with a conversational agent (introvert or extrovert) as opposed to a non-personified interface, regardless of their own personality. Results suggested that this preference may be task dependent: when CA’s goal was to provide information, participants preferred an extrovert agent. We did not observe a difference between the preference for introvert and extrovert agents when the task’s goal was to complete an assignment.

We investigate how users touch arbitrary shapes. First, we performed semi-structured interviews with a fifteen-shape set as prop to identify touch strategies. Results reveal four main potential touch strategies, from which we devised nine mathematical candidate models. We investigate the ability of these models to predict human behaviour in a controlled experiment. We found that the center of a shape’s bounding box best approximates a user’s target location when touching arbitrary shapes. Our findings not only invite designers to use a larger variety of shapes, but can also be used to design touch interaction adapted to user behaviour using our model. As an example, they are likely to be valuable for the creation of applications exposing shapes of various complexities, like drawing applications.

Bau and Mackays OctoPocus dynamic guide helps novices learn, execute, and remember 2D surface gestures. We adapt OctoPocus to 3D mid-air gestures in Virtual Reality (VR) using an optimization-based recognizer, and by introducing an optional exploration mode to help visualize the spatial complexity of guides in a 3D gesture set. A replication of the original experiment protocol is used to compare OctoPocus in VR with a VR implementation of a crib-sheet. Results show that despite requiring 0.9s more reaction time than crib-sheet, OctoPocus enables participants to execute gestures 1.8s faster with 13.8 percent more accuracy during training, while remembering a comparable number of gestures. Subjective ratings support these results, 75 percent of participants found OctoPocus easier to learn and 83 percent found it more accurate. We contribute an implementation and empirical evidence demonstrating that an adaptation of the OctoPocus guide to VR is feasible and beneficial.

We investigate the performance of one-handed touch input on the side of a mobile phone. A first experiment examines grip change and subjective preference when reaching for side targets using different fingers. Results show all locations can be reached with at least one finger, but the thumb and index are most preferred and require less grip change for positions along the sides. Two following experiments examine taps and flicks using the thumb and index finger in a new two-dimensional input space. A side-touch sensor is simulated with a combination of capacitive sensing and motion tracking to distinguish touches on the lower, middle, or upper edges. When tapping, index and thumb speeds are similar with thumb more accurate and comfortable, and the lower edge is most reliable with the middle edge most comfortable. When flicking with the thumb, the upper edge is fast and rated highly.

When automating tasks using some form of artificial intelligence, some inaccuracy in the result is virtually unavoidable. In many cases, the user must decide whether to try the automated method again, or fix it themselves using the available user interface. We argue this decision is influenced by both perceived automation accuracy and degree of task "controllability" (how easily and to what extent an automated result can be manually modified). This relationship between accuracy and controllability is investigated in a 750-participant crowdsourced experiment using a controlled, gamified task. With high controllability, self-reported satisfaction remained constant even under very low accuracy conditions, and overall, a strong preference was observed for using manual control rather than automation, despite much slower performance and regardless of very poor controllability.

Previous research demonstrated the ability for users to accurately recognize Spatiotemporal Vibrotactile Patterns (SVP): sequences of vibrations on different motors occurring either sequentially or simultaneously. However, the experiments were only run in a lab setting and the ability for users to recognize SVP in a real-world environment remains unclear. In this paper, we investigate how several factors may affect recognition: (1) physical activity (running), (2) cognitive task (i.e. primary task, typing), (3) distribution of the vibration motors across body parts and (4) temporality of the patterns. Our results suggest that physical activity has very little impact, specifically compared to cognitive task, location of the vibrations or temporality. We discuss these results and propose a set of guidelines for the design of SVPs.

Wearables are increasingly used during training to quantify performance and provide valuable real-time information. However, interacting with these devices in motion may disrupt the movements of the activity. We propose a method of interaction involving tapping specific locations on the body, identify candidate locations for running and cycling, and compare them in a series of controlled experiments with athletes. A purpose-built prototype measures speed of interaction and gives feedback cues for athletes to report the physical effects on the activity itself. Our results suggest that specific locations are faster and have minimal disruption to movement, even under induced fatigue conditions. The overall method is fast - 1.31s for running and 1.65s for cycling. Preferred locations differ significantly across sports, with stable body parts ranking higher. We effectively demonstrated the use of a single hand for interaction during running with two distinct tap gestures. A set of guidelines inform the design of new sports technologies.

We present Follow-My-Lead, an alternative indoor navigation technique that uses visual information recorded on an actual navigation path as a navigational guide. Its design revealed a trade-off between the fidelity of information provided to users and their effort to acquire it. Our first experiment revealed that scrolling through a continuous image stream of the navigation path is highly informative, but it becomes tedious with constant use. Discrete image checkpoints require less effort, but can be confusing. A balance may be struck by adding fast video transitions between image checkpoints, but precise control is required to handle difficult situations. Authoring still image checkpoints is also difficult, and this inspired us to invent a new technique using video checkpoints. We conducted a second experiment on authoring and navigation performance and found video checkpoints plus fast video transitions to be better than both image checkpoints plus fast video transitions and traditional written instructions.

We propose Augmented Letters, a new technique aimed at augmenting gesture-based techniques such as Marking Menus [9] by giving them natural, mnemonic associations. Augmented Letters gestures consist of the initial of command names, sketched by hand in the Unistroke style, and affixed with a straight tail. We designed a tentative touch device interaction technique that supports fast interactions with large sets of commands, is easily discoverable, improves user's recall at no speed cost, and supports fluid transition from novice to expert mode. An experiment suggests that Augmented Letters outperform Marking Menu in terms of user recall.