With an increasing ability to answer and fulfill user requests, voice-enabled Conversational Agents (CAs) are becoming more and more powerful. However, as the complexity of the requests increase, the time for the CAs to process and fulfill the tasks can become longer. In other cases where input prediction is available, some requests can be processed and answered even before the user is finished saying the command. However, the effects of these positive and negative delays in system response time are still under-explored. In this paper, we systematically analyze the effects of different response delays on usability and acceptability considering three common interaction techniques for voice-enabled CAs. Our results reveal that an unnaturally long positive delay in system response time leads users to assume that an error occurred, while a negative delay is perceived by the users as rude. Based on our findings, we present design guidelines for voice-enabled CAs.

After their success in the smart home, voice assistants are becoming increasingly popular in automotive user interfaces. These voice assistants are traditionally designed to provide a human-like dialog with the user. Thus, when processing voice input, especially dealing with uncertainty is an important factor that needs to be considered when designing system responses. While state-of-the-art voice assistants offer responses based on their certainty of what they understood, these response-thresholds are largely under-explored. In this work, we close this gap by providing a user-centered approach to investigate which responses are acceptable for voice input users depending on input certainty. Through findings from semi-structured online interviews with 101 participants, we provide insights about designing voice user interface responses based on system certainty. Our findings reveal a sweet spot for executing a task versus requesting additional user input. Further, we provide data-driven guidelines for different in-car voice assistant behaviors.

Recent technological advances have made head-mounted displays (HMDs) smaller and untethered, fostering the vision of ubiquitous interaction in a digitally augmented physical world. Consequently, a major part of the interaction with such devices will happen on the go, calling for interaction techniques that allow users to interact while walking. In this paper, we explore lateral shifts of the walking path as a hands-free input modality. The available input options are visualized as lanes on the ground parallel to the user's walking path. Users can select options by shifting the walking path sideways to the respective lane. We contribute the results of a controlled experiment with 18 participants, confirming the viability of our approach for fast, accurate, and joyful interactions. Further, based on the findings of the controlled experiment, we present three example applications.

Using auditory input while driving is becoming increasingly popular for making distraction-free inputs while driving. However, we argue that auditory input is more than just using speech. Thus, in this work, we explore using Non-Verbal Auditory Input (NVAI) for interacting with smart assistants while driving. Through an online study with 100 participants, we initially investigated users' input preferences for binary, discrete, and continuous data types. After identifying the top three modalities for NVAI, we subsequently conducted an in-person study with 16 participants. In our study, the participants tested these input modalities for three different input data types regarding their accuracy, driver-distraction, and social acceptability, while operating a driving simulator. The results reveal that, although clapping hands for making input was initially preferred in our online survey, it is snapping fingers for binary input and discrete input and humming for making continuous input that is the preferred NVAI modality while driving.

Up to 20% of residential votes and up to 70% of absentee votes in Switzerland are cast online. The Swiss system aims to provide individual verifiability by different verification codes. The voters have to carry out verification on their own, making the usability and UX of the interface of great importance. To improve the usability, we first performed an evaluation with 12 human-computer interaction experts to uncover usability weaknesses of the Swiss Internet voting interface. Based on the experts' findings, related work, and an exploratory user study with 36 participants, we propose a redesign that we evaluated in a user study with 49 participants. Our study confirmed that the redesign indeed improves the detection of incorrect votes by 33% and increases the trust and understanding of the voters. Our studies furthermore contribute important lessons for designing verifiable e-voting systems in general.

Virtual Reality Environments (VRE) create an immersive user experience through visual, aural, and haptic sensations. However, the latter is often limited to vibrotactile sensations that are not able to actively provide kinesthetic motion actuation. Further, such sensations do not cover natural representations of physical forces, for example, when lifting a weight. We present PneumAct, a jacket to enable pneumatically actuated kinesthetic movements of arm joints in VRE. It integrates two types of actuators inflated through compressed air: a Contraction Actuator and an Extension Actuator. We evaluate our PneumAct jacket through two user studies with a total of 32 participants: First, we perform a technical evaluation measuring the contraction and extension angles of different inflation patterns and inflation durations. Second, we evaluate PneumAct in three VRE scenarios comparing our system to traditional controller-based vibrotactile and a baseline without haptic feedback.

With the proliferation of room-scale Virtual Reality (VR), more and more users install a VR system in their homes. When users are in VR, they are usually completely immersed in their application. However, sometimes passersby invade these tracking spaces and walk up to users that are currently immersed in VR to try and interact with them. As this either scares the user in VR or breaks the user's immersion, research has yet to find a way to seamlessly represent physical passersby in virtual worlds. In this paper, we propose and evaluate three different ways to represent physical passersby in a Virtual Environment using Augmented Virtuality. The representations encompass showing a Pointcloud, showing a 3D-Model, and showing an Image Overlay of the passerby. Our results show that while an Image Overlay and a 3D-Model are the fastest representations to spot passersby, the 3D-Model and the Pointcloud representations were the most accurate.

From voice commands and air taps to touch gestures on frames: Various techniques for interacting with head-mounted displays (HMDs) have been proposed. While these techniques have both benefits and drawbacks dependent on the current situation of the user, research on interacting with HMDs has not concluded yet. In this paper, we add to the body of research on interacting with HMDs by exploring foot-tapping as an input modality. Through two controlled experiments with a total of 36 participants, we first explore direct interaction with interfaces that are displayed on the floor and require the user to look down to interact. Secondly, we investigate indirect interaction with interfaces that, although operated by the user's feet, are always visible as they are floating in front of the user. Based on the results of the two experiments, we provide design recommendations for direct and indirect foot-based user interfaces.

Room-scale Virtual Reality (VR) systems have arrived in users' homes where tracked environments are set up in limited physical spaces. As most Virtual Environments (VEs) are larger than the tracked physical space, locomotion techniques are used to navigate in VEs. Currently, in recent VR games, point & teleport is the most popular locomotion technique. However, it only allows users to select the position of the teleportation and not the orientation that the user is facing after the teleport. This results in users having to manually correct their orientation after teleporting and possibly getting entangled by the cable of the headset. In this paper, we introduce and evaluate three different point & teleport techniques that enable users to specify the target orientation while teleporting. The results show that, although the three teleportation techniques with orientation indication increase the average teleportation time, they lead to a decreased need for correcting the orientation after teleportation.

Hover, touch, and force are promising input modalities that get increasingly integrated into screens and everyday objects. However, these interactions are often limited to flat surfaces and the integration of suitable sensors is time-consuming and costly. To alleviate these limitations, we contribute Trilaterate: A fabrication pipeline to 3D print custom objects that detect the 3D position of a finger hovering, touching, or forcing them by combining multiple capacitance measurements via capacitive trilateration. Trilaterate places and routes actively-shielded sensors inside the object and operates on consumer-level 3D printers. We present technical evaluations and example applications that validate and demonstrate the wide applicability of Trilaterate.

With smartphones being a prime example, touchscreens became one of the most widely used interface to interact with computing systems. Compared to other touchscreen devices, smartphones pose additional challenges as the hand that interacts with the device is commonly used to also hold the device. Consequently, determining how fingers of the hand holding the device can interact with the screen is a non-trivial challenge. A body of recent work investigated the comfortable area in controlled lab studies. This poses limitations as it is based on the assumption that the grips used in the studies are representative for normal smartphone use. In this paper, we extend previous work by providing insights from in-the-wild studies using two different apps that were deployed in the Android App Store. Comparing our results with previous work we confirm that our data fits previously proposed models. Further analyzing the data, we highlight the sweet spot, the position that is touched if the input can be performed on the whole screen.

Using assistance technology always comes with the challenge of social acceptability. While an accessibility device might have great implications for a person with disabilities, it might come with unpleasant social implications. In this paper, we want to assess the social implications of flying companion quadcopters for navigating persons with visual impairments. We conducted an acceptability study with 15 sighted and 5 visually impaired participants and report on the results.

Every day people rely on navigation systems when exploring unknown urban areas. Many navigation systems use multimodal feedback like visual, auditory or tactile cues. Although other systems exist, users mostly rely on a visual navigation using their smartphone. However, a problem with visual navigation systems is that the users have to shift their attention to the navigation system and then map the instructions to the real world. We suggest using in-situ navigation instructions that are presented directly in the environment by augmenting the reality using a projector-quadcopter. Through a user study with 16 participants, we show that using in-situ instructions for navigation leads to a significantly higher ability to observe real-world points of interest. Further, the participants enjoyed following the projected navigation cues.

We present VRHapticDrones, a system utilizing quadcopters as levitating haptic feedback proxy. A touchable surface is attached to the side of the quadcopters to provide unintrusive, flexible, and programmable haptic feedback in virtual reality. Since the users' sense of presence in virtual reality is a crucial factor for the overall user experience, our system simulates haptic feedback of virtual objects. Quadcopters are dynamically positioned to provide haptic feedback relative to the physical interaction space of the user. In a first user study, we demonstrate that haptic feedback provided by VRHapticDrones significantly increases users' sense of presence compared to vibrotactile controllers and interactions without additional haptic feedback. In a second user study, we explored the quality of induced feedback regarding the expected feeling of different objects. Results show that VRHapticDrones is best suited to simulate objects that are expected to feel either light-weight or have yielding surfaces. With VRHapticDrones we contribute a solution to provide unintrusive and flexible feedback as well as insights for future VR haptic feedback systems.

Recent advances in technology and miniaturization allow the building of self-levitating tangible interfaces. This includes flying tangibles, which extend the mid-air interaction space from 2D to 3D. While a number of theoretical concepts about interaction with levitating tangibles were previously investigated by various researchers, a user-centered evaluation of the presented interaction modalities has attracted only minor attention from prior research. We present Flyables, a system adjusting flying tangibles in 3D space to enable interaction between users and levitating tangibles. Interaction concepts were evaluated in a user study(N=17), using quadcopters as operable levitating tangibles. Three different interaction modalities are evaluated to collect quantitative data and qualitative feedback. Our findings show preferred user interaction modalities using Flyables. We conclude our work with a discussion and future research within the domain of human-drone interaction.

Manual assembly at production is a mentally demanding task. With rapid prototyping and smaller production lot sizes, this results in frequent changes of assembly instructions that have to be memorized by workers. Assistive systems compensate this increase in mental workload by providing "just-in-time" assembly instructions through in-situ projections. The implementation of such systems and their benefits to reducing mental workload have previously been justified with self-perceived ratings. However, there is no evidence by objective measures if mental workload is reduced by in-situ assistance. In our work, we showcase electroencephalography (EEG) as a complementary evaluation tool to assess cognitive workload placed by two different assistive systems in an assembly task, namely paper instructions and in-situ projections. We identified the individual EEG bandwidth that varied with changes in working memory load. We show, that changes in the EEG bandwidth are found between paper instructions and in-situ projections, indicating that they reduce working memory compared to paper instructions. Our work contributes by demonstrating how design claims of cognitive demand can be validated. Moreover, it directly evaluates the use of assistive systems for delivering context-aware information. We analyze the characteristics of EEG as real-time assessment for cognitive workload to provide insights regarding the mental demand placed by assistive systems.

Although a large number of navigation support systems for visually impaired people have been proposed in the past, navigating through unknown environments is still a major challenge for visually impaired travelers. Existing systems provide navigation information through headphones, speakers or tactile actuators. In this paper, we propose to use small lightweight quadcopters instead to provide navigation information for people with visual impairments. Using a leashed or free-floating quadcopter, the user is navigated by the distinct sound that the quadcopter emits and a haptic stimulus provided by the leash. In a user with 14 visually impaired participants, we compared leashed quadcopter navigation, free-floating quadcopter navigation, and traditional audio navigation. The results show that compared to audio navigation, participants navigate significantly faster with a free-floating quadcopter and make fewer navigation errors using the quadcopter navigation methods.

Even though, three-dimensional representations of architectural models exist, experiencing these models like one would experience a fully constructed building is still a major challenge. With Virtual Reality (VR) it is now possible to experience a number of scenarios in a virtual environment. Also prototyping interactive architecture elements, which might be very expensive, becomes possible. Thus, researchers and designers can already start to define user interfaces for interactive architectural elements, before they were even built. However, it is still an open question how exactly VR technologies can support experiencing interactive architecture. To answer this question, we compared experiencing three-dimensional architectural models and interactive architectural elements through a 2D screen & mouse+keyboard navigation, an head mounted display (HMD) & keyboard navigation, and an HMD & walking for navigation. The results of our study show challenges and opportunities regarding the immersion of these experiences.

More and more industrial manufacturing companies are outsourcing assembly tasks to sheltered work organizations where cognitively impaired workers are employed. To facilitate these assembly tasks assistive systems have been introduced to provide cognitive assistance. While previous work found that these assistive systems have a great impact on the workers' performance in giving assembly instructions, these systems are further capable of detecting errors and notifying the worker of an assembly error. However, the topic of how assembly errors are presented to cognitively impaired workers has not been analyzed scientifically. In this paper, we close this gap by comparing tactile, auditory, and visual error feedback in a user study with 16 cognitively impaired workers. The results reveal that visual error feedback leads to a significantly faster assembly time compared to tactile error feedback. Further, we discuss design implications for providing error feedback for workers with cognitive impairments.

With increasing complexity of assembly tasks and an increasing number of product variants, instruction systems providing cognitive support at the workplace are becoming more important. Different instruction systems for the workplace provide instructions on phones, tablets, and head-mounted displays (HMDs). Recently, many systems using in-situ projection for providing assembly instructions at the workplace have been proposed and became commercially available. Although comprehensive studies comparing HMD and tablet-based systems have been presented, in-situ projection has not been scientifically compared against state-of-the-art approaches yet. In this paper, we aim to close this gap by comparing HMD instructions, tablet instructions, and baseline paper instructions to in-situ projected instructions using an abstract Lego Duplo assembly task. Our results show that assembling parts is significantly faster using in-situ projection and locating positions is significantly slower using HMDs. Further, participants make less errors and have less perceived cognitive load using in-situ instructions compared to HMD instructions.

Exploratory Learning with Virtual Companions Sharing Attention and Context (EVA) is a concept for mediated teaching and learning that sits at the intersection of exploratory learning, telepresence, and attention awareness. The companion teacher is informed about the attentional state and environment of the learner, and can refer directly to this environment through marking or annotation. To the learner, the companion is virtual -- either human or automatic -- and, if human, either physically copresent or remote. The content and style of presentation are tailored to the learner's momentary level of interest or focus, and her attention can be guided to salient environmental elements (e.g. visual) in order to convey desired information. We define a design space for such systems, which applies to learning in Augmented Reality and Virtual Reality, and can be employed as a framework for design and evaluation. We demonstrate this through trials with two proof-of-concept systems, one in AR and one in VR, with a human companion. We conclude that the EVA design space defines a powerful set of systems for learning and finish by presenting guidelines for making such systems maximally effective.

Devices with touchscreens have an inherent latency. When a user's finger drags an object across the screen the object follows with a latency of around 100ms for current devices. Previous work showed that latencies down to 25ms reduce users' performance and that even 10ms latency is noticeable. In this paper we demonstrate an approach that reduces latency using a predictive model. Extrapolating the finger's movement we predict where the finger will be in the next moment. Comparing different prediction approaches we show for three different tasks that prediction using neural networks is more precise than linear and polynomial extrapolation. Furthermore, we show through a Fitts' Law dragging experiment that reducing touch latency can significantly increases users' performance. As the approach is software-based it can easily be integrated into existing mobile applications and systems.

Mobile phone-based Virtual Reality (VR) is rapidly growing as a platform for stereoscopic 3D and non-3D digital content and applications. The ability to track eye gaze in these devices would be a tremendous opportunity on two fronts: firstly, as an interaction technique, where interaction is currently awkward and limited, and secondly, for studying human visual behavior. We propose a method to add eye gaze tracking to these existing devices using their on-board display and camera hardware, with a minor modification to the headset enclosure. We present a proof-of-concept implementation of the technique and show results demonstrating its feasibility. The software we have developed will be made available as open source to benefit the research community.

Even after decades of research about navigation support for visually impaired people, moving independently still remains a major challenge. Previous work in HCI explored a large number of navigation aids, including auditory and tactile guidance systems. In this poster we propose a novel approach to guide visually impaired people. We use small lightweight drones that can be perceived through the distinct sound and the airflow they naturally produce. We describe the interaction concept we envision, first insights from proof-of-concept tests with a visually impaired participant, and provide an overview of potential application scenarios.

Today's working society tries to integrate more and more impaired workers into everyday working processes. One major scenario for integrating impaired workers is in the assembly of products. However, the tasks that are being assigned to cognitively impaired workers are easy tasks that consist of only a small number of assembly steps. For tasks with a higher number of steps, cognitively impaired workers need instructions to help them with assembly. Although supervisors provide general support and assist new workers while learning new assembly steps, sheltered work organizations often provide additional printed pictorial instructions that actively guide the workers. To further improve continuous instructions, we built a system that uses in-situ projection and a depth camera to provide context-sensitive instructions. To explore the effects of in-situ instructions, we compared them to state-of-the-art pictorial instructions in a user study with 15 cognitively impaired workers at a sheltered work organization. The results show that using in-situ instructions, cognitively impaired workers can assemble more complex products up to 3 times faster and with up to 50% less errors. Further, the workers liked the in-situ instructions provided by our assistive system and would use it for everyday assembly.

Order Picking is not only one of the most important but also most mentally demanding and error-prone tasks in the industry. Both stationary and wearable systems have been introduced to facilitate this task. Existing stationary systems are not scalable because of the high cost and wearable systems have issues being accepted by the workers. In this paper, we introduce a mobile camera-projector cart called OrderPickAR, which combines the benefits of both stationary and mobile systems to support order picking through Augmented Reality. Our system dynamically projects in-situ picking information into the storage system and automatically detects when a picking task is done. In a lab study, we compare our system to existing approaches, i.e, Pick-by-Paper, Pick-by-Voice, and Pick-by-Vision. The results show that using the proposed system, order picking is almost twice as fast as other approaches, the error rate is decreased up to 9 times, and mental demands are reduced up to 50%.

With the opportunity to customize ordered products, assembly tasks are becoming more and more complex. To meet these increased demands, a variety of interactive instruction systems have been introduced. Although these systems may have a big impact on overall efficiency and cost of the manufacturing process, it has been difficult to optimize them in a scientific way. The challenge is to introduce performance metrics that apply across different tasks and find a uniform experiment design. In this paper, we address this challenge by proposing a standardized experiment design for evaluating interactive instructions and making them comparable with each other. Further, we introduce a General Assembly Task Model, which differentiates between task-dependent and task-independent measures. Through a user study with 12 participants, we evaluate the experiment design and the proposed task model using an abstract pick-and-place task and an artificial industrial task. Finally, we provide paper-based instructions for the proposed task as a baseline for evaluating Augmented Reality instructions.