Cyclists need interfaces such as on-vehicle displays or augmented-reality (AR) glasses for effective communication with autonomous vehicles (AVs) when human drivers are no longer present. Interfaces must handle complex situations involving multiple AVs around a cyclist. Holistic AV-Cyclist Interfaces (HACIs) are a novel solution; they group interfaces into a multimodal interconnected system to support the rider. However, the best way to present information is uncertain. We explored this in a scenario with three AVs using CycleARcade, a new multi-user AR platform for designing and evaluating HACIs. Cyclists and HCI researchers collaboratively created and tested HACIs within CycleARcade through a novel iterative participatory design method. We synthesised three HACIs from this process and assessed them with riders in CycleARcade. Participants preferred HACIs with AR displays integrated into the environment to avoid road distractions, paired with spatial audio communicating AV proximity. These findings provide crucial input for the real-world deployment of AVs.

Touchless displays often use mid-air gestures to control on-screen cursors for pointer interactions. Area cursors can simplify touchless cursor input by implicitly targeting nearby widgets without the cursor entering the target. However, for displays with dense target layouts, the cursor still has to arrive close to the widget, meaning the benefits of area cursors for time-to-target and effort are diminished. Through two experiments, we demonstrate for the first time that fine-tuning the mapping between hand and cursor movements (control-display gain – CDG) can address the deficiencies of area cursors and improve the performance of touchless interaction. Across several display sizes and target densities (representative of myriad public displays used in retail, transport, museums, etc), our findings show that the forgiving nature of an area cursor compensates for the imprecision of a high CDG, helping users interact more effectively with smaller and more controlled hand/arm movements.

Extended Reality (XR) interactions often rely on spatial hand or controller inputs - necessitating dexterous wrist, hand and finger movements including pressing virtual buttons, pinching to select, and performing hand gestures. However, there are scenarios where such dependencies may render XR devices and apps inaccessible to users - from situational/temporary impairments such as encumbrance, to physical motor impairments. In this paper, we contribute to a growing literature considering facial input as an alternative. In a user study (N=20) we systematically evaluate the usability of 53 Facial Action Units in VR, deriving a set of optimal (comfort, effort, performance) FAUs for interaction. We then use these facial inputs to drive and evaluate (N=10) two demonstrator apps: VR locomotion, and AR web browsing, showcasing how close facial interaction can get to existing baselines, and demonstrating that FAUs offer a viable, generalizable input modality for XR devices.

The social cues drivers exchange with cyclists to negotiate space-sharing will disappear as autonomous vehicles (AVs) join our roads, leading to safety concerns. External Human-Machine Interfaces (eHMIs) on vehicles can replace driver social signals, but how these should be designed to communicate with cyclists is unknown. We evaluated three eHMIs across multiple traffic scenarios in two stages. First, we compared eHMI versatility, acceptability and usability in a VR cycling simulator. Cyclists preferred colour-coded signals communicating AV intent, easily seen through quick glances. Second, we refined the interfaces based on our findings and compared them outdoors. Participants cycled around a moving car with real eHMIs. They preferred eHMIs using large surfaces on the vehicle and animations reinforcing colour changes. We conclude with novel design guidelines for versatile eHMIs based on first-hand interaction feedback. Our findings establish the factors that enable AVs to operate safely around cyclists across different traffic scenarios.

Social anxiety is a prevalent mental health concern that impacts quality of life and makes social spaces less accessible. We conducted two studies with socially anxious participants, investigating using affective haptic comfort objects to provide calming support during social exposure. Participatory prototyping informed the design and use of the intervention, which was then evaluated between-groups with a social exposure task. Treatment participants held their preferred vibration-augmented prototype during this task; control participants did not. We observed no change in physiological measures, but treatment participants exhibited a significantly broader distribution of psychological anxiety scores. Participants in both studies found their objects pleasant and calming, made positive emotional associations with resonant stimuli, and used their objects to afford self-soothing tactile experiences. We discuss how future designers can facilitate calming affective haptic interfaces for socially anxious settings.

This paper introduces design patterns for dashboards to inform dashboard design processes. Despite a growing number of public examples, case studies, and general guidelines there is surprisingly little design guidance for dashboards. Such guidance is necessary to inspire designs and discuss tradeoffs in, e.g., screenspace, interaction, or information shown. Based on a systematic review of 144 dashboards, we report on eight groups of design patterns that provide common solutions in dashboard design. We discuss combinations of these patterns in "dashboard genres" such as narrative, analytical, or embedded dashboard. We ran a 2-week dashboard design workshop with 23 participants of varying expertise working on their own data and dashboards. We discuss the application of patterns for the dashboard design processes, as well as general design tradeoffs and common challenges. Our work complements previous surveys and aims to support dashboard designers and researchers in co-creation, structured design decisions, as well as future user evaluations about dashboard design guidelines. Detailed pattern descriptions and workshop material can be found online: https://dashboarddesignpatterns.github.io.

Touchless input could transform clinical activity by allowing health professionals direct control over medical imaging systems in a sterile manner. Currently, users face the issues of being unable to directly manipulate imaging in aseptic environments, as well as needing to touch shared surfaces in other hospital areas. Unintended input is a key challenge for touchless interaction and could be especially disruptive in medical contexts. We evaluated four clutching techniques with 34 health professionals, measuring interaction performance and interviewing them to obtain insight into their views on clutching, and touchless control of medical imaging. As well as exploring the performance of the different clutching techniques, our analysis revealed an appetite for reliable touchless interfaces, a strong desire to reduce shared surface contact, and suggested potential improvements such as combined authentication and touchless control. Our findings can inform the development of novel touchless medical systems and identify challenges for future research.

We present the design and development of a data visualization service (RAMPVIS) in response to the urgent need to support epidemiological modeling workflows during the COVID-19 pandemic. Facing a set of demanding requirements and several practical challenges, our small team of volunteers had to rely on existing knowledge and components of services computing, while thinking on our feet in configuring services composition and adopting suitable approaches to services engineering. Through developing the RAMPVIS service, we have gained useful experience of ensuring conformation to services computing standards, enabling rapid development and early deployment, and facilitating effective and efficient maintenance and operation with limited resources. This experience can be valuable to the ongoing effort for combating the COVID-19 pandemic, and provides a blueprint for visualization service development when future needs for visual analytics arise during emergency response.

In the process of developing an infrastructure for providing visualization and visual analytics (VIS) tools to epidemiologists and modeling scientists, we encountered a technical challenge for applying a number of visual designs to numerous datasets rapidly and reliably with limited development resources. In this paper, we present a technical solution to address this challenge. Operationally, we separate the tasks of data management, visual designs, and plots and dashboard deployment in order to streamline the development workflow. Technically, we utilize: an ontology to bring datasets, visual designs, and deployable plots and dashboards under the same management framework; multi-criteria search and ranking algorithms for discovering potential datasets that match a visual design; and a purposely-design user interface for propagating each visual design to appropriate datasets (often in tens and hundreds) and quality-assuring the propagation before the deployment. This technical solution has been used in the development of the RAMPVIS infrastructure for supporting a consortium of epidemiologists and modeling scientists through visualization.

Smartphones provide access to increasing amounts of personal and sensitive information, yet are often only secured using methods that are prone to observational attacks. We present HapticLock, a novel authentication method for mobile devices that uses non-visual interaction modalities for discreet PIN entry that is difficult to attack by shoulder surfing. A usability experiment (N=20) finds effective PIN entry in secure conditions: e.g., in 23.5s with 98.3% success rate for a four-digit PIN entered from a random start digit. A shoulder surfing experiment (N=15) finds that HapticLock is highly resistant to observational attacks. Even when interaction is highly visible, attackers need to guess the first digit when PIN entry begins with a random number, yielding a very low success rate for shoulder surfing. Furthermore, a device can be hidden from view during authentication. Our use of haptic interaction modalities gives privacy-conscious mobile device users a usable and secure authentication alternative for sensitive situations.

When users are undertaking mentally demanding visuals tasks, it can be beneficial to convey information through the auditory or tactile modality instead. A fundamental problem when mapping information to sound or vibration is establishing which polarity the mapping should use. Magnitude estimation is a popular method of establishing polarity preferences, however the effectiveness of this approach remains unclear, especially in more ecologically valid contexts. We investigate what impact the polarity of a data-sound or data-vibration mapping has on how well users can interpret these mappings, under two different levels of mental workload. Our results show that polarity does not affect error rate or cognitive workload, although may affect response time. We also found that induced cognitive load may influence usability. An implication of this is that commonly used methods of establishing data mappings need to be revisited, with cognitive load in mind, to help designers create more usable auditory and vibrotactile displays.

Ultrasound haptic devices can create parametric audio as well as contactless haptic feedback. We investigate if multimodal output from these devices can influence the perception of haptic feedback. We used a magnitude estimation experiment to evaluate perceived roughness of an ultrasound haptic pattern. We found that white noise audio from the haptics device increased perceived roughness and pure tones did not, and that lower rendering frequencies may increase perceived roughness. Our results show that multimodal output has the potential to expand the range of sensations that can be presented by an ultrasound haptic device, paving the way to richer mid-air haptic interfaces.

Ultrasound haptic patterns can be rendered by continuously moving an ultrasonic focal point. It is not known how this focal point motion affects haptic perception. We present two psychophysical experiments investigating the perception of an ultrasound haptic focal point moving along a circular path. Our first experiment finds that a sensation of motion is perceived at speeds up to 17 revs–1, similar to the ‘flutter’ sensation. Our second experiment found a linear relationship between movement speed and perceived intensity up to this speed. Our findings give new insight into ultrasound haptic perception and give designers empirical evidence to inform contactless haptic feedback design.

This paper presents a survey informing a user-first approach to designing calming affective haptic stimuli by eliciting user preferences in different social scenarios. Prior affective haptics research presented users with stimuli and recorded emotional responses. By contrast this work focuses on the sensations users wish to experience and how these can be simulated using haptics. The survey (n=81) investigated which users preferences in four social situations to reduce social anxiety. Using thematic analysis of responses we created a coding scheme of stimuli derived from real-world experiences to emulate with affective haptics. By cross-referencing these categories with affective haptics research, we provide recommendations to designers about which calming stimuli users wish to experience socially and how they can be implemented.

Wireless power transfer creates new opportunities for interaction with tangible and wearable devices, by freeing designers from the constraints of an integrated power source. We explore the use of focused ultrasound as a means of transferring power to a distal device, transforming passive props into dynamic active objects. We analyse the ability to transfer power from an ultrasound array commonly used for mid-air haptic feedback and investigate the practical challenges of ultrasonic power transfer (e.g., receiving and rectifying energy from sound waves). We also explore the ability to power electronic components and multimodal actuators such as lights, speakers and motors. Finally, we describe exemplar wearable and tangible device prototypes that are activated by UltraPower, illustrating the potential applications of this novel technology.

Current solutions for creating co-located Mixed Reality (MR) experiences typically rely on platform-specific synchronisation of spatial anchors or Simultaneous Localisation and Mapping (SLAM) data across clients, often coupled to cloud services. This introduces significant costs (in development and deployment), constraints (with interoperability across platforms often limited), and privacy concerns. For practitioners, support is needed for creating platform-agnostic co-located MR experiences. This paper explores the utility of aligned SLAM solutions by 1) surveying approaches toward aligning disparate device coordinate spaces, formalizing their theoretical accuracy and limitations; 2) providing skeleton implementations for audience-based, small-scale and large-scale co-location using said alignment approaches; and 3) detailing how we can assess the accuracy and safety of 6DoF/SLAM tracking solutions for any arbitrary device and dynamic environment without the need for an expensive ground truth optical tracking, by using trilateration and a $30 laser distance meter. Through this, we hope to further democratise the creation of cross-platform co-located MR experiences.

Mixed Reality (MR), Augmented Reality (AR) and Virtual Reality (VR) headsets can improve upon existing physical multi-display environments by rendering large, ergonomic virtual display spaces whenever and wherever they are needed. However, given the physical and ergonomic limitations of neck movement, users may need assistance to view these display spaces comfortably. Through two studies, we developed new ways of minimising the physical effort and discomfort of viewing such display spaces. We first explored how the mapping between gaze angle and display position could be manipulated, helping users view wider display spaces than currently possible within an acceptable and comfortable range of neck movement. We then compared our implicit control of display position based on head orientation against explicit user control, finding significant benefits in terms of user preference, workload and comfort for implicit control. Our novel techniques create new opportunities for productive work by leveraging MR headsets to create interactive wide virtual workspaces with improved comfort and usability. These workspaces are flexible and can be used on-the-go, e.g., to improve remote working or make better use of commuter journeys.

Acoustic levitation enables a radical new type of human-computer interface composed of small levitating objects. For the first time, we investigate the selection of such objects, an important part of interaction with a levitating object display. We present Point-and-Shake, a mid-air pointing interaction for selecting levitating objects, with feedback given through object movement. We describe the implementation of this technique and present two user studies that evaluate it. The first study found that users could accurately (96%) and quickly (4.1s) select objects by pointing at them. The second study found that users were able to accurately (95%) and quickly (3s) select occluded objects. These results show that Point-and-Shake is an effective way of initiating interaction with levitating object displays.

Young children with visual impairments tend to engage less with their surroundings, limiting the benefits from activities at school. We investigated novel ways of using sound from a bracelet, such as speech or familiar noises, to tell children about nearby people, places and activities, to encourage them to engage more during play and help them move independently. We present a series of studies, the first two involving visual impairment educators, that give insight into challenges faced by visually impaired children at school and how sound might help them. We then present a focus group with visually impaired children that gives further insight into the effective use of sound. Our findings reveal novel ways of combining sounds from wearables with sounds from the environment, motivating audible beacons, devices for audio output and proximity estimation. We present scenarios, findings and a design space that show the novel ways such devices could be used alongside wearables to help visually impaired children at school.

We demonstrate a technique for rendering textured haptic surfaces in mid-air, using an ultrasound haptic display. Our technique renders tessellated 3D `haptic' shapes with different waveform properties, creating surfaces with distinct perceptions.

We present rhythmic micro-gestures, micro-movements of the hand that are repeated in time with a rhythm. We present a user study that investigated how well users can perform rhythmic micro-gestures and if they can use them eyes-free with non-visual feedback. We found that users could successfully use our interaction technique (97% success rate across all gestures) with short interaction times, rating them as low difficulty as well. Simple audio cues that only convey the rhythm outperformed animations showing the hand movements, supporting rhythmic micro-gestures as an eyes-free input technique.

Acoustic levitation enables new types of human-computer interface, where the content that users interact with is made up from small objects held in mid-air. We show that acoustically-levitated objects can form mid-air widgets that respond to interaction. Users can interact with them using in-air hand gestures. Sound and widget movement are used as feedback about the interaction.

When users want to interact with an in-air gesture system, they must first address it. This involves finding where to gesture so that their actions can be sensed, and how to direct their input towards that system so that they do not also affect others or cause unwanted effects. This is an important problem which lacks a practical solution. We present an interaction technique which uses multimodal feedback to help users address in-air gesture systems. The feedback tells them how ("do that") and where ("there") to gesture, using light, audio and tactile displays. By doing that there, users can direct their input to the system they wish to interact with, in a place where their gestures can be sensed. We discuss the design of our technique and three experiments investigating its use, finding that users can "do that" well (93.2%-99.9%) while accurately (51mm-80mm) and quickly (3.7s) finding "there".

Visually impaired children often have difficulty with everyday activities like locating items, e.g. favourite toys, and moving safely around the home. It is important to assist them during activities like these because it can promote independence from adults and helps to develop skills. Our demonstration shows our work towards a multimodal sensing and output system that adapts the lighting conditions at home to help visually impaired children with such tasks.

In this paper we describe a demonstration of our multimodal affective feedback designs, used in research to expand the emotional expressivity of interfaces. The feedback leverages inherent associations and reactions to thermal, vibrotactile, auditory and abstract visual designs to convey a range of affective states without any need for learning feedback encoding. All combinations of the different feedback channels can be utilised, depending on which combination best conveys a given state. All the signals are generated from a mobile phone augmented with thermal and vibrotactile stimulators, which will be available to conference visitors to see, touch, hear and, importantly, feel.

Above-device gesture interfaces let people interact in the space above mobile devices using hand and finger movements. For example, users could gesture over a mobile phone or wearable without having to use the touchscreen. We look at how above-device interfaces can also give feedback in the space over the device. Recent haptic and wearable technologies give new ways to provide tactile feedback while gesturing, letting touchless gesture interfaces give touch feedback. In this paper we take a first detailed look at how tactile feedback can be given during above-device interaction. We compare approaches for giving feedback (ultrasound haptics, wearables and direct feedback) and also look at feedback design. Our findings show that tactile feedback can enhance above-device gesture interfaces.

Around-device interaction lets people use their phones without having to pick them up or reach out and touch them. This allows interaction when touch may not be available; for example, users could gesture to interact with their phones while cooking, avoiding touching the screen with messy hands. Well-designed feedback helps users overcome uncertainty during gesture interaction, however giving effective feedback from small devices can be a challenge and detailed visual feedback will not always be suitable. My thesis research looks at how other types of feedback may be used effectively during around-device interaction to help users.

Gestures above a mobile phone would let users interact with their devices quickly and easily from a distance. While both researchers and smartphone manufacturers develop new gesture sensing technologies, little is known about how best to design these gestures and interaction techniques. Our research looks at creating usable and socially acceptable above-device interaction techniques. We present an initial gesture collection, a preliminary evaluation of these gestures and some design recommendations. Our findings identify interesting areas for future research and will help designers create better gesture interfaces.