The sound produced when touching fabrics, like a blanket, often provides information regarding the fabric’s texture properties (e.g., its roughness). Fabric roughness is one of the most important aspects of assessing fabric tactile properties. Prior research has demonstrated that touch-related sounds can alter the perception of textures. However, understanding touch-related sound of digital fabric textures, and how they could convey affective responses remain a challenge. In this study, we mapped digital fabric textures using mid-air haptics stimuli and examined how auditory manipulation influences people’s roughness perception. Through qualitative interviews, participants detailed that while rubbing sounds smoothen fabric texture perception, pure tone sounds of 450Hz and 900Hz accent roughness perception. The rubbing sound of fabric evoked associations with soft-materials and led to more calming experiences. In addition, we discussed how haptic interaction can be extended to multisensory modes, revealing a new perspective of mapping multisensory experiences for digital fabrics.

Human perception of touch is a complex interplay of sensory inputs, extending beyond tactile stimuli to include visual and auditory cues. Yet, the potential of olfactory cues (e.g., scents) to enrich our understanding and experiences of digital touch remains largely unexplored. This paper explores the integration of scent with digital haptic feedback, specifically focusing on the perception of textiles using an ultrasonic mid-air haptic device. In an initial validation study, we identified scents associated with the tactile properties of roughness, smoothness, coldness, and warmth based on participant ratings (n = 31). In a subsequent immersive Virtual Reality (VR) experiment, we explored how these scents might affect perceptions of digital textiles presented using ultrasonic mid-air haptics (n = 32). Our results suggested that while scents did not affect the tactile properties of digital textiles, participants reported the sensations feeling more realistic and more arousing according to the scent delivered. This multimodal integration of scent with digital haptic stimuli can open up the creation of more immersive experiences and more in-depth understanding of digital material properties.

Phased arrays of transducers have been quickly evolving in terms of software and hardware with applications in haptics (acoustic vibrations), display (levitation), and audio. Most recently, Multimodal Particle-based Displays (MPDs) have even demonstrated volumetric content that can be seen, heard, and felt simultaneously, without additional instrumentation. However, current software tools only support individual modalities and they do not address the integration and exploitation of the multi-modal potential of MPDs. This is because there is no standardized presentation pipeline tackling the challenges related to presenting such kind of multi-modal content (e.g., multi-modal support, multi-rate synchronization at 10 KHz, visual rendering or synchronization and continuity). This article presents OpenMPD, a low-level presentation engine that deals with these challenges and allows structured exploitation of any type of MPD content (i.e., visual, tactile, audio). We characterize OpenMPD’s performance and illustrate how it can be integrated into higher-level development tools (i.e., Unity game engine). We then illustrate its ability to enable novel presentation capabilities, such as support of multiple MPD contents, dexterous manipulations of fast-moving particles, or novel swept-volume MPD content.

Acoustic levitation has emerged as a promising approach for mid-air displays, by using multiple levitated particles as 3D voxels, cloth and thread props, or high-speed tracer particles, under the promise of creating 3D displays that users can see, hear and feel with their bare eyes, ears and hands. However, interaction with this mid-air content always occurred at a distance, since external objects in the display volume (e.g. user’s hands) can disturb the acoustic fields and make the particles fall. This paper proposes TipTrap, a co-located direct manipulation technique for acoustically levitated particles. TipTrap leverages the reflection of ultrasound on the users’ skin and employs a closed-loop system to create functional acoustic traps 2.1 mm below the fingertips, and addresses its 3 basic stages: selection, manipulation and deselection. We use Finite-Differences Time Domain (FDTD) simulations to explain the principles enabling TipTrap, and explore how finger reflections and user strategies influence the quality of the traps (e.g. approaching direction, orientation and tracking errors), and use these results to design our technique. We then implement the technique, characterizing its performance with a robotic hand setup and finish with an exploration of the ability of TipTrap to manipulate different types of levitated content.

3D selection in dense VR environments (e.g., point clouds) is extremely challenging due to occlusion and imprecise mid-air input modalities (e.g., 3D controllers and hand gestures). In this paper, we propose "Slicing-Volume", a hybrid selection technique that enables simultaneous 3D interaction in mid-air, and a 2D pen-and-tablet metaphor in VR. Inspired by well-known slicing plane techniques in data visualization, our technique consists of a 3D volume that encloses target objects in mid-air, which are then projected to a 2D tablet view for precise selection on a tangible physical surface. While slicing techniques and tablets-in-VR have been previously explored, in this paper, we evaluated the potential of this hybrid approach to improve accuracy in highly occluded selection tasks, comparing different multimodal interactions (e.g., Mid-air, Virtual Tablet and Real Tablet). Our results showed that our hybrid technique significantly improved overall accuracy of selection compared to Mid-air selection only, thanks to the added haptic feedback given by the physical tablet surface, rather than the added visualization given by the tablet view.

Phased Arrays of Transducers (PATs) allow accurate control of ultrasound fields, with applications in haptics, levitation (i.e. displays) and parametric audio. However, algorithms for multi-point levitation or tactile feedback are usually limited to computing solutions in the order of hundreds of sound-fields per second, preventing the use of multiple high-speed points, a feature that can broaden the scope of applications of PATs. We present GS-PAT , a GPU multi-point phase retrieval algorithm, capable of computing 17K solutions per second for up to 32 simultaneous points in a mid-end consumer grade GPU (NVidia GTX 1660). We describe the algorithm and compare it to state of the art multi-point algorithms used for ultrasound haptics and levitation, showing similar quality of the generated sound-fields, and much higher computation rates. We then illustrate how the shift in paradigm enabled by GS-PAT (i.e. real-time control of several high-speed points) opens new applications for PAT technologies, such as in volumetric fully coloured displays, multi-point spatio-temporal tactile feedback, parametric audio and simultaneous combinations of these modalities.

Scale adaptive techniques for VR navigation enable users to navigate spaces larger than the real space available, while allowing precise interaction when required. However, due to these techniques gradually scaling displacements as the user moves (changing user's speed), they introduce a Drift effect. That is, a user returning to the same point in VR will not return to the same point in the real space. This mismatch between the real/virtual spaces can grow over time, and turn the techniques unusable (i.e., users cannot reach their target locations). In this paper, we characterise and analyse the effects of Drift, highlighting its potential detrimental effects. We then propose two techniques to correct Drift effects and use a data driven approach (using navigation data from real users with a specific scale adaptive technique) to tune them, compare their performance and chose an optimum correction technique and configuration. Our user study, applying our technique in a different environment and with two different scale adaptive navigation techniques, shows that our correction technique can significantly reduce Drift effects and extend the life-span of the navigation techniques (i.e., time that they can be used before Drift draws targets unreachable), while not hindering users' experience.

Virtual Reality allow users to explore virtual environments naturally, by moving their head and body. However, the size of the environments they can explore is limited by real world constraints, such as the tracking technology or the physical space available. Existing techniques removing these limitations often break the metaphor of natural navigation in VR (e.g. steering techniques), involve control commands (e.g., teleporting) or hinder precise navigation (e.g., scaling user's displacements). This paper proposes NaviFields, which quantify the requirements for precise navigation of each point of the environment, allowing natural navigation within relevant areas, while scaling users' displacements when travelling across non-relevant spaces. This expands the size of the navigable space, retains the natural navigation metaphor and still allows for areas with precise control of the virtual head. We present a formal description of our NaviFields technique, which we compared against two alternative solutions (i.e., homogeneous scaling and natural navigation). Our results demonstrate our ability to cover larger spaces, introduce minimal disruption when travelling across bigger distances and improve very significantly the precise control of the viewpoint inside relevant areas.

Interaction in VR involves large body movements, easily inducing fatigue and discomfort. We propose Erg-O, a manipulation technique that leverages visual dominance to maintain the visual location of the elements in VR, while making them accessible from more comfortable locations. Our solution works in an open-ended fashion (no prior knowledge of the object the user wants to touch), can be used with multiple objects, and still allows interaction with any other point within user's reach. We use optimization approaches to compute the best physical location to interact with each visual element, and space partitioning techniques to distort the visual and physical spaces based on those mappings and allow multi-object retargeting. In this paper we describe the Erg-O technique, propose two retargeting strategies and report the results from a user study on 3D selection under different conditions, elaborating on their potential and application to specific usage scenarios.