When communicating over long distances via telerobotic platforms, the ability to perform semantic haptic interactions is often lacking, stifling feelings of presence for those involved. One such interaction is handshaking—a haptic gesture used in various social contexts. This paper presents a within-subjects user study ( \(N=18\) ), where participants felt 14 unimodal actuated profiles and 1 no haptic (control) profile. The actuation of these profiles was facilitated by the Handshaking Over Long Distances (HOLD) system which uses radial squeeze, skin-stretch via a rocker, and vibration to simulate a handshake in the context of a telerobotic exchange. Participants’ impressions of the profiles were assessed via questionnaires. The findings from this study suggest that the type of haptic feedback (lateral skin stretch, radial squeeze, or vibration) has a larger effect on user preference and presence compared to the differences in profiles within each feedback type. All three feedback types provided a more preferable handshaking experience for the pilot user and increased presence compared to the control. However, radial squeeze was the most effective. This work highlights the importance of haptics, especially radial squeeze, in the remote handshaking experience and exhibits wearable haptic devices’ potential to improve the effectiveness of remote communication tools.

Touchscreen-based rendering of graphics using vibrations, sonification, and text-to-speech is a promising approach for nonvisual access to graphical information, but extracting trends from complex data representations nonvisually is challenging. This work presents the design of a multimodal feedback scheme with integrated spatial audio for the exploration of histograms and scatter plots on touchscreens. We detail the hardware employed and the algorithms used to control vibrations and sonification adjustments through the change of pitch and directional stereo output. We conducted formative testing with 5 blind or visually impaired participants, and results illustrate that spatial audio has the potential to increase the identification of trends in the data, at the expense of a skewed mental representation of the graph. This design work and pilot study are critical to the iterative, human-centered approach of rendering multimodal graphics on touchscreens and contribute a new scheme for efficiently capturing data trends in complex data representations.

Vibration plays a significant role in the way users interact with touchscreens. For many users, vibration affords tactile alerts and other enhancements. For eyes-free users and users with visual impairments, vibration can also serve a more primary role in the user interface, such as indicating streets on maps, conveying information about graphs, or even specifying basic graphics. However, vibration is rarely used in current user interfaces beyond basic cuing. Furthermore, designers and developers who do actually use vibration more extensively are often unable to determine the exact properties of the vibration signals they are implementing, due to out-of-the-box software and hardware limitations. We make two contributions in this work. First, we investigate the contextual properties of touchscreen vibrations and how vibrations can be used to effectively convey traditional, embossed elements, such as dashes and dots. To do so, we developed an open source, Android-based library to generate vibrations that are perceptually salient and intuitive, improving upon existing vibration libraries. Second, we conducted a user study with 26 blind or visually impaired users to evaluate and categorize the effects with respect to traditional tactile line profiles. We have established a range of vibration effects that can be reliably generated by our haptic library and are perceptible and distinguishable by users.

While text-to-speech software has largely made textual information accessible in the digital space, analogous access to graphics still remains an unsolved problem. Because of their portability and ubiquity, several studies have alluded to touchscreens as a potential platform for such access, yet there is still a gap in our understanding of multimodal information transfer in the context of graphics. The current research demonstrates feasibility for following lines, a fundamental graphical concept, via vibrations and sounds on commercial touchscreens. Two studies were run with 21 blind and visually impaired participants ( N = 12; N = 9). The first study examined the presentation of straight, linear lines using a multitude of line representations, such as vibration-only, auditory-only, vibration lines with auditory borders, and auditory lines with vibration borders. The results of this study demonstrated that both auditory and vibratory bordered lines were optimal for precise tracing, although both vibration- and auditory-only lines were also sufficient for following, with minimal deviations. The second study examined the presentation of curving, non-linear lines. Conditions differed on the number of auditory reference points presented at the inflection and deflection points. Participants showed minimal deviation from the lines during tracing, performing nearly equally in both 1- and 3-point conditions. From these studies, we demonstrate that line following via multimodal feedback is possible on touchscreens, and we present guidelines for the presentation of such non-visual graphical concepts.