Haptic feedback, commonly experienced as vibrotactile cues on mobile devices, increases user performance and enhances user experience, but research addressing users with upper-body motor impairments remains scarce.In particular, variations in how touch input is performed across different motor abilities raise questions about the optimal location, on-device or on-body, for vibrotactile feedback to maximize its effectiveness.In this work, we apply design thinking to explore alternative approaches for delivering haptic feedback at locations distant from the on-screen touch point, such as the user's hand, wrist, forearm, or even the other arm.To inform our design explorations, we leverage empirical findings from a dataset of touch gestures performed on mobile devices by users with various upper-body motor impairments.We present future research opportunities at the intersection of haptic technology, wearable devices, accessible computing, and touchscreen input.

Vibrotactile feedback is known to have a positive impact on the user performance and experience of touchscreen input. Unfortunately, not all touchscreen devices have built-in haptics capabilities, public displays being a notable example, while the design options for vibrotactile feedback on conventional mobile devices, such as smartphones and tablets, are still limited. In this work, we propose leveraging commodity wearable devices, such as smartwatches, rings, and armbands, to enable vibrotactile feedback as closely as possible to the touch point when the touchscreen does not natively support haptics, and to complement existing haptics capabilities when the touchscreen already features them. To this end, we introduce Wear + Touch, a set of four interaction techniques designed to augment touchscreen input with vibrations delivered through wearable devices, from substituting to complementing, extending, and expanding on-screen vibrotactile feedback. We demonstrate Wear + Touch with several practical implementations involving a large touchscreen display, smartphone, tablet, smartwatch, smart ring, and armband as well as a custom wearable device that we designed for vibrations delivered simultaneously on the user’s index finger, wrist, and forearm. We also report empirical results from two controlled experiments involving a total of 40 participants, conducted to understand the multi-faceted user experience of vibrotactile feedback delivered through the Wear + Touch techniques. In the first experiment, we reveal a favorable user experience of vibrations delivered on the index finger, wrist, and forearm, in this order, when the touchscreen lacks haptics capabilities. In the second experiment, we reveal benefits of reinforcing and expanding on-screen vibrations with vibrotactile feedback on the index finger, wrist, and arm, respectively. We use our findings to suggest future work opportunities for combined vibrotactile feedback involving both touchscreens and wearable devices.

The evolving landscape of distributed user interfaces requires the prototyping stage also be distributed between users, tasks, platforms, and environments. To create a cohesive distribution of the user interface elements in such ecosystems, paired sketching has emerged as a collaborative design method that leverages multiple stakeholders’ strengths, including designers, developers, and end users, working in pairs. In the context of developer experience applied to paired sketching for distributed user interfaces, we decomposed a workflow into four disciplines according to the Software and Systems Process Engineering Meta-Model (SPEM) notation. First, we defined a protocol to deploy paired sketching of distributed user interfaces, supported by UbiSketch , a collaborative software environment tailored featuring sketch recognition and whiteboarding. Second, to evaluate paired sketching for engineering interactive systems, we conducted an experiment involving five pairs of stakeholders who sketched a distributed user interface for inside-the-vehicule interaction distributed on four platforms: smartphone, tablet, pen display, and tabletop. Empirical results from questionnaires, reactivity, intention, perceived satisfaction, and free comments, suggest a preference order in which the tabletop is ranked first, followed by the tablet, smartphone, and pen display. Based on these results, we discuss the potential of paired sketching for distributed user interfaces.

We introduce ChairMX, on-chair input technology for novel interactive media consumption experiences through the chair. ChairMX is the end result of an elaborate four-stage research and development process involving a systematic literature review of the extent in which chairs have been used in interactive systems, involvement of potential end users to elicit their preferences for on-chair input, engineering on-chair gesture input recognition technology with wearables, and exploration of application opportunities. By leveraging a ring-like form factor wearable, ChairMX is a readily deployable technology, working for uninstrumented chairs everywhere and enabling a variety of use case scenarios involving interactive media consumption, from the living room to public environments.

We introduce VIREO, a web-based software tool for graphical authoring of vibrotactile feedback for mobile and wearable applications. VIREO enables flexible specification of vibrotactile patterns with model-based and free-draw input, and is compatible with devices that run JavaScript, either natively or in a web browser. We demonstrate VIREO with applications developed for smartphones, smartwatches, armbands, and smartglasses, and we present the results of a usability evaluation study with sixteen participants represented by coders with various programming experience. We discuss our contributions in the context of the results of a Systematic Literature Review conducted on the topic of software tools, editors, and platforms developed in the scientific community for authoring vibrotactile feedback. Given that one finding of our review is the little availability of such contributions, we release VIREO as a free resource on the web for researchers and practitioners to author and integrate vibrotactile feedback in mobile and wearable applications.