We present empirical results from a study conducted with eleven users with upper-body motor impairments who imagined input devices and corresponding gestures to operate them for performing common tasks in interactive systems. We report a strong preference for embodying devices (80%), primarily through the hands, rather than holding them, and identify ten device archetypes, among which smartphones (36.4%) and remote controls (27.3%) were most prevalent. We also observed a diversity of gestures to operate imagined devices involving both unimanual and bimanual input with little consensus (.069) across participants, which we analyzed in relation to their self-reported motor impairments. Based on these findings, we propose design recommendations for accessible interactions involving imagined input devices, structured through the lenses of ability-based and ability-mediating design. We also outline future work opportunities for imagination-powered accessible computing, in which users’ imagination plays a central role.

Haptic feedback, commonly experienced as vibrotactile cues on mobile devices, increases user performance and enhances user experience, but research addressing users with upper-body motor impairments remains scarce.In particular, variations in how touch input is performed across different motor abilities raise questions about the optimal location, on-device or on-body, for vibrotactile feedback to maximize its effectiveness.In this work, we apply design thinking to explore alternative approaches for delivering haptic feedback at locations distant from the on-screen touch point, such as the user's hand, wrist, forearm, or even the other arm.To inform our design explorations, we leverage empirical findings from a dataset of touch gestures performed on mobile devices by users with various upper-body motor impairments.We present future research opportunities at the intersection of haptic technology, wearable devices, accessible computing, and touchscreen input.

We present empirical results from a gesture elicitation study conducted with eleven wheelchair users that proposed on-body, in-air, and on-wheelchair gestures to effect twenty-one referents representing common actions, types of digital content, and navigation commands for interactive systems. We report a large preference for on-body (47.6%) and in-air (40.7%) compared to on-wheelchair (11.7%) gestures, mostly represented by touch input on different parts of the body and hand poses performed in mid-air with one hand. Following an agreement analysis that revealed low consensus (≤ 5.5%) between users, although high perceived gesture ease, goodness, and social acceptability within users, we examine our participants’ gesture characteristics in relation to their self-reported motor impairments, e.g., low strength, rapid fatigue, etc. We highlight the need for personalized gesture sets, tailored to and reflective of both users’ preferences and specific motor abilities, an implication that we examine through the lenses of ability-based design.

We examine touchscreen stroke-gestures and mid-air motion-gestures articulated by users with upper-body motor impairments with devices worn on the wrist, finger, and head. We analyze users’ gesture input performance in terms of production time, articulation consistency, and kinematic measures, and contrast the performance of users with upper-body motor impairments with that of a control group of users without impairments. Our results, from two datasets of 7,290 stroke-gestures and 3,809 motion-gestures collected from 28 participants, reveal that users with upper-body motor impairments take twice as much time to produce stroke-gestures on wearable touchscreens compared to users without impairments, but articulate motion-gestures equally fast and with similar acceleration. We interpret our findings in the context of ability-based design and propose ten implications for accessible gesture input with upper-body wearables for users with upper-body motor impairments.

We examine accessible interactions for wheelchair users and public displays with three studies. In a first study, we conduct a Systematic Literature Review, from which we report very few scientific papers on this topic and a preponderant focus on touch input. In a second study, we conduct a Systematic Video Review using YouTube as a data source, and unveil accessibility challenges for public displays and several input modalities alternative to direct touch. In a third study, we conduct semi-structured interviews with eleven wheelchair users to understand their experience interacting with public displays and to collect their preferences for more accessible input modalities. Based on our findings, we propose the “assisted interaction” phase to extend Vogel and Balakrishnan’s four-phase interaction model with public displays, and the “ability” dimension for cross-device interaction design to support, via users’ personal mobile devices, independent use of interactive public displays.

We conduct an examination of the accessibility challenges experienced by people with upper body motor impairments when interacting with television. We report findings from a study with N=41 people with motor impairments (spinal cord injury, cerebral palsy, muscular dystrophy) and document their challenges and coping strategies for using the TV remote control, but also their television watching experience and expectations of suitable assistive technology for television. Our results show that, despite several accessible remote control products available on the market, the majority of our participants preferred to DIY and hack, and to adopt coping strategies to be able to use conventional TV remote controls. We contrast their experience against that of a control group with N=41 people without impairments. We reflect about the DIY culture and people with motor impairments, and we propose future work directions to increase the accessibility of interactions with television.

We examine the articulation characteristics of stroke-gestures produced by people with upper body motor impairments on touchscreens as well as the accuracy rates of popular classification techniques, such as the $-family, to recognize those gestures. Our results on a dataset of 9,681 gestures collected from 70 participants reveal that stroke-gestures produced by people with motor impairments are recognized less accurately than the same gesture types produced by people without impairments, yet still accurately enough (93.0%) for practical purposes; are similar in terms of geometrical criteria to the gestures produced by people without impairments; but take considerably more time to produce (3.4s vs. 1.7s) and exhibit lower consistency (-49.7%). We outline a research roadmap for accessible gesture input on touchscreens for users with upper body motor impairments, and we make our large gesture dataset publicly available in the community.