Drawing is a fundamental skill for children as it allows them to communicate their thoughts visually and develop their fine motor skills. For blind and low vision (BLV) students, traditional drawing tools can be challenging to use. For instance, it is difficult to determine the position and orientation of lines and shapes. Moreover, there are additional challenges related to the other sensory feedback available while drawing, such as haptic feedback and auditory feedback, which play a critical role in the tactile drawing process. There has been growing interest in developing accessible digital drawing tools that can be useful and usable for BLV students. These tools can overcome challenges faced by BLV students by providing them with more precise control over their drawing through additional feedback and by providing more flexibility in terms of input and output modalities. In this context, this work investigates the potential of bend gesture-based interaction as the primary input modality on a flexible smartphone-sized device, aiming to create an accessible drawing tool that corresponds to users’ existing mental model. We utilised the participatory design approach to design a bend gesture-based digital drawing tool named Naamya by involving BLV students and their teachers in the design process.

In the field of human-computer interaction, deformable user interfaces are gaining more attention. Gestures performed by bending the deformable device are commonly investigated in the literature. These bend gestures' inherent tactile and kinaesthetic feedback and their association with spatial directions benefit both blind and low vision (BLV) and sighted users. Bend gestures are naturally associated with two descriptors of bend, which are location and direction. One potential opportunity to extend this gesture space is to investigate additional descriptors of bend gestures without introducing new locations. In this context, size and angle are the two common additional descriptors of bend gestures. In this work, we conducted two (preliminary and main) studies to investigate BLV and sighted users' performance and preference for multiple magnitude levels of size and angle of bend on a smartphone-sized deformable prototype. Finally, we discuss the results and propose user group-specific design recommendations.

Reading and creating graphical information is a difficult task for users with visual impairment and blindness. It becomes even more challenging on touchscreen devices due to the lack of tactile buttons. However, advancement in flexible displays and electronics offers the potential use of physical deformation as an additional input modality. These deformation-based gestures provide innate tactile and kinesthetic feedback, which are essential for non-visual interaction. In this paper, I describe my Ph.D. research on non-visual drawing using the deformation gesture-based input method. This work is currently in the initial phase. This research aims to understand the preference and performance of deformation-based gestures on a smartphone-sized flexible handheld device and evaluate the effect of deformation and touch input modalities in the non-visual primitive geometric shape drawing. The expected outcome of this research can be useful for user interface designers and developers and researchers in developing accessible applications for future flexible devices.

Students with visual impairment and blindness learn 3D shapes using physical models, which pose portability issues and high associated costs. Although tactile and kinesthetic feedback systems have been proposed for non-visual exploration of 3D virtual objects, such systems tend to suffer from similar shortcomings. In this research, we propose TMOVE, a low-cost handheld tactile feedback actuator to provide tactile, vibrotactile, and combined feedback for the exploration of virtual space. We conducted a preliminary study with 10 blindfolded sighted participants to compare the perceived 3D experience and the effectiveness of using the feedback modalities in the exploration of two coplanar virtual line segments. We found that all the feedback modalities are equally effective in the exploration of virtual space, and multimodal feedback offers an enhanced 3D perception of virtual objects. We believe the findings presented in this paper will be helpful for designers and researchers in developing low-cost tactile feedback systems.

The landscape orientation of smartphone offers a better aspect ratio and extensive view for watching media and photography. However, it presents challenges of occlusion, reachability, and frequent re-gripping in one-handed interactions. To address these issues we took the opportunity of deformation gestures to interact with future flexible smartphones. A preliminary survey was conducted to understand one-handed landscape mode usage patterns. Then, the 1st study was conducted to identify 3 most preferred one-handed landscape mode grips. In the 2nd study, we gathered unique user-defined deformation gestures to identify the set of most natural and intuitive gestures corresponding to each grip. We also found 3 gestures that can be performed in more than one grip. Finally, we discuss the influence of the grips on performing gestures.