Despite the potential of multimodal assistive technologies (MATs) to convey visual information, such as music notation, to blind or low-vision (BLV) individuals, we do not fully understand how MATs can be used to improve music reading and memorization. Through ideation and co-design workshops, we explored how modalities, such as sound and vibration, can improve music reading and memorization through hands-free timely interactions and reminders. Our design workshops presented a unique opportunity for BLV musicians and learners to collaborate and actively engage in the research and design process informed by their individual perspectives and lived experiences. We classified the complex challenges of reading and memorizing music into intrinsic (related to the cognitive aspects of music understanding) and extraneous (pertaining to external factors such as interaction and access) complexities and found that specific modalities are well suited to tackle particular problems. We conclude by outlining design implications and future research directions aimed at developing MATs that holistically improve music learning for BLV people.

Musical haptic wearables (MHWs) that convey information through vibrotactile feedback holds the potential to support the music learning of a blind or low vision (BLV) music learner. Yet, it is unclear how these technologies can give functional support to a BLV person. We also investigated material preferences in such technologies to understand the role of non-functional aesthetic experiences in shaping their music learning. We conducted 5 co-design workshops with 10 BLV participants. Participants explored eleven materials common in a music learning environment and engaged in bodystorming with a prototype that communicated six vibrotactile patterns. Through thematic analysis, we found that MHWs with vibrotactile alerts and variations in vibration are suited to communicate instructional information, aid music reading and support technical guidance and practice. We categorized the participants’ material experiences into sensorial, interpretive, and affective levels. Based on our findings, we discuss considerations when designing vibrotactile interactions to support music learning for BLV people and highlight material experiences that should be emphasized to make the music learning experience wholesome for BLV music learners.

Learning to play a musical instrument and engaging in musical activities have enabled blind and/or low vision people to develop self-identity, find community and pursue music as a career. However, blind and/or low vision music learners face complex obstacles to learn music. They are highly reliant on their learning environment and music teachers for accommodations and flexibility. Prior research has identified the challenges faced by blind and/or low vision musicians and recognized the importance of touch for music reading and physical guidance. However, limited research has addressed these challenges through the development of assistive technology. The development of music computer technologies with haptics and the affordances of wearable technologies provides encouraging opportunities to develop haptic wearable devices to support blind and/or low vision music learning. I identify three unexplored research questions: (1) what design considerations must be addressed in future assistive technologies for BLV music learning, (2) how can wearable technologies with vibrotactile feedback support BLV student-teacher interactions, and (3) what are the long-term benefits and limitations of the use of assistive technologies for BLV music learning? I outline my research to date and highlight my findings.