Combining conversational AI with refreshable tactile displays (RTDs) offers significant potential for creating accessible data visualization for people who are blind or have low vision (BLV). To support researchers and developers building accessible data visualizations with RTDs, we present a multimodal data interaction architecture along with an open-source reference implementation. Our system is the first to combine touch input with a conversational agent on an RTD, enabling deictic queries that fuse touch context with spoken language, such as "what is the trend between these points?" The architecture addresses key technical challenges, including touch sensing on RTDs, visual-to-tactile encoding, integrating touch context with conversational AI, and synchronizing multimodal output. Our contributions are twofold: (1) a technical architecture integrating RTD hardware, external touch sensing, and conversational AI to enable multimodal data interaction; and (2) an open-source reference implementation demonstrating its feasibility. This work provides a technical foundation to support future research in multimodal accessible data visualization.

3D-printed models are increasingly used to provide people who are blind or have low vision (BLV) with access to maps, educational materials, and museum exhibits. Recent research has explored interactive 3D-printed models (I3Ms) that integrate touch gestures, conversational dialogue, and haptic vibratory feedback to create more engaging interfaces. Prior research with sighted people has found that imbuing machines with human-like behaviours, i.e., embodying them, can make them appear more lifelike, increasing social perception and presence. Such embodiment can increase engagement and trust. This work presents the first exploration into the design of embodied I3Ms and their impact on BLV engagement and trust. In a controlled study with 12 BLV participants, we found that I3Ms using specific embodiment design factors, such as haptic vibratory and embodied personified voices, led to an increased sense of liveliness and embodiment, as well as engagement, but had mixed impact on trust.

Despite the recent surge of research efforts to make data visualizations accessible to people who are blind or have low vision (BLV), how to support BLV people's data analysis remains an important and challenging question. As refreshable tactile displays (RTDs) become cheaper and conversational agents continue to improve, their combination provides a promising approach to support BLV people's interactive data exploration and analysis. To understand how BLV people would use and react to a system combining an RTD with a conversational agent, we conducted a Wizard-of-Oz study with 11 BLV participants, where they interacted with line charts, bar charts, and isarithmic maps. Our analysis of participants' interactions led to the identification of nine distinct patterns. We also learned that the choice of modalities depended on the type of task and prior experience with tactile graphics, and that participants strongly preferred the combination of RTD and speech to a single modality. In addition, participants with more tactile experience described how tactile images facilitated a deeper engagement with the data and supported independent interpretation. Our findings will inform the design of interfaces for such interactive mixed-modality systems.

3D printed models have been used to improve access to graphical information by people who are blind, offering benefits over conventional accessible graphics. Here we investigate an interactive 3D printed model (I3M) that combines a conversational interface with haptic vibration and touch to provide more natural and accessible experiences. Specifically, we co-designed a multimodal model of the Solar System with nine blind people and evaluated the prototype with another seven blind participants. We discuss our journey from a design perspective, focusing on touch, conversational and multimodal interactions. Based on our experience, we suggest design recommendations that consider blind users’ desire for independence and control, customisation, comfort and use of prior experience.

This paper presents a systematic literature review of 292 publications from 97 unique venues on touch-based graphics for people who are blind or have low vision, from 2010 to mid-2020. It is the first review of its kind on touch-based accessible graphics. It is timely because it allows us to assess the impact of new technologies such as commodity 3D printing and low-cost electronics on the production and presentation of accessible graphics. As expected our review shows an increase in publications from 2014 that we can attribute to these developments. It also reveals the need to: broaden application areas, especially to the workplace; broaden end-user participation throughout the full design process; and conduct more in situ evaluation. This work is linked to an online living resource to be shared with the wider community.

A critical component of the worldwide response to the novel coronavirus COVID-19 pandemic has been providing the general public with information about the virus and the health measures designed to slow its spread. Much of this information has been presented as visual graphics. Have people who are blind or have low vision (BLV) been able to gain access to this information through nonvisual media and, if so, how? We investigate this issue using Australia as a case study.

While developments in 3D printing have opened up opportunities for improved access to graphical information for people who are blind or have low vision (BLV), they can provide only limited detailed and contextual information. Interactive 3D printed models (I3Ms) that provide audio labels and/or a conversational agent interface potentially overcome this limitation. We conducted a Wizard-of-Oz exploratory study to uncover the multi-modal interaction techniques that BLV people would like to use when exploring I3Ms, and investigated their attitudes towards different levels of model agency. These findings informed the creation of an I3M prototype of the solar system. A second user study with this model revealed a hierarchy of interaction, with BLV users preferring tactile exploration, followed by touch gestures to trigger audio labels, and then natural language to fill in knowledge gaps and confirm understanding.

The difficulty and consequent fear of travel is one of the most disabling consequences of blindness and severe vision impairment, affecting confidence and quality of life. Traditional tactile graphics are vital in the Orientation and Mobility training process, however 3D printing may have the capacity to enable production of more meaningful and inclusive maps. This study explored the use of 3D printed maps on site at a public event to examine their suitability and to identify guidelines for the design of future 3D maps. An iterative design process was used in the production of the 3D maps, with feedback from visitors who are blind or have low vision informing the recommendations for their design and use. For example, it was found that many representational 3D icons could be recognised by touch without the need for a key and that such a map helped form mental models of the event space. Complex maps, however, require time to explore and should be made available before an event or at the entrance in a comfortable position. The maps were found to support the orientation and mobility process, and importantly to also promote a positive message about inclusion and accessibility.