Blind and Low-vision (BLV) people face challenges in their daily lives, especially regarding independent navigation in unfamiliar settings. Assistive robots have recently emerged as assistive tools for people with special needs. Drones, in particular, have unique advantages over ground robots. Our work aims at designing a drone assistant in addition to primary mobility aids (i.e., white cane) to improve spatial awareness and navigation in unfamiliar settings. Using a Research-through-Design method with 21 BLV users, we identified challenging scenarios and the need for two distinct operating modes (Explorer and Team). We then deployed a low-fidelity prototype in a Wizard-of-Oz study. Participants were enthusiastic about the assistive drone. The outcomes were reviewed by low vision specialists and drone experts to critically assess the strengths and limitations of this assistive tool. Based on these results, we provide recommendations for future design of assistive drones tailored to BLV users.

Current scene perception tools for Blind and Low Vision (BLV) individuals rely on spoken descriptions but lack engaging representations of visually pleasing distant environmental landscapes (Vista spaces). Our proposed Scene2Audio framework generates comprehensible and enjoyable nonverbal audio using generative models informed by psychoacoustics, and principles of scene audio composition. Through a user study with 11 BLV participants, we found that combining the Scene2Audio sounds with speech creates a better experience than speech alone, as the sound effects complement the speech making the scene easier to imagine. A mobile app “in-the-wild” study with 7 BLV users for more than a week further showed the potential of Scene2Audio in enhancing outdoor scene experiences. Our work bridges the gap between visual and auditory scene perception by moving beyond purely descriptive aids, addressing the aesthetic needs of BLV users.

Drones have gained traction as a versatile form of assistive robots for Blind and Low Vision (BLV) people. Nonetheless, novel interaction techniques are required to enable BLV people to communicate with drones naturally. In this work, we built an LLM-powered assistive drone for BLV users. We leverage an LLM to translate high-level user goals to step-by-step instructions for the drone and to extract visual information from the images. Through a formative study with BLV users (N=9), we identified envisioned use cases and desired interaction modalities. Then, we took a participatory and iterative approach to build a prototype, incorporating feedback received from 3 BLV users, as well as 5 domain experts. Finally, we conducted a user study with an additional 6 BLV participants to evaluate the iterated prototype, and received positive feedback. This work is contributing to a growing body of research on harnessing the power of LLMs to build a more inclusive world.

Ensuring the safety of personnel at large industrial sites is a significant challenge. Misunderstandings and increased safety risks are related to communication, as traditional non-targeted alerts like sirens are often ineffective. To address this, we propose a drone-based system that uses voice interaction to deliver timely and targeted safety advisories. This paper presents a multi-stage, user-centered study, beginning with formative interviews with security and safety managers and culminating in an on-site evaluative study with personnel at two biomass power plants. Through our study, we identify key design insights for creating a drone that is perceived as an official, authoritative, and non-threatening entity. We provide design guidance for using sound and movement to signal intent, creating a voice persona that balances authority with clarity, and designing interaction flows to manage confusion and non-compliance. The results demonstrate the system's potential not only to enhance safety compliance, but also to mitigate interpersonal confrontation. First, we provide an empirically-grounded understanding of contextual safety challenges, which informs a set of design principles for intelligible and socially acceptable voice advisories from drones. Second, we present results from an in-situ evaluation of a working prototype that demonstrates its effectiveness in a real-world setting.

Figure 1: Current status and future vision of drone-assisted firefighting in Thailand: enhancing situational awareness, resource management, and decision-making in forest and industrial fire emergencies.

International audience

International audience

Programming has become a highly sought-after skill in STEM-related studies and careers, but it has only reached a fraction of students with visual impairments. Therefore, there is a need to explore new methods for teaching and learning. This study aims to understand the potential of using drones to create an authentic learning environment to help students with visual impairments learn programming. Based on a month-long engagement with five students with visual impairments, we present insights on using drones to support programming education for students with visual impairments.

Self-localization and wayfinding are challenging tasks for people with visual impairments (PVIs), severely impacting independent mobility. Visual “You-are-here” (YAH) maps are useful for assisting local wayfinding of sighted users. They are used to self-localize and display points of interest, landmarks and routes in the surroundings. However, these maps are not always available and rarely accessible to PVIs. Relying on an iterative participatory design process with eight end-users with visual impairments, we created a proof of concept of a mobile audio-vibratory YAH map. Our design is based on either a tablet or a smartphone to ensure a small and portable solution. A user study with ten PVIs showed that the audio-vibratory YAH map that we designed provides the user with a good understanding of the surroundings and wayfinding cues. Surprisingly, the results show that the audio-vibratory YAH map prototype was as usable as the control condition (audio-tactile YAH map with a tactile overlay), with similar user satisfaction and cognitive load. A follow-up field study with two participants showed the effectiveness of the prototype for assisting in crossroad understanding. To conclude, our innovative design of a mobile audio-vibratory YAH map can overcome the portability and printing issues associated with tactile overlays and can be an appropriate solution for assisting the pedestrian navigation of PVIs.

The understanding of multi-level spatial topologies is a difficult and frequent challenge in people with visual impairments daily life, impacting their independent mobility. Using the tools of the “maker” movement, and following an iterative co-design process with Orientation and Mobility instructors, we created an innovative tool (3D printed interactive model of a train station) for teaching complex spatial knowledge. Then, we did a comparative study with end users between the 3D interactive model that we designed and two 2D interactive tactile maps representing the same location. Our results show that the 3D interactive model is useful and usable, provides better satisfaction and is preferred to 2D tactile maps. In addition, complex spatial notions are better understood with the 3D model. Altogether, these results suggest that the “maker movement” may empower special education teachers with adapted and innovative tools.

We present a set of 8 thumb-to-finger microgestures (TTF µGestures) that can be used as an additional modality to enrich touch interaction in eyes-free situations. TTF µGestures possess characteristics especially suited for people with visual impairment (PVI). They have never been studied specifically for PVI to improve accessibility of touchscreen devices. We studied a set of 33 common TTF µGestures to determine which are feasible and usable without seeing while the index is touching a surface. We found that the constrained position of the hand and the absence of vision prevent participants from being able to efficiently target a specific phalanx.

The lockdown period related to the COVID-19 pandemic has had a strong impact on the educational system in general, but more particularly on the special education system. Indeed, in the case of people with visual impairments, the regular tools relying heavily on images and videos were no longer usable. This specific situation highlighted an urgent need to develop tools that are accessible and that can provide solutions for remote teaching with people with VI. However, there is little work on the difficulties that this population encounters when they learn remotely as well as on the current practices of special education teachers. Such a lack of understanding limits the development of remote teaching systems that are adapted. In this paper, we conducted an online survey regarding the practices of 59 professionals giving lessons to pupils with VI, followed by a series of focus groups with special education workers facing teaching issues during the lockdown period. We followed an iterative design process where we designed successive low-fidelity prototypes to drive successive focus groups. We contribute with an analysis of the issues faced by special education teachers in this situation, and a concept to drive the future development of a tool for remote graphic-based teaching with pupils with VI.

Raised-line graphics are tactile documents made for people with visual impairments (VI). Their exploration relies on a complex two-handed behavior. To better understand the cognitive processes underlying this exploration, we proposed a new method based on “tactile fixations”. A tactile fixation occurs when a finger is stationary within a specific spatial and temporal window. It is known that stationary fingers play an active role when exploring tactile graphics, but they have never been defined or studied before. In this study, we first defined the concept of tactile fixation, then we conducted a behavioral study with ten participants with VI in order to assess the role of tactile fixations under different conditions. The results show that tactile fixations vary according to different factors such as the graphic type as well as the involved hand and the aim of the exploration.

Board games allow us to share collective entertainment experiences. They entertain because of the interactions between players, physical manipulation of tokens and decision making. Unfortunately, most board games exclude people with visual impairments as they were not initially designed for players with special needs. Through a user-centered design process with an accessible game library and visually impaired players, we observed challenges and solutions in making existing board games accessible through handcrafted solutions (tactile stickers, braille labels, etc.). In a second step, we used Spatial Augmented Reality (SAR), to make existing board games inclusive by adding interactivity (GameARt). In a case study with an existing board game considered as not accessible (Jamaica), we designed an interactive SAR version with touch detection (JamaicAR). We evaluated this prototype in a user study with 5 groups of 3 players each, including sighted, low vision and blind players. All players, independent of visual status, were able to play the Augmented Reality game. Moreover, the game was rated positively by all players regarding attractiveness, play engrossment, enjoyment and social connectivity. Our work shows that Spatial Augmented Reality has the potential to make board games accessible to people with visual impairments when handcrafted adaptations fall short.

Tangible User Interfaces (TUI) have been found to be relevant tools for collaborative learning by providing a shared workspace and enhancing joint visual attention. Researchers have explored the use of TUIs in a variety of curricular activities and found them particularly interesting for spatial exploration. However, very few studies have explored how TUIs could be used as a collaborative medium for people with visual impairments (VIs). In this study, we investigated the effect of tangible interaction (a small tangible robot) in a spatial collaborative task (a treasure hunt) involving two people with VIs. The aim was to evaluate the impact of the design of the TUI on the collaboration and the strategies used to perform the task. The experiment involved six dyads of people with VIs. The results showed that the collaboration was impacted by the interaction design and open interesting perspectives on the design of collaborative games for people with VIs.

Visually impaired people encounter many challenges in their everyday life, especially when it comes to navigating and representing space. The issue of shopping is addressed mostly on the level of navigation and product detection, but conveying clues about the object position to the user is rarely implemented. This work presents a prototype of vibrotactile wristband using spatiotemporal patterns to help visually impaired users reach an object in the 2D plane in front of them. A pilot study on twelve blindfolded sighted subjects showed that discretizing space in a seven by seven targets matrix and conveying clues with a discrete pattern on the vertical axis and a continuous pattern on the horizontal axis is an intuitive and effective design.

Addressing the needs of visually impaired people is of continued interest in Human Computer Interaction (HCI) research. Yet, one of the major challenges facing researchers in this field continues to be how to design adequate quantitative empirical evaluation for these users in HCI. In this paper, we analyse a corpus of 178 papers on technologies designed for people with visual impairments, published since 1988, and including at least one quantitative empirical evaluation (243 evaluations in total). To inform future research in this area, we provide an overview, historic trends and a unified terminology to design and report quantitative empirical evaluations. We identify open issues and propose a set of guidelines to address them. Our analysis aims to facilitate and stimulate future research on this topic.

Despite being included in mainstream schools, visually impaired children still face barriers to social engagement and participation. Games could potentially help, but games that cater for both visually impaired and sighted players are scarce. We used a co-design approach to design and evaluate a robot-based educational game that could be inclusive of both visually impaired and sighted children in the context of mainstream education. We ran a focus group discussion with visual impairment educators to understand barriers to inclusive play. And then a series of co-design workshops to engage visually impaired and sighted children and educators in learning about robot technology and exploring its potential to support inclusive play experiences. We present design guidelines and an evaluation workshop of a game prototype, demonstrating group dynamics conducive to collaborative learning experiences, including shared goal setting/execution, closely coupled division of labour, and interaction symmetry.

Low vision is a visual impairment that affects one's capacity to perform daily activities, among others, reading. Since difficulty to access printed materials is an obstacle to education, employment and social activities, adaptations of these material is often required, such as the usage of large print or audio transcriptions. The advent of digital documents, on computers and mobile devices, has enhanced the accessibility of text for people with low vision; however, there is still a lack of supply regarding accessible books for recreation reading. In this work, we study how an accessible reader can impact reader's engagement with comics, for both children and adults. We summarise the results of online surveys that have been conducted about the current usage of comics by reader with low vision and how professional attempt to provide them. This led to the design of ALCOVE, an accessible comic reader for people with low vision. We report the results of a user study with 11 participants with low vision, that examined both the usefulness of the proposed comic reader and the reading presentation mode. Results show that the participants found our application easy and very convenient to use.

Visual graphics are widely spread in digital media and are useful in many contexts of daily life. However, access to this type of graphical information remains a challenging task for people with visual impairments (VI). In this study, we designed and evaluated an on-hand vibrotactile interface that enables users with VI to explore digital graphics presented on tablets. We first conducted a set of exploratory tests with both people with VI and blindfolded (BF) people to investigate several design factors. We then conducted a comparative experiment to verify that on-hand vibrotactile cues (indicating direction and progression) can enhance the non-visual exploration of digital graphics. The results based on 12 participants with VI and 12 BF participants confirmed the usability of the technique and revealed that the visual status of the users does not impact graphics identification and comparison tasks.

Interactive tactile graphics have shown a true potential for people with visual impairments, for instance for acquiring spatial knowledge. Until today, however, they are not well adopted in real-life settings (e.g. special education schools). One obstacle consists in the creation of these media, which requires specific skills, such as the use of vector-graphic software for drawing and inserting interactive zones, which is challenging for stakeholders (social workers, teachers, families of people with visual impairments, etc.). We explored how a Spatial Augmented Reality approach can enhance the creation of interactive tactile graphics by sighted users. We developed the system using a participatory design method. A user study showed that the augmented reality device allowed stakeholders (N=28) to create interactive tactile graphics more efficiently than with a regular vector-drawing software (baseline), independently of their technical background.

The development of novel shape-changing or actuated tabletop tangible interfaces opens new perspectives for the design of physical and dynamic maps, especially for visually impaired (VI) users. Such maps would allow non-visual haptic exploration with advanced functions, such as panning and zooming. In this study, we designed an actuated tangible tabletop interface, called BotMap, allowing the exploration of geographic data through non-visual panning and zooming. In BotMap, small robots represent landmarks and move to their correct position whenever the map is refreshed. Users can interact with the robots to retrieve the names of the landmarks they represent. We designed two interfaces, named Keyboard and Sliders, which enable users to pan and zoom. Two evaluations were conducted with, respectively, ten blindfolded and eight VI participants. Results show that both interfaces were usable, with a slight advantage for the Keyboard interface in terms of navigation performance and map comprehension, and that, even when many panning and zooming operations were required, VI participants were able to understand the maps. Most participants managed to accurately reconstruct maps after exploration. Finally, we observed three VI people using the system and performing a classical task consisting in finding the more appropriate itinerary for a journey.

Raised-line diagrams are widely used by visually impaired (VI) people to read maps, drawings or graphs. While previous work has identified general exploration strategies for raised-line drawings, we have limited knowledge on how this exploration is performed in detail and how it extends to other types of diagrams such as maps or graphs, frequently used in specialized schools. Such information can be crucial for the design of accessible interfaces on touchscreens. We conducted a study in which participants were asked to explore five types of raised-line diagrams (common drawings, perspective drawings, mathematical graphs, neighborhood maps, and geographical maps) while tracking both hands fingers. Relying on a first set of results, we proposed a set of design guidelines for touch interfaces.

We report on the design process leading to the creation of MapSense, a multi-sensory interactive map for visually impaired children. We conducted a formative study in a specialized institute to understand children's educational needs, their context of care and their preferences regarding interactive technologies. The findings (1) outline the needs for tools and methods to help children to acquire spatial skills and (2) provide four design guidelines for educational assistive technologies. Based on these findings and an iterative process, we designed and deployed MapSense in the institute during two days. It enables collaborations between children with a broad range of impairments, proposes reflective and ludic scenarios and allows caretakers to customize it as they wish. A field experiment reveals that both children and caretakers considered the system successful and empowering.

Maps are essential in everyday life, but inherently inaccessible to visually impaired users. They must be transcribed to non-editable tactile graphics, or rendered on very expensive shape changing displays. To tackle these issues, we developed a tangible tabletop interface that enables visually impaired users to build tangible maps on their own, using a new type of physical icon called Tangible Reels. Tangible Reels are composed of a sucker pad that ensures stability, with a retractable reel that renders digital lines tangible. In order to construct a map, audio instructions guide the user to precisely place Tangible Reels onto the table and create links between them. During subsequent exploration, the device provides the names of the points and lines that the user touches. A pre-study confirmed that Tangible Reels are stable and easy to manipulate, and that visually impaired users can understand maps that are built with them. A follow-up experiment validated that the designed system, including non-visual interactions, enables visually impaired participants to quickly build and explore maps of various complexities.

Tactile raised-line maps are paper maps widely used by visually impaired people. We designed a mobile technique, based on hand tracking and a smartwatch, in order to leverage pervasive access to virtual maps. We use the smartwatch to render localized text-to-speech and vibratory feedback during hand exploration, but also to provide filtering functions activated by swipe gestures. We conducted a first study to compare the usability of a raised-line map with three virtual maps (plain, with filter, with filter and grid). The results show that virtual maps are usable, and that adding a filter, or a filter and a grid, significantly speeds up data exploration and selection. The results of a following case study showed that visually impaired users were able to achieve a complex task with the device, i.e. finding spatial correlations between two sets of data.

Tactile relief maps are used by visually impaired people to acquire mental representation of space, but they retain important limitations (limited amount of information, braille text, etc.). Interactive maps may overcome these limitations. However, usability of these two types of maps has never been compared. It is then unknown whether interactive maps are equivalent or even better solutions than traditional raised-line maps. This study presents a comparison of usability of a classical raised-line map versus an interactive map composed of a multitouch screen, a raised-line overlay, and audio output. Both maps were tested by 24 blind participants. We measured usability as efficiency, effectiveness, and satisfaction. Our results show that replacing braille with simple audio-tactile interaction significantly improved efficiency and user satisfaction. Effectiveness was not related to the map type but depended on users’ characteristics as well as the category of assessed spatial knowledge. Long-term evaluation of acquired spatial information revealed that maps, whether interactive or not, are useful to build robust survey-type mental representations in blind users. Altogether, these results are encouraging as they show that interactive maps are a good solution for improving map exploration and cognitive mapping in visually impaired people.

In the last few decades, a growing number of Electronic Orientation Aids (EOA) has been developed with the purpose of improving the autonomy of visually impaired people. However, the majority of those systems are not used by the blind due to limited usability. The main challenges to be addressed are about interaction and guidance. To address these issues, we designed a multimodal (input and output) Virtual Environment (VE) that simulates different interactions that could be used for space perception and guidance in an EOA. This platform subserves two goals: help designers to systematically test guidance strategies (i.e. for the development of new EOAs) and train blind people to use interactive EOAs, with an emphasis on cognitive mapping enhancement. In a multimodal VE, both objectives are assessed in a controlled, cost-effective, safe and flexible environment.

We have developed a real-time portable object recognition system based on a bio-inspired image analysis software to increase blind people autonomy by localizing and identifying surrounding objects. A working prototype of this system has been tested on the issue of currency bill recognition encountered by most of the blind people. Seven blind persons were involved in an experiment which demonstrated that the usability of the system was good enough for such a device to be used daily in real-life situations.

There are many electronic devices for the visually impaired but few actually get used on a daily basis. This is due in part to the fact that many devices often fail to address the real needs of the users. In this study, we begin with a review of the existing literature followed by a survey of 54 blind people which suggests that one particular function could be particularly useful in a new device, namely, the ability to localize objects. We have tested the possibility of using a sound rendering system to indicate a particular spatial location, and propose to couple this with a biologically inspired image processing system that can locate visual patterns that correspond to particular objects and places. We believe that such a system can address one of the major problems faced by the visually handicapped, namely their difficulty in localizing specific objects.