Design workshops are often inaccessible to blind people due to a strong reliance on visual design methods. This paper presents an adapted low-fidelity prototyping method to enable blind persons to actively participate in the early design stages of developing assistive devices. We chose the development of a 2D tactile display as a case study. Therefore, we conducted four workshops with blind participants to develop a haptic toolkit and adapt the structure of prototyping workshops based on the preferences of the target group. The resulting toolkit and workshop structure were applied in six prototyping workshops with 12 blind participants to design a new 2D tactile display to test the new method. Our findings show that the participation of blind individuals is highly relevant for structuring accessible prototyping workshops, deciding group size, and developing accessible prototyping materials. Furthermore, our work lays the foundation for further investigation of the role of sighted assistants and the design of workshops that truly center blind persons as primary contributors.

Designing accessible lectures and lecture materials is crucial to promote inclusive higher education.We conducted need-finding interviews with 12 students who are blind or have low vision to learn their perspectives on how lectures and lecture material could become more accessible through Artificial Intelligence (AI) technologies.Key insights from the interviews reveal that students envision AI to automatically customize lecture material, connect disparate information sources, for example, to better keep track of the current lecture slide, and enhance interaction and engagement with lecture material.Based on these insights, we developed the LectureAssistant prototype, employing an iterative design process with visually impaired users that features AI-assisted video navigation and chatbot interaction.In a final evaluation with seven students, the participants expressed enthusiasm for features such as AI-powered video search and the possibility of asking questions about visual content in the current video frame.They provided valuable suggestions for future improvements, including notifications for lecture slide transitions and the provision of a short overview function for a slide.Insights from the study indicate great potential of the prototype to improve accessibility of lecture videos for students with visual impairments, although they also point to crucial areas for improvement, such as more reliable and personalized image descriptions.

The use of assistive technologies for visually impaired musicians has been well documented, but the ability to visually read music and play simultaneously is still largely unexplored.Extended Reality (XR) could offer promising technology to support visually impaired people in this area.We address this research gap by examining the effectiveness of XR as a tool to read and play music at the same time, specifically for keyboard instruments.To achieve this, we chose a dual-phase human-centered approach that involved user participation.First, we conducted need-finding interviews with ten visually impaired participants to gather information about persisting challenges and their ideas on how to improve sheet music reading.Based on the results of these interviews, we then developed the XRMusic4VIP prototype.In the second phase, we conducted a user study with the same participants to evaluate the effectiveness of the prototype.Our findings indicate that the XRMusic4VIP prototype successfully enables the simultaneous reading of music and the playing of a keyboard instrument.Participants reported regaining their joy in making music and some of those who stopped playing because of their visual impairment can even imagine starting again.

Complex, poorly-designed tactile line chartDynamic audio-tactile UI.

In a world driven by data visualization, ensuring the inclusive accessibility of charts for Blind and Visually Impaired (BVI) individuals remains a significant challenge. Charts are usually presented as raster graphics without textual and visual metadata needed for an equivalent exploration experience for BVI people. Additionally, converting these charts into accessible formats requires considerable effort from sighted individuals. Digitizing charts with metadata extraction is just one aspect of the issue; transforming it into accessible modalities, such as tactile graphics, presents another difficulty. To address these disparities, we propose Chart4Blind, an intelligent user interface that converts bitmap image representations of line charts into universally accessible formats. Chart4Blind achieves this transformation by generating Scalable Vector Graphics (SVG), Comma-Separated Values (CSV), and alternative text exports, all comply with established accessibility standards. Through interviews and a formal user study, we demonstrate that even inexperienced sighted users can make charts accessible in an average of 4 minutes using Chart4Blind, achieving a System Usability Scale rating of 90%. In comparison to existing approaches, Chart4Blind provides a comprehensive solution, generating end-to-end accessible SVGs suitable for assistive technologies such as embossed prints (papers and laser cut), 2D tactile displays, and screen readers. For additional information, including open-source codes and demos, please visit our project page https://moured.github.io/chart4blind/.

Transparent objects, such as glass walls and doors, constitute architectural obstacles hindering the mobility of people with low vision or blindness. For instance, the open space behind glass doors is inaccessible, unless it is correctly perceived and interacted with. However, traditional assistive technologies rarely cover the segmentation of these safety-critical transparent objects. In this paper, we build a wearable system with a novel dual-head Transformer for Transparency (Trans4Trans) perception model, which can segment general- and transparent objects. The two dense segmentation results are further combined with depth information in the system to help users navigate safely and assist them to negotiate transparent obstacles. We propose a lightweight Transformer Parsing Module (TPM) to perform multi-scale feature interpretation in the transformer-based decoder. Benefiting from TPM, the double decoders can perform joint learning from corresponding datasets to pursue robustness, meanwhile maintain efficiency on a portable GPU, with negligible calculation increase. The entire Trans4Trans model is constructed in a symmetrical encoder-decoder architecture, which outperforms state-of-the-art methods on the test sets of Stanford2D3D and Trans10K-v2 datasets, obtaining mIoU of 45.13% and 75.14%, respectively. Through a user study and various pre-tests conducted in indoor and outdoor scenes, the usability and reliability of our assistive system have been extensively verified. Meanwhile, the Tran4Trans model has outstanding performances on driving scene datasets. On Cityscapes, ACDC, and DADA-seg datasets corresponding to common environments, adverse weather, and traffic accident scenarios, mIoU scores of 81.5%, 76.3%, and 39.2% are obtained, demonstrating its high efficiency and robustness for real-world transportation applications.

It is much more difficult for people with visual impairments to plan and implement a journey to unknown places than for sighted people, because in addition to the usual travel arrangements, they also need to know whether the different parts of the travel chain are accessible at all. The need for information is presumably therefore very high and ranges from knowledge about the accessibility of public transport as well as outdoor and indoor environments. However, to the best of our knowledge, there is no study that examines in-depth requirements of both the planning of a trip and its implementation, looking separately at the various special needs of people with low vision and blindness. In this paper, we present a survey with 106 people with visual impairments, in which we examine the strategies they use to prepare for a journey to unknown buildings, how they orient themselves in unfamiliar buildings and what materials they use. Our analysis shows that requirements for people with blindness and low vision differ. The feedback from the participants reveals that there is a large information gap, especially for orientation in buildings, regarding maps, accessibility of buildings and supporting systems. In particular, there is a lack of availability of indoor maps.

A lot of information is nowadays presented graphically. However, students with blindness do not have access to visual information. Providing an alternative text is not always the appropriate solution as exploring graphics to discover information independently is a fundamental part of the learning process. In this work, we introduce a mobile audio-tactile learning environment, which facilitates the incorporation of real educational material. We evaluate our system by comparing three methods of interaction with tactile graphics: A tactile graphic augmented by (1) a document with key index information in Braille, (2) a digital document with key index information and (3) the TPad system, an audio-tactile solution meeting the specific needs within the school context. Our study shows that the TPad system is suitable for educational environments. Moreover, compared to the other methods TPad is faster to explore tactile graphics and it suggests a promising effect on the memorization of information.

Digital navigation tools for helping people with visual impairments have become increasingly popular in recent years. While conventional navigation solutions give routing instructions to the user, systems such as GoogleMaps, BlindSquare, or Soundscape offer additional information about the surroundings and, thereby, improve the orientation of people with visual impairments. However, these systems only provide information about static environments, while dynamic scenes comprising objects such as bikes, dogs, and persons are not considered. In addition, both the routing and the information about the environment are usually conveyed by speech. We address this gap and implement a mobile system that combines object identification with a sonification interface. Our system can be used in three different scenarios of macro and micro navigation: orientation, obstacle avoidance, and exploration of known and unknown routes. Our proposed system leverages popular computer vision methods to localize 18 static and dynamic object classes in real-time. At the heart of our system is a mixed reality sonification interface which is adaptable to the user's needs and is able to transmit the recognized semantic information to the user. The system is designed in a user-centered approach. An exploratory user study conducted by us showed that our object-to-sound mapping with auditory icons is intuitive. On average, users perceived our system as useful and indicated that they want to know more about their environment, apart from wayfinding and points of interest.

Pedestrian navigation systems are rarely accessible or suit the needs of persons with visual impairments. They usually lack a standardized grammar for their speech instructions, forcing users to learn new types of instructions for each new system. Thus, we propose (1) a German grammar with syntax rules and vocabulary for mobile pedestrian navigation systems that take into account the special requirements of people with visual impairments. (2) a set of rules for specifying what should be spoken and when, given GPS accuracy in a city [18]. We describe (3) the methodology used to obtain the grammar as well as (4) a qualitative evaluation with orientation and mobility experts and with people with visual impairments who deployed our grammar during a user study. Our approach is the first of its kind, as there is no such grammar neither for German nor for English, as far as we know. It serves as a contribution to standardize pedestrian navigation speech instructions for people with visual impairments.

In this demonstration, we show a web-based sonification platform that allows blind users to interactively experience various information using two nowadays widespread technologies: modern web browsers that implement high-level JavaScript APIs and touch-sensitive displays. This way, blind users can easily access information such as, for example, maps or graphs. Our current prototype provides various sonifications that can be switched depending on the image type and user preference. The prototype runs in Chrome and Firefox on PCs, smart phones, and tablets.

In this paper, we present an approach to automatically revealing phonological classes within historically related languages. A newly created bilingual German-Dutch pronunciation dictionary is used for learning phonological similarities between the onsets, nuclei and codas of these two languages via EM-based clustering. Our evaluation is twofold: we apply the models to predict from a German word the phonemes of a Dutch cognate. The results show that it is harder to predict the pronunciation of the nucleus and the coda than the onset. We also evaluate our approach qualitatively, finding meaningful classes caused by historical sound changes.

An approach to automatic detection of syllable boundaries is presented. We demonstrate the use of several manually constructed grammars trained with a novel algorithm combining the advantages of treebank and bracketed corpora training. We investigate the effect of the training corpus size on the performance of our system. The evaluation shows that a hand-written grammar performs better on finding syllable boundaries than does a treebank grammar.

An approach to automatic detection of syllable structure is presented. We demonstrate a novel application of EM-based clustering to multivariate data, exemplified by the induction of 3- and 5-dimensional probabilistic syllable classes. The qualitative evaluation shows that the method yields phonologically meaningful syllable classes. We then propose a novel approach to grapheme-to-phoneme conversion and show that syllable structure represents valuable information for pronunciation systems.