Existing visual AI assistive technologies have usability gaps, and may need additional adaptations and features to serve users’ needs. We propose A11yExtensions, in-situ interventions that augment existing mobile AI assistive technology with add-on services. Add-ons include features that have been researched but are not yet deployed (e.g., cross-checking AI results), or that are only available in certain applications (e.g., camera aiming assistance). Through co-design sessions with two blind accessibility professionals, we designed and implemented three exemplar extensions, leveraging mobile automation tools to invoke add-ons, enabling just-in-time interventions for adaptability. We found that A11yExtensions provide opportunities to test new features and a new degree of flexibility and customization, though they introduce additional onboarding and communication challenges. We also derived a design space of accessibility extensions as a basis for future extension designs. Overall, A11yExtensions is a demonstration of the effectiveness of deploying new features in-situ via automation, with the technologies people actually use in their day-to-day lives.

Tactile graphics present visual information to blind and visually-impaired individuals in an accessible way, through touch. Current methods for producing tactile graphics, such as embossing or swell-paper printing, have limitations such as durability - and the tools required to produce them are limited in expressiveness. In this project, we explore embroidery as a medium for producing tactile graphics. Embroidery, traditionally known for its variety and visual beauty, offers not just improved durability and ease of production - but the ability to convey information through a broad range of stitch types. Following an exploration of the design space of embroidered tactile graphics, we identify key perceptual properties that impact how embroidered textures are differentiated. Based on these differences, we introduce an optimization algorithm for assigning textures to regions of tactile graphics in a way that makes them diverse and legible. We implement an end-to-end pipeline for producing embroidered tactile graphics and evaluate the comprehensibility and legibility of our design with 6 blind participants. Our findings showed that embroidered tactile graphics present information accurately and comprehensively, and that measurable properties, such as the use of spacing and distinctiveness, were an important factor of expressive and legible design.

For non-technical domain experts and designers it can be a substantial challenge to create designs that meet domain specific goals. This presents an opportunity to create specialized tools that produce optimized designs in the domain. However, implementing domain-specific optimization methods requires a rare combination of programming and domain expertise. Creating flexible design tools with re-configurable optimizers that can tackle a variety of problems in a domain requires even more domain and programming expertise. We present OPTIMISM, a toolkit which enables programmers and domain experts to collaboratively implement an optimization component of design tools. OPTIMISM supports the implementation of metaheuristic optimization methods by factoring them into easy to implement and reuse components: objectives that measure desirable qualities in the domain, modifiers which make useful changes to designs, design and modifier selectors which determine how the optimizer steps through the search space, and stopping criteria that determine when to return results. Implementing optimizers with OPTIMISM shifts the burden of domain expertise from programmers to domain experts.