Millions of people around the world experience blindness or moderate to severe visual disability, who need to rely on screen readers to perceive the content of phone screens. Guidelines and testing tools developed to aid software developers suffer from inconsistency in categorizing accessibility issues and not faithfully representing real user experience. In this paper, we aim to construct a better classification of accessibility issues, integrating feedback from screen reader users to existing computational methods. First, we conduct a systematic literature review, investigating 31 papers that demonstrated automated interventions for mobile accessibility. We juxtapose their computationally addressed issues with real user experience, by observing blind users’ interaction on 4 apps across 20 user studies. Synthesizing the two studies, we construct a categorization and guideline for screen reader accessibility issues on mobile, aimed to initiate a more user-aware understanding and subsequent interventions towards accessible mobile app development.

Testing for accessibility is a significant step when developing software, as it ensures that all users, including those with disabilities, can effectively engage with web and mobile applications. While automated tools exist to detect accessibility issues in software, none are as comprehensive and effective as the process of user testing, where testers with various disabilities evaluate the application for accessibility and usability issues. However, user testing is not popular with software developers as it requires conducting lengthy interviews with users and later parsing through large recordings to derive the issues to fix. In this paper, we explore how large language models (LLMs) like GPT 4.0, which have shown promising results in context comprehension and semantic text generation, can mitigate this issue and streamline the user testing process. Our solution, called Reca11, takes in auto-generated transcripts from user testing video recordings and extracts the accessibility and usability issues mentioned by the tester. Our systematic prompt engineering determines the optimal configuration of input, instruction, context and demonstrations for best results. We evaluate Reca11's effectiveness on 36 user testing sessions across three applications. Based on the findings, we investigate the strengths and weaknesses of using LLMs in this space.

An estimated 43.3 million people worldwide live with blindness and rely on screen readers (SRs) to access the web. To support accessible development, software teams often rely on automated tools like WAVE and Lighthouse to detect accessibility issues. However, these tools primarily rely on static rule-based analysis and are largely limited to detecting labeling errors relevant to screen reader users. They fail to capture dynamic accessibility issues—specifically, whether user interface (UI) elements can be located and activated using a screen reader, which is essential for accessing core webpage functionality. To address this gap, we present A1 1yNavigator, an automated accessibility testing tool that simulates screen reader navigation to detect UI elements that cannot be either (1) located or (2) activated via the screen reader. A11yNavigator leverages NVDA, one of the most widely used screen readers, and supports three common navigation strategies: Tab, Arrow, and quick Navigation keys. We evaluate A1 1yNavigator across 26 real-world websites and demonstrate its effectiveness in uncovering issues missed by existing tools. Our results highlight its high precision and recall in detecting barriers that go beyond static analysis.

Ads are integral to the contemporary Android ecosystem, generating revenue for free-to-use applications. However, injected as third-party content, ads are displayed on native apps in pervasive ways that affect easy navigation. Ads can prove more disruptive for blind users, who rely on screen readers for navigating an app. While the literature has looked into either the accessibility of web advertisements or the privacy and security implications of mobile ads, a research gap on the accessibility of mobile ads remains, which we aim to bridge. We conduct an empirical study analyzing 500 ad screens in Android apps to categorize and examine the accessibility issues therein. Additionally, we conduct 15 qualitative user interviews with blind Android users to better understand the impact of those accessibility issues, how users interact with ads and their preferences. Based on our findings, we discuss the design and practical strategies for developing accessible ads.

It is crucial to make software, with its ever-growing influence on everyday lives, accessible to all, including people with disabilities. Despite promoting software accessibility through government regulations, development guidelines, tools and frameworks, investigations reveal a marketplace of inaccessible web and mobile applications. To better understand the limitations of contemporary software industry in adopting accessibility practices, it is necessary to construct a holistic view that combines the perspectives of software practitioners, stakeholders and end users. In this paper, we collect 637 conversations from Twitter to synthesize and qualitatively analyze discussions posted about software accessibility. Our findings observe an active community that provides feedback on inaccessible software, shares personal accounts of development practices and advocates for inclusivity. By perceiving software accessibility from process, profession and people viewpoints, we present current conventions, challenges and possible resolutions with four emergent themes: cost and incentives, awareness and advocacy, technology and resources, and integration and inclusion.

Mobile apps, an essential technology in today’s world, should provide equal access to all, including 15% of the world population with disabilities. Assistive Technologies (AT), with the help of Accessibility APIs, provide alternative ways of interaction with apps for disabled users who cannot see or touch the screen. Prior studies have shown that mobile apps are prone to the under-access problem, i.e., a condition in which functionalities in an app are not accessible to disabled users, even with the use of ATs. We study the dual of this problem, called the over-access problem, and defined as a condition in which an AT can be used to gain access to functionalities in an app that are inaccessible otherwise. Over-access has severe security and privacy implications, allowing one to bypass protected functionalities using ATs, e.g., using VoiceOver to read notes on a locked phone. Over-access also degrades the accessibility of apps by presenting to disabled users information that is actually not intended to be available on a screen, thereby confusing and hindering their ability to effectively navigate. In this work, we first empirically study overly accessible elements in Android apps and define a set of conditions that can result in over-access problem. We then present OverSight, an automated framework that leverages these conditions to detect overly accessible elements and verifies their accessibility dynamically using an AT. Our empirical evaluation of OverSight on real-world apps demonstrates OverSight’s effectiveness in detecting previously unknown security threats, workflow violations, and accessibility issues.

The outcome of software products primarily depends on the developers, including their emotion or sentiment in a software development environment. Developer emotions have been observed to be correlated to several patterns, for instance, task resolution time, developer turnover, etc. by conducting sentiment analysis on software collaborative artifacts like Commits. This study aims to quantify the impact of those patterns by finding a relation between developer sentiment and software bugs. To do so, Fix-Inducing Changes — changes that introduce bugs to the system — are detected, along with changes that precede or fix those bugs. Sentiment of these changes are determined from their Commit messages using Senti4SD. It is statistically observed that Commits that introduce, precede or fix bugs are significantly more negative than regular Commits, with a higher proportion of emotional (non-neutral) messages. It is also found that a distinction between buggy and correct fixes exists based on the message's neutrality.

Developer emotion or sentiment in a software development environment has the potential to affect performance, and consequently, the software itself. Sentiment analysis, conducted to analyze online collaborative artifacts, can derive effects of developer sentiment. This study aims to understand how developer sentiment is related to bugs, by analyzing the difference of sentiment between regular and Fix-Inducing Changes (FIC) - changes to code that introduce bugs in the system. To do so, sentiment is extracted from Pull Requests of 6 well known GitHub repositories, which contain both code and contributor discussion. Sentiment is calculated using a tool specializing in the software engineering domain: SentiStrength-SE. Next, FICs are detected from Commits by filtering the ones that fix bugs and tracking the origin of the code these remove. Commits are categorized based on FICs and assigned separate sentiment scores (-4 to +4) based on different preceding artifacts - Commits, Comments and Reviews from Pull Requests. The statistical result shows that FICs, compared to regular Commits, contain more positive Comments and Reviews. Commits that precede an FIC have more negative messages. Similarly, all the Pull Request artifacts combined are more negative for FICs than regular Commits.