The Website Accessibility Conformance Evaluation Methodology (WCAG-EM) directs an auditor to select samples of web pages for accessibility evaluation. However, the sampling method of WCAG-EM cannot statistically claim to represent pages outside of the selected sample. To generate an optimal representative sample, we previously proposed OPTIMAL-EM. In this article, we apply the framework to explore the relationships between two metrics in the framework: web page complexity and accessibility. We first cluster pages by their structure by employing t-distributed Stochastic Neighbour Embedding (t-SNE) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) by representing pages using two approaches: one including all HTML elements and another focusing on block-level HTML elements. We then compare the average complexity of a cluster and the variance of the complexity of a given cluster against the average accessibility of that cluster. To measure their correlations, we conducted a two-stage evaluation: an initial experiment with a website of 388 pages and a validation study with three additional random sites of 500 pages. Our experiments demonstrate that clusters characterised by lower variability in complexity tend to exhibit fewer accessibility barriers, suggesting that standardised and templated web design could be more beneficial to accessibility than more complex pages. Furthermore, the main contribution of this article is showing that by identifying and focusing on clusters with high complexity variance, auditors can target their efforts more effectively, prioritising areas more likely to present accessibility challenges. By providing a more systematic and scalable sampling method, our approach can optimise conformance evaluations of large websites by reducing the human effort involved, focusing on a more targeted assessment of more representative pages.

Traditional methods for selecting web pages for evaluation lack a systematic approach. Web accessibility is crucial to improve equal access and usability for individuals with disabilities. However, current approaches to accessibility evaluation are often time-consuming and resource-intensive. By optimising the accessibility evaluation process, we can more effectively identify and address accessibility issues on the web. This paper addresses the challenge of efficiently evaluating web accessibility across heterogeneous web pages. Drawing inspiration from census studies, we developed a framework to measure population-sourcing methods that leverage modern templated web development processes. This approach allows us to cluster heterogeneous web pages and select representative samples that reflect the entire cluster, streamlining the evaluation process. In this paper, we demonstrate that our clustering method effectively groups web pages with similar accessibility characteristics. Using statistical tooling and incorporating additional features from server log files, we aim to improve the accuracy and efficiency of accessibility audits. Our findings have several implications for web accessibility evaluation: our approach offers a more systematic and objective way of selecting pages for evaluation, reducing the reliance on ad-hoc heuristics. Secondly, by optimising the evaluation process, we envision that organisations can allocate their resources more efficiently, ensuring a broader coverage of web pages while maintaining high accessibility standards. Finally, incorporating server log files as a data source highlights the potential of leveraging existing web analytics data for accessibility assessment, providing additional insights and opportunities for improvement. • Framework for Web Accessibility Evaluation. OPTIMAL-EM enhances WCAG-EM with six metrics. • Comparison of Population Sourcing. Server logs outperform crawling in coverage. • Impact of Complexity and Popularity. Server logs offer a more balanced page sample. • Practical Implications. OPTIMAL-EM may improve audit efficiency and resource use.

Web accessibility evaluation is a costly and complex process due to limited time, resources and ambiguity. To optimise the accessibility evaluation process, we aim to reduce the number of pages auditors must review by employing statistically representative pages, reducing a site of thousands of pages to a manageable review of archetypal pages. Our paper focuses on representativeness, one of six proposed metrics that form our methodology, to address the limitations we have identified with the W3C Website Accessibility Conformance Evaluation Methodology (WCAG-EM). These include the evaluative scope, the non-probabilistic sampling approach, and the potential for bias within the selected sample. Representativeness, in particular, is a metric to assess the quality and coverage of sampling. To measure this, we systematically evaluate five web page representations with a website of 388 pages, including tags, structure, the DOM tree, content, and a mixture of structure and content. Our findings highlight the importance of including structural components in representations. We validate our conclusions using the same methodology for three additional random sites of 500 pages. As an exclusive attribute, we find that features derived from web content are suboptimal and can lead to lower quality and more disparate clustering for optimised accessibility evaluation.