Visual analytics dashboards are widely used for decision-making, yet persistent usability problems, including information overload, graph literacy demands, and limited discoverability of functionality, can limit their effectiveness. Surface-level customisation, such as filtering, view switching, and chart selection, can address these issues, but users often do not engage with these customisations because they perceive them as time-consuming or fail to discover them. We investigate nudging as a lightweight intervention to promote beneficial customisation during task execution. Drawing on evidence that Just-in-Time Prompts (JITP) primarily improve efficiency while Enabling Social Comparison (ESC) improves effectiveness, we evaluate their combination. In a within-subjects study (N = 41) across three dashboards, the combined intervention increased task correctness from 34% to 93% and reduced completion time by 52% compared to an unassisted control. The benefit of guidance is not proportional to its amount: tighter alignment (more pre-configured settings matched to the task) yields the highest absolute performance, while looser alignment (fewer pre-configured settings) delivers greater improvement per unit of help. Overall, transparent, user-controlled nudging can substantially enhance dashboard usability while preserving autonomy.

The Website Accessibility Conformance Evaluation Methodology (WCAG-EM) directs an auditor to select samples of web pages for accessibility evaluation. However, the sampling method of WCAG-EM cannot statistically claim to represent pages outside of the selected sample. To generate an optimal representative sample, we previously proposed OPTIMAL-EM. In this article, we apply the framework to explore the relationships between two metrics in the framework: web page complexity and accessibility. We first cluster pages by their structure by employing t-distributed Stochastic Neighbour Embedding (t-SNE) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) by representing pages using two approaches: one including all HTML elements and another focusing on block-level HTML elements. We then compare the average complexity of a cluster and the variance of the complexity of a given cluster against the average accessibility of that cluster. To measure their correlations, we conducted a two-stage evaluation: an initial experiment with a website of 388 pages and a validation study with three additional random sites of 500 pages. Our experiments demonstrate that clusters characterised by lower variability in complexity tend to exhibit fewer accessibility barriers, suggesting that standardised and templated web design could be more beneficial to accessibility than more complex pages. Furthermore, the main contribution of this article is showing that by identifying and focusing on clusters with high complexity variance, auditors can target their efforts more effectively, prioritising areas more likely to present accessibility challenges. By providing a more systematic and scalable sampling method, our approach can optimise conformance evaluations of large websites by reducing the human effort involved, focusing on a more targeted assessment of more representative pages.

Visual analytics dashboards have become key tools for decision-making. Yet, information overload and high expectations of the actual graph literacy of users hinder their effective use. Customisation features such as global filters (filters that update all views), visualisation hiding, and graph switching have been proposed to improve dashboard usability and accommodate user diversity. However, time constraints and a lack of awareness of the available customisation functionalities are known to be barriers to engaging with customisations. In this paper, we examine how visual analytics dashboard customisation affects usability and identify existing risks and barriers. We found that short-term surface customisations improve the usability of dashboards, both objectively (users are quicker and more correct) and subjectively (users report a lower cognitive load). For this to happen, the engagement with customisations must not be exploratory but meaningful (i.e., longer than 5 seconds). We also identified a set of customisations that, if not implemented carefully, are detrimental to cognitive load and completion times. Informed by our findings, we provide insights for building user models to capture the usability of dashboards-in-use and associated adaptive interaction techniques to assist users when needed.

Visual analytics dashboards have become essential tools for decisionmaking.However, information overload and mismatches between designers' expected graph literacy and users' actual graph literacy can limit their effectiveness.Customisation has been proposed to mitigate these challenges and accommodate diverse user needs.Yet, customising dashboards is often time-consuming; users may not be aware of existing customisation features, or they may not have sufficient technical skills to use them.In this paper, we conduct an experiment (N=50) to examine if we can use nudging techniques to promote short-term surface customisations while not sacrificing the usability of interactive visual analytics dashboards.We found that while nudges do not necessarily increase the use of customisation functionalities, they benefit usability.Specifically, the Social Comparisons nudge supports decision-making, while the Just-in-Time Prompts nudge reduces task completion time.Our findings suggest that nudges should be tailored to graph literacy as users with moderate graph literacy can benefit the most from nudges.

Traditional methods for selecting web pages for evaluation lack a systematic approach. Web accessibility is crucial to improve equal access and usability for individuals with disabilities. However, current approaches to accessibility evaluation are often time-consuming and resource-intensive. By optimising the accessibility evaluation process, we can more effectively identify and address accessibility issues on the web. This paper addresses the challenge of efficiently evaluating web accessibility across heterogeneous web pages. Drawing inspiration from census studies, we developed a framework to measure population-sourcing methods that leverage modern templated web development processes. This approach allows us to cluster heterogeneous web pages and select representative samples that reflect the entire cluster, streamlining the evaluation process. In this paper, we demonstrate that our clustering method effectively groups web pages with similar accessibility characteristics. Using statistical tooling and incorporating additional features from server log files, we aim to improve the accuracy and efficiency of accessibility audits. Our findings have several implications for web accessibility evaluation: our approach offers a more systematic and objective way of selecting pages for evaluation, reducing the reliance on ad-hoc heuristics. Secondly, by optimising the evaluation process, we envision that organisations can allocate their resources more efficiently, ensuring a broader coverage of web pages while maintaining high accessibility standards. Finally, incorporating server log files as a data source highlights the potential of leveraging existing web analytics data for accessibility assessment, providing additional insights and opportunities for improvement. • Framework for Web Accessibility Evaluation. OPTIMAL-EM enhances WCAG-EM with six metrics. • Comparison of Population Sourcing. Server logs outperform crawling in coverage. • Impact of Complexity and Popularity. Server logs offer a more balanced page sample. • Practical Implications. OPTIMAL-EM may improve audit efficiency and resource use.

Parkinson's disease (PD) is a chronic neurological disease that negatively influences patients' quality of life (QoL). Reduced social interactions are a result of both PD symptoms and impairment in QoL, which is known as social withdrawal. Smartphones are suitable for monitoring social behaviour under the notion of digital phenotyping, as people spend significant time socialising on them. We implement year-long longitudinal research using smartphones to study social withdrawal in PD. In addition to an entire year of 24/7 continuous monitoring on smartphones, weekly social ratings and QoL are provided as self-report ground truth. Twenty-three features are extracted from more than 10 million raw data entries collected from eight participants. These features are then used to build models to reflect participants' self-report ratings. We consider the interactions that happen on smartphones, such as calls, messages and social media, and use smartphone sensors to infer face-to-face interactions. By applying multiple linear regression and Naive Bayes, our model is significantly better than just assigning mean and random guesses. The results suggest that our approach provides a more granular method for tracking social behaviour in people with PD via smartphones, and it could benefit wider communities where the social impact on patients needs attention.

Web accessibility evaluation is a costly and complex process due to limited time, resources and ambiguity. To optimise the accessibility evaluation process, we aim to reduce the number of pages auditors must review by employing statistically representative pages, reducing a site of thousands of pages to a manageable review of archetypal pages. Our paper focuses on representativeness, one of six proposed metrics that form our methodology, to address the limitations we have identified with the W3C Website Accessibility Conformance Evaluation Methodology (WCAG-EM). These include the evaluative scope, the non-probabilistic sampling approach, and the potential for bias within the selected sample. Representativeness, in particular, is a metric to assess the quality and coverage of sampling. To measure this, we systematically evaluate five web page representations with a website of 388 pages, including tags, structure, the DOM tree, content, and a mixture of structure and content. Our findings highlight the importance of including structural components in representations. We validate our conclusions using the same methodology for three additional random sites of 500 pages. As an exclusive attribute, we find that features derived from web content are suboptimal and can lead to lower quality and more disparate clustering for optimised accessibility evaluation.

A significant proportion of smartphone notifications are indicative of human behaviour (e.g. delivery updates for purchased items, physical activity summaries, and notification of updates to subscribed content). However, present attempts to understand human behaviour from smartphone traces typically focus on sensors such as location, accelerometer and proximity, overlooking the potential for notifications as a valuable data source. In this paper, we propose a general framework that provides end-to-end processing of notifications to understand behavioural aspects. We realise the framework with an implementation that tackles the specific use case of establishing prior buying behaviour from associated notifications. To evaluate the framework and implementation, we conduct a longitudinal user study in which we collect more than 250, 000 notifications, from twelve users, over an average of three months. We apply knowledge-based and machine learning techniques to those notifications to assess the tasks of the proposed framework. The results show a substantial difference in the performance between the methods used to extract behavioural features from the collected notifications.

Familiarity is a quality of user experience that has traditionally been difficult to define, capture, and quantify. Existing works on measuring familiarity with interactive systems have relied on surveys and self-reporting, which is obtrusive and prone to biases. Here, we propose a data-driven methodology to associate low-level activity patterns with familiarity. As a proof-of-concept, this methodology was tested on a website with 35,819 users over the course of 18 months, including 268 revisiting users who had reported their levels of familiarity with the platform. By using activity patterns as features of predictive models, we were able to successfully classify users with higher levels of familiarity with an accuracy of 82.7%. These results suggest that there is a relationship between user familiarity and activity patterns involving the exploration and use of navigational artefacts including breadcrumbs, navigation bars, and sidebar areas. This research opens up further opportunities for unobtrusively analysing the user experience on the Web.

Web pages are composed of different kinds of elements (menus, adverts, etc.). Segmenting pages into their elements has long been important in understanding how people experience those pages and in making those experiences “better.” Many approaches have been proposed that relate the resultant elements with the underlying source code; however, they do not consider users’ interactions. Another group of approaches analyses eye movements of users to discover areas that interest or attract them (i.e., areas of interest or AOIs). Although these approaches consider how users interact with web pages, they do not relate AOIs with the underlying source code. We propose a novel approach that integrates web page and eye tracking data driven approaches for automatic AOI detection. This approach segments an entire web page into its AOIs by considering users’ interactions and relates AOIs with the underlying source code. Based on the Adjusted Rand Index measure, our approach provides the most similar segmentation to the ground-truth segmentation compared to its individual components.

Parkinson's disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. Measuring the symptoms and progress of the disease, and medication effectiveness is currently performed using subjective measures and visual estimation. We developed and evaluated a mobile application, STOP for tracking hand's motor symptoms, and a medication journal for recording medication intake. We followed 13 PD patients from two countries for a 1-month long real-world deployment. We found that PD patients are willing to use digital tools, such as STOP, to track their medication intake and symptoms, and are also willing to share such data with their caregivers and medical personnel to improve their own care.

People with autism often exhibit different visual behaviours from neurotypical users. To explore how these differences are exhibited on the Web, we model visual behaviour by combining pupillary response, which is an unobtrusive measure of physiological arousal, with eye-tracking scan paths that indicate visual attention. We evaluated our approach with two populations: 19 neurotypical users and 19 users with autism. We observe differences in their visual behaviours as, in certain instances, individuals with autism exhibit a lower arousal response to affective contents. While this is consistent with the literature on autism, we confirm this phenomenon on the Web. We discuss how our modelling method can be used to identify possible UX issues such as the presence of stress, cognitive load and differences in the perception of Web elements in relation to physiological arousal.

Anecdotal evidence suggests that people with autism may have different processing strategies when accessing the web. However, limited empirical evidence is available to support this. This paper presents an eye tracking study with 18 participants with high-functioning autism and 18 neurotypical participants to investigate the similarities and differences between these two groups in terms of how they search for information within web pages. According to our analysis, people with autism are likely to be less successful in completing their searching tasks. They also have a tendency to look at more elements on web pages and make more transitions between the elements in comparison to neurotypical people. In addition, they tend to make shorter but more frequent fixations on elements which are not directly related to a given search task. Therefore, this paper presents the first empirical study to investigate how people with autism differ from neurotypical people when they search for information within web pages based on an in-depth statistical analysis of their gaze patterns.

The use of mobile devices during television viewing is now commonplace, and broadcasters are increasingly supplying programme-related ‘companion content’. To produce an optimal user experience, it is important to determine how the delivery of companion content affects and is perceived by the viewer — without this we risk distracting the viewer, leading to frustration and disengagement. We present a controlled study investigating how attention, cognitive load (as measured by the NASA TLX), and users’ preferences are affected by the provision of two different content delivery modes: pushed and pulled . We find that delivery mode affected the temporal distribution of gaze to the tablet, with a consistent viewing pattern for pushed updates, which attracted attention within a few seconds, and a more diverse set of viewing patterns when updates were pulled. Cognitive load was similar in both conditions, and there was no consensus as to which mode was preferred, but users showed strong, polarised individual preferences. The advantages of each delivery mode are presented as a set of recommendations for the delivery of companion content.

OBJECTIVE: The study sought to explore to what extent geolocation data has been used to study serious mental illness (SMI). SMIs such as bipolar disorder and schizophrenia are characterized by fluctuating symptoms and sudden relapse. Currently, monitoring of people with an SMI is largely done through face-to-face visits. Smartphone-based geolocation sensors create opportunities for continuous monitoring and early intervention. MATERIALS AND METHODS: We searched MEDLINE, PsycINFO, and Scopus by combining terms related to geolocation and smartphones with SMI concepts. Study selection and data extraction were done in duplicate. RESULTS: Eighteen publications describing 16 studies were included in our review. Eleven studies focused on bipolar disorder. Common geolocation-derived digital biomarkers were number of locations visited (n = 8), distance traveled (n = 8), time spent at prespecified locations (n = 7), and number of changes in GSM (Global System for Mobile communications) cell (n = 4). Twelve of 14 publications evaluating clinical aspects found an association between geolocation-derived digital biomarker and SMI concepts, especially mood. Geolocation-derived digital biomarkers were more strongly associated with SMI concepts than other information (eg, accelerometer data, smartphone activity, self-reported symptoms). However, small sample sizes and short follow-up warrant cautious interpretation of these findings: of all included studies, 7 had a sample of fewer than 10 patients and 11 had a duration shorter than 12 weeks. CONCLUSIONS: The growing body of evidence for the association between SMI concepts and geolocation-derived digital biomarkers shows potential for this instrument to be used for continuous monitoring of patients in their everyday lives, but there is a need for larger studies with longer follow-up times.

We report the process used to create artefacts for self-reporting Parkinson's Disease symptoms. Our premise was that a technology-based approach would provide participants with an effective, flexible, and resilient technique. After testing four prototypes using Bluetooth, NFC, and a microcontroller we accomplished almost full compliance and high acceptance using a paper diary to track day-to-day fluctuations over 49 days. This diary is tailored to each patient's condition, does not require any handwriting, allows for implicit reminders, provides recording flexibility, and its answers can be encoded automatically. We share five design implications for future Parkinson's self-reporting artefacts: reduce participant completion demand, design to offset the effect of tremor on input, enable implicit reminders, design for positive and negative consequences of increased awareness of symptoms, and consider the effects of handwritten notes in compliance, encoding burden, and data quality.

There are many mechanisms to sense arousal. Most of them are either intrusive, prone to bias, costly, require skills to set-up or do not provide additional context to the user's measure of arousal. We present arousal detection through the analysis of pupillary response from eye trackers. Using eye-trackers, the user's focal attention can be detected with high fidelity during user interaction in an unobtrusive manner. To evaluate this, we displayed twelve images of varying arousal levels rated by the International Affective Picture System (IAPS) to 41 participants while they reported their arousal levels. We found a moderate correlation between the self-reported arousal and the algorithm's arousal rating, r(47)=0.46, p<.01. The results show that eye trackers can serve as a multi-sensory device for measuring arousal, and relate the level of arousal to the user's focal attention. We anticipate that in the future, high fidelity web cameras can be used to detect arousal in relation to user attention, to improve usability, UX and understand visual behaviour.

Parkinson's disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. It is commonly accepted that improving medication adherence alleviates symptoms and maintains motor capabilities. Not following the medication regimen (e.g., skipping or over-medicating) may worsen side-effects, which mislead clinicians and patients. We developed and evaluated a mobile application, STOP, for screening the PD symptoms and medication intake. It contains a game for tracking the PD symptoms, and a medication journal for recording medical intake and adherence. We conducted a 1-month long real-world deployment with 13 PD patients from two countries. We found that the application medication adherence tracking provides non-bias information, and users are receptive to share such data with their care and medical personnel.

Parkinson's Disease impairs the motor, cognitive and emotional functioning of people. Clinicians do not get an accurate image of the disease because patients visit them every six months and their symptoms can change within hours. Technology has been used to tackle this problem, but most approaches disrupt people's routines or are uncomfortable to use. We aim to monitor Parkinson's in an unobtrusive, longitudinal and naturalistic way based on digital biomarkers inferred from smartphone-collected heterogeneous data. We use Parkinson's clinical scales and self-reporting of symptoms as ground truth to evaluate our methodology. Here, we present three insights we gained after tracking four people 24/7 for four months: a) the monitoring smartphone needs to be people's primary device, b) participants prefer a paper diary for symptom self-reporting, and c) social and phone interaction will be explored as digital biomarkers. We expect this approach to improve patient's quality of life and the efficiency of health services.

The number of users required for usability studies has been a controversial issue over 30 years. Some researchers suggest a certain number of users to be included in these studies. However, they do not focus on eye tracking studies for analysing eye movement sequences of users (i.e., scanpaths) on web pages. We investigate the effects of the number of users on scanpath analysis with our algorithm that was designed for identifying the most commonly followed path by multiple users. Our experimental results suggest that it is possible to approximate the same results with a smaller number of users. The results also suggest that more users are required when they serendipitously browse on web pages in comparison with when they search for specific information or items. We observed that we could achieve 75% similarity to the results of 65 users with 27 users for searching tasks and 34 users for browsing tasks. This study guides researchers to determine the ideal number of users for analysing scanpaths on web pages based on their budget and time.

We present an experiment comparing performance of 20 novice evaluators of accessibility carrying out Web Content Accessibility Guidelines 2.0 conformance reviews working individually to performance obtained when they work in teams of two. They were asked to first carry out an individual assessment of a web page. Later on, they were matched randomly to constitute a group of two and they were asked to revise their initial assessment and to produce a group assessment of the same page. Results indicate that significant differences were found for sensitivity (inversely related to false negatives: +8%) and agreement (when measured in terms of the majority view: +10%). Members of groups exhibited strong agreement on the evaluation results among them and with the group outcome. Other measures of validity and reliability are not significantly affected by group work. Practical implications of these findings are that, for example, when it is important to reduce the false-negative rate, then employing a group of two people is more useful than having individuals carrying out the assessment. Openings for future research include further explorations of whether similar results hold for groups larger than two or what is the effect of mixing people with different accessibility background. When novice accessibility evaluators work in groups, their ability to identify all the true problems increases (by 8%). Likewise, reliability of group evaluations increases (by 10%). Individual or group evaluations can be considered as equivalent methods with respect to false positives (if differences up to 8% in correctness are tolerated). Individual or group evaluations can be considered as equivalent methods with respect to overall effectiveness (if differences up to 11% in F -measure are tolerated).

Eye tracking studies have widely been used in improving the design and usability of web pages and in the research of understanding how users navigate them. However, there is limited research in clustering users’ eye movement sequences (i.e., scanpaths) on web pages to identify a general direction they follow. Existing research tends to be reductionist, which means that the resulting path is so short that it is not useful. Moreover, there is little work on correlating users’ scanpaths with visual elements of web pages and the underlying source code, which means the result cannot be used for further processing. In order to address these limitations, we introduce a new concept in clustering scanpaths called Scanpath Trend Analysis (STA) that not only considers the visual elements visited by all users, but also considers the visual elements visited by the majority in any order. We present an algorithm which automatically does this trend analysis to identify a trending scanpath for multiple web users in terms of visual elements of a web page. In contrast to existing research, the STA algorithm first analyzes the most visited visual elements in given scanpaths, clusters the scanpaths by arranging these visual elements based on their overall positions in the individual scanpaths, and then constructs a trending scanpath in terms of these visual elements. This algorithm was experimentally evaluated by an eye tracking study on six web pages for two different kinds of tasks (12 cases in total). Our experimental results show that the STA algorithm generates a trending scanpath that addresses the reductionist problem of existing work by preventing the loss of commonly visited visual elements for all cases. Based on the statistical tests, the STA algorithm also generates a trending scanpath that is significantly more similar to the inputted scanpaths compared to other existing work in 10 out of 12 cases. In the remaining cases, the STA algorithm still performs significantly better than some other existing work. This algorithm contributes to behavior analysis research on the web that can be used for different purposes: for example, re-engineering web pages guided by the trending scanpath to improve users’ experience or guiding designers to improve their design.

The equality of access – accessibility – is difficult to quantify, define, or agree upon. Our previous work analysed the responses of web accessibility specialists in regard to a number of pre-defined definitions of accessibility. While uncovering much, this analysis did not allow us to quantify the communities’ understanding of the relationship accessibility has with other domains and assess how the community scopes accessibility. In this case, we asked over 300 people, with an interest in accessibility, to answer 33 questions surrounding the relationship between accessibility, user experience (UX), and usability; inclusion and exclusion; and evaluation, in an attempt to harmonise our understanding of web accessibility. We found that respondents think that accessibility and usability are highly related and also think that accessibility is applicable to everyone and not just people with disabilities. Respondents strongly agree that accessibility must be grounded on user-centred practices and that accessibility evaluation is more than just inspecting source code; however, they are divided as to whether training in ‘Web Content Accessibility Guidelines’ is necessary or not to assess accessibility. These perceptions are important for usability and UX professionals, developers of automated evaluation tools, and those practitioners running website evaluations.

The perception of the visual complexity of World Wide Web (Web) pages is a topic of significant interest. Previous work has examined the relationship between complexity and various aspects of presentation, including font styles, colours and images, but automatically quantifying this dimension of a web page at the level of the document remains a challenge. In this paper we demonstrate that areas of high complexity can be identified by detecting areas, or ‘chunks’, of a web page high in block-level elements. We report a computational algorithm that captures this metric and places web pages in a sequence that shows an 86% correlation with the sequences generated through user judgements of complexity. The work shows that structural aspects of a web page influence how complex a user perceives it to be, and presents a straightforward means of determining complexity through examining the DOM.

Dynamic micro-content—interactive or updating widgets and features—is now widely used on the Web, but there is little understanding of how people allocate attention to it. In this article we present the results of an eye-tracking investigation examining how the nature of dynamic micro-content influences whether or not the user views it. We propose and validate the Dynamic Update Viewing-likelihood (DUV) model, a CHi-squared Automatic Interaction Detector (CHAID) model that predicts with around 80% accuracy whether users view dynamic updates as a function of how they are initiated, their size, and their duration. The model is constructed with data from live Web sites and does not rely on knowledge of the user's task to make its predictions, giving it a high level of external validity. We discuss one example of its application: informing how dynamic content should be presented in audio via assistive technology for people with visual impairments.

Visual aesthetics enhances user experience in the context of the World Wide Web (Web). Accordingly, many studies report positive relationships between Web aesthetics and facets of user experience like usability and credibility, but does this hold for accessibility also? This paper describes an empirical investigation towards this end. The aesthetic judgements of 30 sighted Web users were elicited to understand what types of Web design come across as being visually pleasing. Participants judged 50 homepages based on Lavie and Tractinsky's classical and expressive Web aesthetics framework. A cross-section of the homepages were then manually audited for accessibility compliance by 11 Web accessibility experts who used a heuristic evaluation technique known as the Barrier Walkthrough (BW) method to check for accessibility barriers that could affect people with visual impairments. Web pages judged on the classical dimension as being visually clean showed significant correlations with accessibility, suggesting that visual cleanness may be a suitable proxy measure for accessibility as far as people with visual impairments are concerned. Expressive designs and other aesthetic dimensions showed no such correlation, however, demonstrating that an expressive or aesthetically pleasing Web design is not a barrier to accessibility.

Previous studies have defined a new type of impairment in which an able-bodied user's behaviour is impaired by both the characteristics of a device and the environment in which it is used. This behavioural change is defined as a situationally-induced impairment and is often associated with small devices used in a mobile setting or constrained environment. Relatively little is known of its extent or the magnitude of the problems faced by the user. Here, we place the work that has been undertaken into a comparative framework of impairment in the context of data input. Our survey finds that 12 problems exist which define all impaired input and we look to understand the similarities and possible solutions in the new impairment group of small device users. For instance, from the literature, we find that users who are both situationally and motor impaired: press keys more than once, do not use enough force and so a key is not activated, and accidentally type keys in reverse order. A wealth of other errors exist in the motor impaired domain but work does not exist to support their existence in the situational domain. However, if commonalities exist for some – it seems work should be enacted to uncover the commonalities of all. Establishing these possible commonalities is important because solutions from one domain can be leveraged into another thereby saving the need to reinvent interventions which already exist elsewhere.

Web accessibility means that disabled people can effectively perceive, understand, navigate, and interact with the web. Web accessibility evaluation methods are needed to validate the accessibility of web pages. However, the role of subjectivity and of expertise in such methods is unknown and has not previously been studied. This article investigates the effect of expertise in web accessibility evaluation methods by conducting a Barrier Walkthrough (BW) study with 19 expert and 57 nonexpert judges. The BW method is an evaluation method that can be used to manually assess the accessibility of web pages for different user groups such as motor impaired, low vision, blind, and mobile users. Our results show that expertise matters, and even though the effect of expertise varies depending on the metric used to measure quality, the level of expertise is an important factor in the quality of accessibility evaluation of web pages. In brief, when pages are evaluated with nonexperts, we observe a drop in validity and reliability. We also observe a negative monotonic relationship between number of judges and reproducibility: more evaluators mean more diverse outputs. After five experts, reproducibility stabilizes, but this is not the case with nonexperts. The ability to detect all the problems increases with the number of judges: With 3 experts all problems can be found, but for such a level 14 nonexperts are needed. Even though our data show that experts rated pages differently, the difference is quite small. Finally, compared to nonexperts, experts spent much less time and the variability among them is smaller, they were significantly more confident, and they rated themselves as being more productive. The article discusses practical implications regarding how BW results should be interpreted, how to recruit evaluators, and what happens when more than one evaluator is hired. Supplemental materials are available for this article. Go to the publisher's online edition of Human-Computer Interaction for statistical details and additional measures for this article. © 2011 Taylor & Francis Group, LLC.

World Wide Web accessibility and best practice audits and evaluations are becoming increasingly complicated, time consuming, and costly because of the increasing number of conformance criteria which need to be tested. In the case of web access by disabled users and mobile users, a number of commonalities have been identified in usage, which have been termed situationally-induced impairments; in effect the barriers experienced by mobile web users have been likened to those of visually disabled and motor impaired users. In this case, we became interested in understanding if it was possible to evaluate the problems of mobile web users in terms of the aggregation of barriers-to-access experienced by disabled users; and in this way attempt to reduce the need for the evaluation of the additional conformance criteria associated with mobile web best practice guidelines. We used the Barrier Walkthrough (BW) method as our analytical framework. Capable of being used to evaluate accessibility in both the disabled and mobile contexts, the BW method would also enable testing and aggregation of barriers across our target user groups. We tested 61 barriers across four user groups each over four pages with 19 experts and 57 non-experts focusing on the validity and reliability of our results. We found that 58% of the barrier types that were correctly found were identified as common between mobile and disabled users. Further, if our aggregated barriers alone were used to test for mobile conformance only four barrier types would be missed. Our results also showed that mobile users and low vision users have the most common barrier types, while low vision and motor impaired users experiencing similar rates of severity in the barriers they experienced. We conclude that the aggregated evaluation results for blind, low vision and motor impaired users can be used to approximate the evaluation results for mobile web users. © 2011 British Informatics Society Limited. Published by Elsevier B.V. All rights reserved.

Web Content Accessibility Guidelines 2.0 (WCAG 2.0) require that success criteria be tested by human inspection. Further, testability of WCAG 2.0 criteria is achieved if 80% of knowledgeable inspectors agree that the criteria has been met or not. In this paper we investigate the very core WCAG 2.0, being their ability to determine web content accessibility conformance. We conducted an empirical study to ascertain the testability of WCAG 2.0 success criteria when experts and non-experts evaluated four relatively complex web pages; and the differences between the two. Further, we discuss the validity of the evaluations generated by these inspectors and look at the differences in validity due to expertise.

Manual accessibility evaluation plays an important role in validating the accessibility of Web pages. This role has become increasingly critical with the advent of the Web Content Accessibility Guidelines (WCAG) 2.0 and their reliance on user evaluation to validate certain conformance measures. However, the role of expertise, in such evaluations, is unknown and has not previously been studied. This paper sets out to investigate the interplay between expert and non-expert evaluation by conducting a Barrier Walkthrough (BW) study with 19 expert and 51 non-expert judges. The BW method provides an evaluation framework that can be used to manually assess the accessibility of Web pages for different user groups including motor impaired, hearing impaired, low vision, cognitive impaired, etc. We conclude that the level of expertise is an important factor in the quality of accessibility evaluation of Web pages. Expert judges spent significantly less time than non-experts; rated themselves as more productive and confident than non-experts; and ranked and rated pages differently against each type of disability. Finally, both effectiveness and reliability of the expert judges are significantly higher than non-expert judges.

The visual complexity of Web pages is much talked about; “complex Web pages are difficult to use,” but often regarded as a subjective decision by the user. This subjective decision is of limited use if we wish to understand the importance of visual complexity, what it means, and how it can be used. We theorize that by understanding a user's visual perception of Web page complexity, we can understand the cognitive effort required for interaction with that page. This is important because by using an easily identifiable measure, such as visual complexity, as an implicit marker of cognitive load, we can design Web pages which are easier to interact with. We have devised an initial empirical experiment, using card sorting and triadic elicitation, to test our theories and assumptions, and have built an initial baseline sequence of 20 Web pages along with a library of qualitative and anecdotal feedback. Using this library, we define visual complexity, ergo perceived interaction complexity, and by taking these pages as “prototypes” and ranking them into a sequence of complexity, we are able to group them into: simple, neutral, and complex. This means we can now work toward a definition of visual complexity as an implicit measure of cognitive load.

The World Wide Web (Web) is a visually complex, dynamic, multimedia system that can be inaccessible to people with visual impairments. SADIe addresses this problem by using Semantic Web technologies to explicate implicit visual structures through a combination of an upper and lower ontology. This is then used to apply transcoding to a range of Websites. This paper describes a user evaluation that was performed using the SADIe system. Four users were presented with a series of Web pages, some having been adapted using SADIe's transcoding functionality and others retaining in their original state. The results of the evaluation showed that providing answers to a fact based question could be achieved more quickly when the information on the page was exposed via SADIe's transcoding. The data obtained during the experiment was analysed and shown to be statistically significant. This suggests that the transcoding techniques offered by SADIe can assist visually impaired users accessing content on the Web.

The rapid advancement of World Wide Web (Web) technology and constant need for attractive Websites produce pages that hinder visually impaired users. We assert that understanding how sighted users browse Web pages can provide important information that will enhance Web Accessibility, especially for visually impaired users. We present an eye tracking study where sighted users' browsing behaviour on nine Web pages was investigated to determine how the page's visual clutter is related to sighted users' browsing patterns. The results show that salient elements attract users' attention first, users spend more time on the main content of the page and users tend to fixate on the first three or four items on the menu lists. Common gaze patterns begin at the salient elements of the page, move to the main content, header, right column and left column of the page and finish at the footer area. We argue that the results should be used as the initial step for proposing guidelines that assist in designing and transforming Web pages for an easier and faster access for visually impaired users.

This paper presents an eye-tracking study that examines how people use the visual elements of Web pages to complete certain tasks. Whilst these elements are available to play their role in these tasks for sighted users, it is not the case for visually disabled users. This lack of access to some visual elements of a page means that visually disabled users are hindered in accomplishing these tasks. Our previous work has introduced a framework that identifies these elements and then reengineers Web pages such that these elements can play their intended roles in an audio, as well as visual presentation. To further improve our understanding of how these elements are used and to validate our framework, we track the eye movements of sighted users performing a number of different tasks. The resulting gaze data show that there is a strong relationship between the aspects of a page that receive visual attention and the objects identified by our framework. The study also shows some limitations, as well as yielding information to address these short-comings. Perhaps the most important result is the support provided for a particular kind of object called a Way Edge - the visual construct used to group content into sections. There is a significant effect of Way Edges on the distribution of attention across tasks. This is a result that not only provides strong evidence for the utility of re-engineering, but also has consequences for our understanding of how people allocate attention to different parts of a page. We speculate that the phenomenon of 'Banner Blindness' owes as much to Way Edges, as it does to colour and font size.

Visually impaired users are hindered in their efforts to access the largest repository of electronic information in the world, namely, the World Wide Web (web). A visually impaired user's information and presentation requirements are different from a sighted user's. These requirements can become problems in that the web is visually centric with regard to presentation and information order/layout. Finding semantic information already encoded directly into documents can help to alleviate these problems. Our approach can be loosely described as follows. For a particular cascading stylesheet (CSS), we provide an extension to an upper-level ontology which represents the interface between web documents and the programmatic transformation mechanism. This extension gives the particular characteristics of the elements appearing in that specific CSS. We can consider this extension to be an annotation of the CSS elements implicitly encoded into the web document. This means that one ontology can be used to accuratly transform every web document that references the CSS used to generate that ontology. Simply one ontology accuratly transforms an entire site using a generalized programmatic machinery able to cope with all sites using CSS. Here we describe our method, implementation, and technical evaluation.

The importance of the World Wide Web for information dissemination is indisputable. However, the dominance of visual design on the Web leaves visually disabled people at a disadvantage. Although assistive technologies, such as screen readers, usually provide basic access to information, the richness of the Web experience is still often lost. In particular, traversing the Web becomes a complicated task since the richness of visual objects presented to their sighted counterparts are neither appropriate nor accessible to visually disabled users. To address this problem, we have proposed an approach called Dante in which Web pages are annotated with semantic information to make their traversal properties explicit. Dante supports usage of different annotation techniques and as a proof-of-concept in this article, pages are annotated manually which when transcoded become rich. We first introduce Dante and then present a user evaluation which compares how visually disabled users perform certain travel-related tasks on original and transcoded versions of Web pages. We discuss the evaluation methodology in detail and present our findings, which provide useful insights into the transcoding process. Our evaluation shows that, in tests with users, document objects transcoded with Dante have a tendency to be much easier for visually disabled users to interact with when traversing Web pages.

Visually impaired users are hindered in their efforts to access the World Wide Web (Web) because their information and presentation requirements are different from those of a sighted user. These requirements can become problems as the Web becomes ever more visually centric with regard to presentation and information order / layout, this can (and does) hinder users who need presentationagnostic access to information. Finding semantic information already encoded directly into pages can help to alleviate these problems and support users who wish to understand the meaning as opposed to the presentation and order of the information. Our solution, Structural-Semantics for Accessibility and Device Independence (SADIe) involves building ontologies of Cascading Sytle-Sheets (CSS) and using those ontologies to transform Web pages.

Anecdotal evidence suggests that Web document summaries provide the sighted reader with a basis for making decisions regarding the route to take within non-linear text; and additional research shows that sighted people use 'Gist' summaries as decision points to bolster their browsing behaviour. Other studies have found that visually impaired users are hindered in their cognition of the content of Web-pages because users must wait for an entire Web-page to be read before deciding on it's usefulness to their current task. In these cases, we draw similarities between sighted and visually impaired users, in that sighted users cannot see the target of a Web Anchor and are therefore 'handicapped'1 by the technology. Previously, we have investigate four simple summarisation algorithms against each other and a manually created summary; producing empirical evidence as a formative evaluation. This evaluation concludes that users prefer simple automatically generated 'gist' summaries thereby reducing cognitive overload and increasing awareness of the focus of the Web-page under investigation. In this paper we focus on the development of 'FireFox' based tool which creates a summary of a Web page 'on-the-fly'. The algorithm used to create this summary is based on the results of our formative evaluation which automatically and dynamically annotates Web pages with the generated 'gist' summary. In this way visually impaired users are supported in their decisions as the relevancy of the page at hand.

Currently, the vast majority of web sites do not support accessibility for visually impaired users. Usually, these users have to rely on screen readers: applications that sequentially read the content of a web page in audio. Unfortunately, screen readers are not able to detect the meaning of the different page objects, and thus the implicit semantic knowledge conveyed in the presentation of the page is lost. One approach described in literature to tackle this problem, is the Dante approach, which allows semantic annotation of web pages to provide screen readers with extra (semantic) knowledge to better facilitate the audio presentation of a web page. Until now, such annotations were done manually, and failed for dynamic pages. In this paper, we combine the Dante approach with a web design method, WSDM, to fully automate the generation of the semantic annotation for visually impaired users. To do so, the semantic knowledge gathered during the design process is exploited, and the annotations are generated as a by-product of the design process, requiring no extra effort from the designer.

Movement, or mobility, is key to the accessibility, design, and usability of many hypermedia resources (websites); and key to good mobility is context and preview by probing. This is especially the case for visually impaired users when a hypertext anchor is inaccurately described or is described out of context. This means confusion and disorientation. Mobility is similarly reduced when the link target of the anchor has no relationship to the expected information present on the hypertext node (web-page). We suggest that confident movement with purpose, ease, and accuracy can only be achieved when complete contextual information and an accurate description of the proposed destination (preview) are available. Our past work (1) deriving mobility heuristics from mobility models, (2) transforming web-pages based on these heuristics, and(3) building tools to analyse and access these transformed pages; has shown us that a tool to expand context and preview would be useful. In this paper we describe the development of such a middleware tool to automatically and dynamically annotate web-pages with additional context information present within the page, and preview information present within hypertext link destinations found on the page.

Movement, or mobility, is a key to the accessibility, design, and usability of many Web sites. While some peripheral mobility issues have been addressed few have centered on the mobility problems of visually impaired users. We use our past work to address these issues and derive mobility heuristics from mobility models, use these heuristics to place mobility objects within a Web page, and describe the construction of a prototype mobility instrument, in the form of a Netscape plug-in, to process these objects. Our past work extends the notion of movement to include environment, feedback and the purpose of the current travel task. Specifically, we likened web use to travelling in a virtual space, compared it to travelling in a physical space, and introduced the idea of mobility - the ease of travel - as opposed to travel opportunity.