Visual analytics dashboards are widely used for decision-making, yet persistent usability problems, including information overload, graph literacy demands, and limited discoverability of functionality, can limit their effectiveness. Surface-level customisation, such as filtering, view switching, and chart selection, can address these issues, but users often do not engage with these customisations because they perceive them as time-consuming or fail to discover them. We investigate nudging as a lightweight intervention to promote beneficial customisation during task execution. Drawing on evidence that Just-in-Time Prompts (JITP) primarily improve efficiency while Enabling Social Comparison (ESC) improves effectiveness, we evaluate their combination. In a within-subjects study (N = 41) across three dashboards, the combined intervention increased task correctness from 34% to 93% and reduced completion time by 52% compared to an unassisted control. The benefit of guidance is not proportional to its amount: tighter alignment (more pre-configured settings matched to the task) yields the highest absolute performance, while looser alignment (fewer pre-configured settings) delivers greater improvement per unit of help. Overall, transparent, user-controlled nudging can substantially enhance dashboard usability while preserving autonomy.

The Website Accessibility Conformance Evaluation Methodology (WCAG-EM) directs an auditor to select samples of web pages for accessibility evaluation. However, the sampling method of WCAG-EM cannot statistically claim to represent pages outside of the selected sample. To generate an optimal representative sample, we previously proposed OPTIMAL-EM. In this article, we apply the framework to explore the relationships between two metrics in the framework: web page complexity and accessibility. We first cluster pages by their structure by employing t-distributed Stochastic Neighbour Embedding (t-SNE) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) by representing pages using two approaches: one including all HTML elements and another focusing on block-level HTML elements. We then compare the average complexity of a cluster and the variance of the complexity of a given cluster against the average accessibility of that cluster. To measure their correlations, we conducted a two-stage evaluation: an initial experiment with a website of 388 pages and a validation study with three additional random sites of 500 pages. Our experiments demonstrate that clusters characterised by lower variability in complexity tend to exhibit fewer accessibility barriers, suggesting that standardised and templated web design could be more beneficial to accessibility than more complex pages. Furthermore, the main contribution of this article is showing that by identifying and focusing on clusters with high complexity variance, auditors can target their efforts more effectively, prioritising areas more likely to present accessibility challenges. By providing a more systematic and scalable sampling method, our approach can optimise conformance evaluations of large websites by reducing the human effort involved, focusing on a more targeted assessment of more representative pages.

Visual analytics dashboards have become key tools for decision-making. Yet, information overload and high expectations of the actual graph literacy of users hinder their effective use. Customisation features such as global filters (filters that update all views), visualisation hiding, and graph switching have been proposed to improve dashboard usability and accommodate user diversity. However, time constraints and a lack of awareness of the available customisation functionalities are known to be barriers to engaging with customisations. In this paper, we examine how visual analytics dashboard customisation affects usability and identify existing risks and barriers. We found that short-term surface customisations improve the usability of dashboards, both objectively (users are quicker and more correct) and subjectively (users report a lower cognitive load). For this to happen, the engagement with customisations must not be exploratory but meaningful (i.e., longer than 5 seconds). We also identified a set of customisations that, if not implemented carefully, are detrimental to cognitive load and completion times. Informed by our findings, we provide insights for building user models to capture the usability of dashboards-in-use and associated adaptive interaction techniques to assist users when needed.

Social media platforms have transformed the way individuals document and revisit personal experiences. We conducted a study of Instagram, one of the most popular mobile social media platforms (about 95 million images are uploaded daily) to investigates the impact of reviewing Instagram posts on episodic memory—the ability to recall specific events and details. Fifty Instagram users participated in a questionnaire-based study, where they were prompted to recall memories associated with three of their posted images across six prompts: story (the story of the event), location, time, people, before/after events, and order. Results revealed significant variations in recall accuracy: participants were better at recalling the story of the events and associated people compared to the temporal details (the timing) of the event (H = 157.04, p < 0.000). A weak but statistically significant correlation (r = 0.66, p < 0.001) was observed between the number of likes a post received and participants’ ability to recall event details. Higher social engagement thus appears to enhance recall accuracy, highlighting the role of social reinforcement in memory processes. To our knowledge, this study is the first to examine the effect of the non-ephemeral platform "Instagram" use on episodic memory. While the study offers promising evidence of Instagram’s cognitive benefits, further research is needed to explore long-term implications and the interplay between social engagement and episodic memory.

Visual analytics dashboards have become essential tools for decisionmaking.However, information overload and mismatches between designers' expected graph literacy and users' actual graph literacy can limit their effectiveness.Customisation has been proposed to mitigate these challenges and accommodate diverse user needs.Yet, customising dashboards is often time-consuming; users may not be aware of existing customisation features, or they may not have sufficient technical skills to use them.In this paper, we conduct an experiment (N=50) to examine if we can use nudging techniques to promote short-term surface customisations while not sacrificing the usability of interactive visual analytics dashboards.We found that while nudges do not necessarily increase the use of customisation functionalities, they benefit usability.Specifically, the Social Comparisons nudge supports decision-making, while the Just-in-Time Prompts nudge reduces task completion time.Our findings suggest that nudges should be tailored to graph literacy as users with moderate graph literacy can benefit the most from nudges.

Traditional methods for selecting web pages for evaluation lack a systematic approach. Web accessibility is crucial to improve equal access and usability for individuals with disabilities. However, current approaches to accessibility evaluation are often time-consuming and resource-intensive. By optimising the accessibility evaluation process, we can more effectively identify and address accessibility issues on the web. This paper addresses the challenge of efficiently evaluating web accessibility across heterogeneous web pages. Drawing inspiration from census studies, we developed a framework to measure population-sourcing methods that leverage modern templated web development processes. This approach allows us to cluster heterogeneous web pages and select representative samples that reflect the entire cluster, streamlining the evaluation process. In this paper, we demonstrate that our clustering method effectively groups web pages with similar accessibility characteristics. Using statistical tooling and incorporating additional features from server log files, we aim to improve the accuracy and efficiency of accessibility audits. Our findings have several implications for web accessibility evaluation: our approach offers a more systematic and objective way of selecting pages for evaluation, reducing the reliance on ad-hoc heuristics. Secondly, by optimising the evaluation process, we envision that organisations can allocate their resources more efficiently, ensuring a broader coverage of web pages while maintaining high accessibility standards. Finally, incorporating server log files as a data source highlights the potential of leveraging existing web analytics data for accessibility assessment, providing additional insights and opportunities for improvement. • Framework for Web Accessibility Evaluation. OPTIMAL-EM enhances WCAG-EM with six metrics. • Comparison of Population Sourcing. Server logs outperform crawling in coverage. • Impact of Complexity and Popularity. Server logs offer a more balanced page sample. • Practical Implications. OPTIMAL-EM may improve audit efficiency and resource use.

Information presentation problems on interactive dashboards are known to hinder decision-making. Since a traditional user-centred approach to designing usable dashboards cannot fully satisfy user demands, needs and skills, we isolate behavioural indicators of usability when users conduct typical information-seeking and comparison tasks. In a first study (N = 50), we identified strategies derived from 486,435 interaction events logged in a controlled setting with synthetic dashboards. User models consisting of these user strategies and graph literacy produced strong signals indicating that usability was predictable. In a second study (N = 65), we tested the initial insights on real-world dashboards. While most of our hypotheses were confirmed, graph literacy emerged as the best predictor of usability. Usability was better predicted in dashboards with problems, suggesting promising opportunities for automated usability evaluation and real-time support for users struggling with visual analytics dashboards.

Web accessibility evaluation is a costly and complex process due to limited time, resources and ambiguity. To optimise the accessibility evaluation process, we aim to reduce the number of pages auditors must review by employing statistically representative pages, reducing a site of thousands of pages to a manageable review of archetypal pages. Our paper focuses on representativeness, one of six proposed metrics that form our methodology, to address the limitations we have identified with the W3C Website Accessibility Conformance Evaluation Methodology (WCAG-EM). These include the evaluative scope, the non-probabilistic sampling approach, and the potential for bias within the selected sample. Representativeness, in particular, is a metric to assess the quality and coverage of sampling. To measure this, we systematically evaluate five web page representations with a website of 388 pages, including tags, structure, the DOM tree, content, and a mixture of structure and content. Our findings highlight the importance of including structural components in representations. We validate our conclusions using the same methodology for three additional random sites of 500 pages. As an exclusive attribute, we find that features derived from web content are suboptimal and can lead to lower quality and more disparate clustering for optimised accessibility evaluation.

This paper contributes with a pragmatic evaluation framework for explainable Machine Learning (ML) models for clinical decision support. The study revealed a more nuanced role for ML explanation models, when these are pragmatically embedded in the clinical context. Despite the general positive attitude of healthcare professionals (HCPs) towards explanations as a safety and trust mechanism, for a significant set of participants there were negative effects associated with confirmation bias, accentuating model over-reliance and increased effort to interact with the model. Also, contradicting one of its main intended functions, standard explanatory models showed limited ability to support a critical understanding of the limitations of the model. However, we found new significant positive effects which repositions the role of explanations within a clinical context: these include reduction of automation bias, addressing ambiguous clinical cases (cases where HCPs were not certain about their decision) and support of less experienced HCPs in the acquisition of new domain knowledge.

Interacting with and making sense of information dashboards is often problematic. Typically, users develop strategies to go around and overcome these problems. These strategies can be conceived as behavioural markers of cognitive processes that indicate problematic interactions. Consequently, if we were able to computationally model these strategies, we could detect if users are encountering problems in real time (and act accordingly). We conducted an experiment (N=63) to identify the interaction strategies users employ on problematic dashboards. We found that while existing challenges impact significantly on user performance, interventions to mitigate such challenges were especially beneficial for those with lower graph literacy. We identified the strategies employed by users when encountering problems: extensive page exploration as a reaction to information overload and use of customisation functionalities when understanding data is problematic. We also found that some strategies are indicators of performance in terms of task completion time and effectiveness: extensive exploration strategies were indicators of lower performance, while the exhibition of customisation strategies is associated with higher effectiveness.

Familiarity is a quality of user experience that has traditionally been difficult to define, capture, and quantify. Existing works on measuring familiarity with interactive systems have relied on surveys and self-reporting, which is obtrusive and prone to biases. Here, we propose a data-driven methodology to associate low-level activity patterns with familiarity. As a proof-of-concept, this methodology was tested on a website with 35,819 users over the course of 18 months, including 268 revisiting users who had reported their levels of familiarity with the platform. By using activity patterns as features of predictive models, we were able to successfully classify users with higher levels of familiarity with an accuracy of 82.7%. These results suggest that there is a relationship between user familiarity and activity patterns involving the exploration and use of navigational artefacts including breadcrumbs, navigation bars, and sidebar areas. This research opens up further opportunities for unobtrusively analysing the user experience on the Web.

Physical activity reconstruction is a process whereby self-trackers reflect on their physical activities and goals in an episodic fashion by recalling a series of past experiences and events. Tracking tools often include spatio-temporal cues (i.e. maps and timelines) to provide further context to these physical activity patterns, yet little is known about whether these features actually aid reflection. Here, we report a four-week user study (N=26) run with FitReconstruct, an app that fosters reconstruction of physical activity data using spatio-temporal and contextual cues to support recall of episodic information. We found that compared to a baseline version, presenting spatio-temporal and contextual cues increased the quality of reflection and user satisfaction. These findings inform design recommendations to support explicit reflective prompts in physical activity tracking apps.

Existing approaches to characterise engagement in online learning focus on features of the interaction of students with the learning platform including the number of posts in forums, downloads of learning materials and time spent watching videos. However, little is known about what students actually do within the learning resources and whether these activities are indicators of learning outcomes. To bridge this gap, we associate low-level activity patterns with particular student engagement modes on a connectivist MOOC (cMOOC) that ran for four weeks and involved 224 students. Our findings indicate that our approach isolates meaningful interactive behavioural markers that are indicators of engagement, and are amenable to computation.

Self-trackers reflect on their personal data to understand their behaviour and plan accordingly. Often, this reflection involves uncertainty, which can affect decision-making. To better understand the role of uncertainty, we conducted an interview study to comprehend how uncertainty influences reflection and the resulting actions. Our findings suggest that, in addition to the conventional role of uncertainty as a barrier, uncertainty also manifests as a trigger and facilitator to reflection. We discuss functionalities to alleviate the negative effects of uncertainty (e.g. incorporating users' expectations in self-tracking), and leverage its positive effects through activity reconstruction mechanisms.

The electrocardiogram (ECG), a graphical representation of the heart's electrical activity, is used for detecting cardiac pathologies. Certain medications can produce a complication known as 'long QT syndrome', shown on the ECG as an increased gap between two parts of the waveform. Self-monitoring for this could be lifesaving, as the syndrome can result in sudden death, but detecting it on the ECG is difficult. Here we evaluate whether using pseudo-colour to highlight wave length and changing the coordinate system can support lay people in identifying increases in the QT interval. The results show that introducing colour significantly improves accuracy, and that whilst it is easier to detect a difference without colour with Cartesian coordinates, the greatest accuracy is achieved when Polar coordinates are combined with colour. The results show that applying simple visualisation techniques has the potential to improve ECG interpretation accuracy, and support people in monitoring their own ECG.

People with autism often exhibit different visual behaviours from neurotypical users. To explore how these differences are exhibited on the Web, we model visual behaviour by combining pupillary response, which is an unobtrusive measure of physiological arousal, with eye-tracking scan paths that indicate visual attention. We evaluated our approach with two populations: 19 neurotypical users and 19 users with autism. We observe differences in their visual behaviours as, in certain instances, individuals with autism exhibit a lower arousal response to affective contents. While this is consistent with the literature on autism, we confirm this phenomenon on the Web. We discuss how our modelling method can be used to identify possible UX issues such as the presence of stress, cognitive load and differences in the perception of Web elements in relation to physiological arousal.

OBJECTIVE: The study sought to quantify a layperson's ability to detect drug-induced QT interval prolongation on an electrocardiogram (ECG) and determine whether the presentation of the trace affects such detection. MATERIALS AND METHODS: Thirty layperson participants took part in a psychophysical and eye-tracking experiment. Following training, participants completed 21 experimental trials, in which each trial consisted of 2 ECGs (a baseline and a comparison stimulus, both with a heart rate of 60 beats/min). The experiment used a 1 alternative forced-choice paradigm, in which participants indicated whether or not they perceived a difference in the QT interval length between the 2 ECGs. The ECG trace was presented in 3 ways: a single complex with the signals aligned by the R wave, a single complex without alignment, and a 10-second rhythm strip. Performance was analyzed using the psychometric function to estimate the just noticeable difference threshold, along with eye-tracking metrics. RESULTS: The just noticeable difference 50% and 75% thresholds were 30 and 88 ms, respectively, showing that the majority of laypeople were able to detect a clinically significant QT-prolongation at a low normal heart rate. Eye movement data indicated that people were more likely to appraise the rhythm strip stimulus systematically and accurately. CONCLUSIONS: People can quickly be trained to self-monitor, which may help with more rapid identification of drug-induced long QT syndrome and prevent the development of life-threatening complications. The rhythm strip is a better form of presentation than a single complex, as it is less likely to be misinterpreted due to artifacts in the signal.

The development of ontology engineering tools has traditionally lacked a user-centred perspective, instead being guided by the need to address particular gaps indicated by anecdotal evidence. This has typically resulted in prototypes that do not obtain traction beyond a narrow scope. Understanding the authoring patterns of ontology engineers is crucial to informing the development of ontology engineering tools that cater for the activity workflows of the users and, consequently, boosting the adoption of these tools. We report evidence about how Protégé is used across three different authoring settings, addressing the threats to validity of relying on a single user study. These settings address the continuum of expertise (from intermediate to expert users), the type of tasks (whether they are free-form or prescriptive) and the effect of the location (laboratory, tutorial or on their own) and how the studies are administered (whether or not there is a close supervision). While there are activity workflows that are particular to settings, the results indicate a number of core workflows that are common to all of them. We discuss actionable recommendations for ontology engineering tools in light of these results.

We report the process used to create artefacts for self-reporting Parkinson's Disease symptoms. Our premise was that a technology-based approach would provide participants with an effective, flexible, and resilient technique. After testing four prototypes using Bluetooth, NFC, and a microcontroller we accomplished almost full compliance and high acceptance using a paper diary to track day-to-day fluctuations over 49 days. This diary is tailored to each patient's condition, does not require any handwriting, allows for implicit reminders, provides recording flexibility, and its answers can be encoded automatically. We share five design implications for future Parkinson's self-reporting artefacts: reduce participant completion demand, design to offset the effect of tremor on input, enable implicit reminders, design for positive and negative consequences of increased awareness of symptoms, and consider the effects of handwritten notes in compliance, encoding burden, and data quality.

There are many mechanisms to sense arousal. Most of them are either intrusive, prone to bias, costly, require skills to set-up or do not provide additional context to the user's measure of arousal. We present arousal detection through the analysis of pupillary response from eye trackers. Using eye-trackers, the user's focal attention can be detected with high fidelity during user interaction in an unobtrusive manner. To evaluate this, we displayed twelve images of varying arousal levels rated by the International Affective Picture System (IAPS) to 41 participants while they reported their arousal levels. We found a moderate correlation between the self-reported arousal and the algorithm's arousal rating, r(47)=0.46, p<.01. The results show that eye trackers can serve as a multi-sensory device for measuring arousal, and relate the level of arousal to the user's focal attention. We anticipate that in the future, high fidelity web cameras can be used to detect arousal in relation to user attention, to improve usability, UX and understand visual behaviour.

Objective: Allergies are increasing, but the reasons for this are unclear. Although environmental factors are thought to be important, there is a lack of data on how they contribute to symptom development. To understand this relationship better, we need accurate data about both symptoms and environmental factors. Our objective here is to ascertain whether experience sampling is a reliable approach for collecting allergy symptom data in the general population, allowing us to map symptoms and understand etiology. Materials and Methods: We conducted a 32-week cross-sectional study where individuals reported their seasonal allergy symptoms and severity via a mobile application. Symptom geographical location and timestamp were also collected automatically. Results: The experience sampling method reliably infers the incidence of seasonal allergies as indicated by the strong correlation (r = 0.93, P < .003) between the reported lack of wellness and the number of antihistamines prescribed by General Practitioners. Discussion and Conclusion: The project has resulted in the first dataset to map allergy symptoms over time and place and reveals periods of peak hay fever symptoms in the UK.

Ontologies are complex systems of axioms in which unanticipated consequences of changes are both frequent, and difficult for ontology authors to apprehend. The effects of modelling actions range from unintended inferences to outright defects such as incoherency or even inconsistency. One of the central ontology authoring activities is verifying that a particular modelling step has had the intended consequences, often with the help of reasoners. For users of Protégé, this involves, for example, exploring the inferred class hierarchy. This paper provides evidence that making entailment set changes explicit to authors significantly improves the understanding of authoring actions regarding both correctness and speed. This is tested by means of the Inference Inspector, a Protégé plugin we created that provides authors with specific details about the effects of an authoring action. We empirically validate the effectiveness of the Inference Inspector in two studies. In a first, exploratory study we determine the feasibility of the Inference Inspector for supporting verification and isolating authoring actions. In a second, controlled study we formally evaluate the Inference Inspector and determine that making changes to key entailment sets explicit significantly improves author verification compared to the standard static hierarchy/frame-based approach. We discuss the advantages of the Inference Inspector for different types of verification questions and find that our approach is best suited for verifying added restrictions where no new signature, such as class names, is introduced, with a 42% improvement in verification correctness.

Parkinson's Disease impairs the motor, cognitive and emotional functioning of people. Clinicians do not get an accurate image of the disease because patients visit them every six months and their symptoms can change within hours. Technology has been used to tackle this problem, but most approaches disrupt people's routines or are uncomfortable to use. We aim to monitor Parkinson's in an unobtrusive, longitudinal and naturalistic way based on digital biomarkers inferred from smartphone-collected heterogeneous data. We use Parkinson's clinical scales and self-reporting of symptoms as ground truth to evaluate our methodology. Here, we present three insights we gained after tracking four people 24/7 for four months: a) the monitoring smartphone needs to be people's primary device, b) participants prefer a paper diary for symptom self-reporting, and c) social and phone interaction will be explored as digital biomarkers. We expect this approach to improve patient's quality of life and the efficiency of health services.

User interface modeling tools can play a useful role for engineering interactive systems. However, little is known about how these tools are used, the strategies and workarounds employed, and whether usability problems are encountered. To start answering this question, we run a user study with software engineers carrying out task modeling activities. The analysis of the data generated by an instrumented CTTE and an eye-tracker uncovers five workflows that illustrate how the task model hierarchy is populated and edited, operators are added to tasks, and when model consistency is checked. These findings inform design implications for task modeling tools and formulate hypotheses for future work.

Remotely stored user interaction logs, which give access to a wealth of data generated by large numbers of users, have been long used to understand if interactive systems meet the expectations of designers. Unfortunately, detailed insight into users' interaction behaviour still requires a high degree of expertise and domain specific knowledge. We present WevQuery, a scalable system to query user interaction logs in order to allow designers to test their hypotheses about users' behaviour. WevQuery supports this purpose using a graphical notation to define the interaction patterns designers are seeking. WevQuery is scalable as the queries can then be executed against large user interaction datasets by employing the MapReduce paradigm. This way WevQuery provides designers effortless access to harvest users' interaction patterns, removing the burden of low-level interaction data analysis. We present two scenarios to showcase the potential of WevQuery, from the design of the queries to their execution on real interaction data accounting for 5.7m events generated by 2,445 unique users.

Many citizen science projects ask people to create an account before they participate - some require it. What effect does the registration process have on the number and quality of contributions? We present a controlled study comparing the effects of mandatory registration with an interface that enables people to participate without registering, but allows them to sign up to 'claim' contributions. We demonstrate that removing the requirement to register increases the number of visitors to the site contributing to the project by 62%, without reducing data quality. We also discover that contribution rates are the same for people who choose to register, and those who remain anonymous, indicating that the interface should cater for differences in participant motivation. The study provides evidence that to maximize contribution rates, projects should offer the option to create an account, but the process should not be a barrier to immediate contribution, nor should it be required.

We present an experiment comparing performance of 20 novice evaluators of accessibility carrying out Web Content Accessibility Guidelines 2.0 conformance reviews working individually to performance obtained when they work in teams of two. They were asked to first carry out an individual assessment of a web page. Later on, they were matched randomly to constitute a group of two and they were asked to revise their initial assessment and to produce a group assessment of the same page. Results indicate that significant differences were found for sensitivity (inversely related to false negatives: +8%) and agreement (when measured in terms of the majority view: +10%). Members of groups exhibited strong agreement on the evaluation results among them and with the group outcome. Other measures of validity and reliability are not significantly affected by group work. Practical implications of these findings are that, for example, when it is important to reduce the false-negative rate, then employing a group of two people is more useful than having individuals carrying out the assessment. Openings for future research include further explorations of whether similar results hold for groups larger than two or what is the effect of mixing people with different accessibility background. When novice accessibility evaluators work in groups, their ability to identify all the true problems increases (by 8%). Likewise, reliability of group evaluations increases (by 10%). Individual or group evaluations can be considered as equivalent methods with respect to false positives (if differences up to 8% in correctness are tolerated). Individual or group evaluations can be considered as equivalent methods with respect to overall effectiveness (if differences up to 11% in F -measure are tolerated).

Ontologies are complex knowledge representation artefacts used widely across biomedical, media and industrial domains. They are used for defining terminologies and providing metadata, especially for linked open data, and as such their use is rapidly increasing, but so far development tools have not benefited from empirical research into the ontology authoring process. This paper presents the results of a study that identifies common activity patterns through analysis of eye-tracking data and the event logs of the popular authoring tool, Protégé. Informed by the activity patterns discovered, we propose design guidelines for bulk editing, efficient reasoning and increased situational awareness. Methodological implications go beyond the remit of knowledge artefacts: we establish a method for studying the usability of software designed for highly specialised complex domains.

The equality of access – accessibility – is difficult to quantify, define, or agree upon. Our previous work analysed the responses of web accessibility specialists in regard to a number of pre-defined definitions of accessibility. While uncovering much, this analysis did not allow us to quantify the communities’ understanding of the relationship accessibility has with other domains and assess how the community scopes accessibility. In this case, we asked over 300 people, with an interest in accessibility, to answer 33 questions surrounding the relationship between accessibility, user experience (UX), and usability; inclusion and exclusion; and evaluation, in an attempt to harmonise our understanding of web accessibility. We found that respondents think that accessibility and usability are highly related and also think that accessibility is applicable to everyone and not just people with disabilities. Respondents strongly agree that accessibility must be grounded on user-centred practices and that accessibility evaluation is more than just inspecting source code; however, they are divided as to whether training in ‘Web Content Accessibility Guidelines’ is necessary or not to assess accessibility. These perceptions are important for usability and UX professionals, developers of automated evaluation tools, and those practitioners running website evaluations.

Ontologies have been employed across scientific and business domains for some time, and the proliferation of linked data means the number and range of potential authors is set to increase significantly. Ontologies using the Web Ontology Language (OWL) are complex artefacts, however: the authoring process requires not only knowledge of the application domain, but also skills in programming and logics. To date, there has been no systematic attempt to understand the effectiveness of existing tools, or explore what users really require to build successful ontologies. Here we address this shortfall, presenting insights from an interview study with 15 ontology authors. We identify the problems reported by authors, and the strategies they employ to solve them. We map the data to a set of design recommendations, which describe how tools of the future can support ontology authoring. A key challenge is dealing with information overload: improving the user's ability to navigate, populate and debug large ontologies will revolutionise the engineering process, and open ontology authoring up to a new generation of users.

The process of authoring ontologies appears to be fragmented across several tools and workarounds, and there exists no well accepted framework for common authoring tasks such as exploring ontologies, comparing versions, debugging, and testing. This lack of an adequate and seamless tool chain potentially hinders the broad uptake of ontologies, especially OWL, as a knowledge representation formalism. We start to address this situation by presenting insights from an interview-based study with 15 ontology experts. We uncover the tensions that may emerge between ontology authors including antagonistic ontology building styles (definition-driven vs. manually crafted hierarchies). We identify the problems reported by the ontology authors and the strategies they employ to solve them. These data are mapped to a set of key design recommendations, which should inform and guide future efforts for improving ontology authoring tool support, thus opening up ontology authoring to a new generation of users. We discuss future research avenues in light of these results.

Compliance to accessibility standards does not guarantee a satisfying user experience on the Web. Both unmet content and functionality expectations have been identified as central factors on the lack of coverage shown by guidelines. We expand on this by examining the role played by subjective dimensions, and particularly expectations, on the perception that users have on web accessibility. We conducted a study with 11 blind users to explore how these expectations shape the perception of web accessibility. Our preliminary findings corroborate that expectations can affect the perception of web accessibility. Additionally, we find that expectations on the Web are built up on previous experiences and prejudices. What is more, we reveal that these expectations are not only shaped by previous Web usage, but also by real life experiences. Our outcomes suggest that user expectations should be considered in user tests.

The fact that several web accessibility metrics exist may be evidence of a lack of a comparison framework that highlights how well they work and for what purposes they are appropriate. In this paper we aim at formulating such a framework, demonstrating that it is feasible, and showing the findings we obtained when we applied it to seven existing automatic accessibility metrics. The framework encompasses validity, reliability, sensitivity, adequacy and complexity of metrics in the context of four scenarios where the metrics can be used. The experimental demonstration of the viability of the framework is based on applying seven published metrics to more than 1500 web pages and then operationalizing the notions of validity-as-conformance, adequacy and complexity. Our findings lead us to conclude that the Web Accessibility Quantitative Metric, Page Measure and Web Accessibility Barrier are the metrics that achieve the highest levels of quality (out of the seven that we examined). Finally, since we did not analyse reliability, sensitivity and validity-in-use, this paper provides guidance to address them in what are new research avenues.

Link annotation with the accessibility level of the target Web page is an adaptive navigation support technique aimed at increasing blind users' orientation in Web sites. In this work, the accessibility level of a page is measured by exploiting data from evaluation reports produced by two automatic assessment tools. These tools support evaluation of accessibility and usability guideline-sets. As a result, links are annotated with a score that indicates the conformance of the target Web page to blind user accessibility and usability guidelines. A user test with 16 users was conducted in order to observe the strategies they followed when links were annotated with these scores. With annotated links, the navigation paradigm changed from sequential to browsing randomly through the subset of those links with high scores. Even if there was not a general agreement on the correspondence between scores and user perception of accessibility, users found annotations helpful when browsing through links related to a given topic.

Sign Languages (SL) are underrepresented in the digital world, which contributes to the digital divide for the Deaf Community. In this paper, our goal is twofold: (1) to review the implications of current SL generation technologies for two key user web tasks, information search and learning and (2) to propose a taxonomy of the technical and functional dimensions for categorizing those technologies. The review reveals that although contents can currently be portrayed in SL by means of videos of human signers or avatars, the debate about how bilingual (text and SL) versus SL-only websites affect signers’ comprehension of hypertext content emerges as an unresolved issue in need of further empirical research. The taxonomy highlights that videos of human signers are ecological but require a high-cost group of experts to perform text to SL translations, video editing and web uploading. Avatar technology, generally associated with automatic text-SL translators, reduces bandwidth requirements and human resources but it lacks reliability. The insights gained through this review may enable designers, educators or users to select the technology that best suits their goals.