Perceived clutter is a potentially important but understudied construct in UX research. In this paper we described the development and assessment of a standardized questionnaire for reliable and valid measurement of perceived clutter of websites. Starting with an initial set of 16 items and two hypothesized factors, a series of exploratory analyses led to a final set of five items, two for the hypothesized construct of Content Clutter (too much irrelevant content like ads and videos) and three for the hypothesized construct of Design Clutter (poor design of relevant information like too much text, an unpleasant layout, or too much visual noise). Confirmatory analyses using an independent dataset showed excellent fit statistics for CFA of the five-item questionnaire and good fit for an SEM of the connections between clutter and other UX constructs. Researchers should exercise caution about generalizing results to other contexts and interfaces, but UX practitioners should be able to use this perceived clutter of websites (PCW) questionnaire when assessing consumer websites.

We replicated and extended previous research to investigate the extent to which perceived ease of use and perceived usefulness account for variation in overall experience, likelihood to recommend, intention to use, and reported usage in a three-month follow-up. Consistent with previous research, we found little effect on structural equation models from varying three measures of perceived ease and two measures of perceived usefulness. All models had statistically significant standardized estimates and squared multiple correlations and had acceptable fit statistics. Despite these manipulations, the models supported a consistent narrative. Both perceived ease and perceived usefulness are important antecedents that either directly or indirectly affect the experiential and intentional outcomes (perceived usefulness somewhat more than perceived ease), with intention to use accounting for 19% of variation in follow-up ratings of usage. These models support UX practitioners by demonstrating the importance of work that improves perceptions of product ease and usefulness and showing that the two-item UX-Lite questionnaire is an effective and efficient measure of perceived ease and usefulness.

Lascu and Clow provide a helpful literature review into web site satisfaction, detail a repeatable process for other researchers and propose their own scale of website interactivity with four subscales with low-to-strong internal reliability. The major disadvantage of the new scale is one-third of the 15-item questionnaire measures aspects of customer service center quality. Although such measures of ‘Customer Centeredness’ are important, it tends to be less universally helpful to many large e-commerce websites which have little telephonic interaction with their customers.

Christophersen and Konradt show that a single-item measure of usability can retain most of the information garnered in multi-item scale. While there is some loss of information and reliability, the authors show that when a single item is needed, it can be reliable, sensitive (discriminate between good and bad usability) and valid (correlates with other known measures of usability).

When designing questionnaires there is a tradition of including items with both positive and negative wording to minimize acquiescence and extreme response biases. Two disadvantages of this approach are respondents accidentally agreeing with negative items (mistakes) and researchers forgetting to reverse the scales (miscoding).

The distribution of task time data in usability studies is positively skewed. Practitioners who are aware of this positive skew tend to report the sample median. Monte Carlo simulations using data from 61 large-sample usability tasks showed that the sample median is a biased estimate of the population median. Using the geometric mean to estimate the center of the population will, on average, have 13% less error and 22% less bias than the sample median. Other estimates of the population center (trimmed, harmonic and Winsorized means) had worse performance than the sample median.

Post-task ratings of difficulty in a usability test have the potential to provide diagnostic information and be an additional measure of user satisfaction. But the ratings need to be reliable as well as easy to use for both respondents and researchers. Three one-question rating types were compared in a study with 26 participants who attempted the same five tasks with two software applications. The types were a Likert scale, a Usability Magnitude Estimation (UME) judgment, and a Subjective Mental Effort Question (SMEQ). All three types could distinguish between the applications with 26 participants, but the Likert and SMEQ types were more sensitive with small sample sizes. Both the Likert and SMEQ types were easy to learn and quick to execute. The online version of the SMEQ question was highly correlated with other measures and had equal sensitivity to the Likert question type.

Correlations between prototypical usability metrics from 90 distinct usability tests were strong when measured at the task-level (r between .44 and .60). Using test-level satisfaction ratings instead of task-level ratings attenuated the correlations (r between .16 and .24). The method of aggregating data from a usability test had a significant effect on the magnitude of the resulting correlations. The results of principal components and factor analyses on the prototypical usability metrics provided evidence for an underlying construct of general usability with objective and subjective factors.

Current methods to represent system or task usability in a single metric do not include all the ANSI and ISO defined usability aspects: effectiveness, efficiency & satisfaction. We propose a method to simplify all the ANSI and ISO aspects of usability into a single, standardized and summated usability metric (SUM). In four data sets, totaling 1860 task observations, we show that these aspects of usability are correlated and equally weighted and present a quantitative model for usability. Using standardization techniques from Six Sigma, we propose a scalable process for standardizing disparate usability metrics and show how Principal Components Analysis can be used to establish appropriate weighting for a summated model. SUM provides one continuous variable for summative usability evaluations that can be used in regression analysis, hypothesis testing and usability reporting.