Measuring user experience is vital for long-term success of interactive products. Questionnaires like the modular extension of the User Experience Questionnaire (UEQ+) are an established instrument for this purpose. Different item formats are available for these questionnaires, such as the number of response options (most frequent 5- or 7- point Likert scales). But the item format of an UX questionnaire can of course influence the measured results. We investigate in this paper if the change to a one-sided polarity of semantic differential items influences the effort of the participants required to answer these items and on the measured scale scores. Therefore, we conducted 6 studies with 438 collected responses for the well-known products Microsoft PowerPoint, WhatsApp and Google Maps. Each product was evaluated by a sample of participants with the original UEQ and a modified version of the UEQ with one-sided polarity. In the modified version, the positive term of the semantic differ ential was always placed in the right position, while it is placed in half of the items in the positive and the other half in the left position in the original UEQ version. The results showed that the effort to complete the questionnaire (completion time and number of required corrections) was lower for the version with one-sided polarity, but the differences were so small that they are not practically relevant. But the results also showed that the change to a one-sided polarity introduced an answer tendency, which impact the scale scores. Therefore, the results obtained with the two versions of the UEQ cannot directly be compared. Based on this, we can conclude that it is not possible to directly compare the scores of the original UEQ scales with the corresponding scores of UEQ+ scales.

In order to be able to meaningfully classify the user experience and thus the popularity of products, UX questionnaires such as the UEQ, SUS or UMUX are frequently used in practice to measure the UX. This makes it possible to specifically evaluate the ratings of pragmatic and hedonic UX factors. However, it is conceivable that, in addition to users’ own perceptions, external factors also have an influence on the evaluation of the UX of products. These include, for example, time or duration of use. It can be assumed that users who rate the UX of a product as good also use this product more frequently and vice versa. Such a consideration of influencing factors is particularly interesting for products that have been used frequently in recent years and thus also during the pandemic. For this reason, Netflix, Microsoft PowerPoint, Zoom and BigBlueButton were selected, which cover the range from primarily hedonic to primarily pragmatic quality. These are examined for their UX ratings as we ll as influencing factors.