There is little research in the field of problematic smartphone use, which directly enquired with users about what they find disturbing about smartphone usage behaviour. This qualitative study addressed sub-critical issues which may be a precursor to more severe problems such as problematic smartphone usage. Ninety-nine participants completed an online questionnaire with open questions regarding disturbing smartphone usage behaviour. Using the method of ‘thematic analysis’, we identified eight themes that we clustered into two overarching themes: reasons for use and consequences of use. Five themes related to reasons for use, including inevitability, habitual use, avoiding unpleasant circumstances, need satisfaction and fulfilling social expectations. Three themes referred to consequences of smartphone usage (i.e. life management, social life, and online life). Across all themes it appeared that the different aspects of disturbing smartphone usage were triggered and sustained by contextual factors and complex social dynamics, suggesting that participants lack control over their actual usage. Therefore, this research provides a holistic view of the sub-critical issues disturbing participants as well as identifies several mechanisms perpetuating these problems. The listing and description of such sub-critical issues and the different mechanisms are of high importance for prevention and the design of interventions.

In recent years, alternative types of usability questionnaires using graphical elements (pictorial scales) or a combination of graphical and verbal elements (hybrid scales) have been introduced. Previous research indicates that these questionnaires have advantages, such as increased respondent motivation, and drawbacks, such as extended questionnaire completion time. This study aimed to systematically investigate the psychometric properties and the respondents’ experience of two versions of a recently developed questionnaire, the Pictorial Usability Inventory (PUI), consisting of a hybrid and pictorial version. Given that questionnaire length is a crucial factor for the usefulness of a scale, the study tested long and short versions (8 items vs 3 items) of both questionnaire types. The study involved an online usability test with 777 participants, who were asked to complete one of the four PUI versions and an established verbal usability scale after solving three tasks on a webpage. The results demonstrated high sensitivity, high convergent validity, and good internal consistency for all four PUI versions. While the long pictorial scale achieved the best psychometric properties overall, participants preferred the hybrid scales, particularly the short version. The study's findings are in line with previous research on pictorial and hybrid instruments and suggest that hybrid instruments, particularly short ones, may be superior to purely pictorial instruments in terms of respondent-centred aspects conceptualised in the term ‘questionnaire experience’.

This field experiment aimed to address the issue of problematic smartphone usage (PSU) by examining the effectiveness of two frequently advised techniques to deal with PSU: moving problematic applications to a different page of the phone and changing the phone into greyscale. For one week, 97 student participants used their smartphone under one of the following three conditions: move app, greyscale, or control condition. Compared to the week before the intervention, participants significantly reduced their objective smartphone usage, but their self-reported PSU increased. These effects also emerged in the control group, which might have been due to heightened awareness of the participants of their smartphone usage. A major implication of the study is the need of using multiple outcome measures to understand the complex relationship between perceptions and behaviour. This study could be used as a blueprint for future research examining the effectiveness of interventions in the field of PSU.

The article reports a longitudinal lab experiment, in which the influence of product aesthetics and inherent product usability was examined over a period of 7 weeks. Using a 2 × 2 × 7 mixed design, visual aesthetics (high vs. low) and usability (high vs. low) were manipulated as between-subjects variables whereas exposure time was used as a repeated-measures variable. One hundred and ten participants took part in the study, during which they carried out typical tasks of operating a fully automated coffee machine. We measured user experience by using the following outcome variables: perceived usability, perceived attractiveness, performance, affect, workload and perceived coffee quality (gustatory aesthetics). We found no effect of visual aesthetics on user experience (including perceived usability as the chief outcome variable), which is in contrast to a considerable number of previous studies. The absence of such an effect might be associated with influencing factors that have not yet been given sufficient attention (e.g., user identification with product, sensory dominance, characteristics of specific products).

This article presents the concept of questionnaire experience (QX), intending to add a new element to the psychometric evaluation of questionnaires, which may eventually help increase the validity and reliability of instruments. The application of QX is demonstrated in the development of the Hybrid System Usability Scale (H-SUS), making use of items comprising pictorial and verbal elements to measure perceived usability. The H-SUS was modelled on the verbal version of the System Usability Scale (SUS). Since previous research showed advantages of pictorial scales over verbal scales (e.g., higher respondent motivation) but also disadvantages (e.g., longer completion times), we assumed that hybrid scales would combine the advantages of both scale types. The goal of this study was to compare the two instruments by assessing traditional psychometric criteria (convergent, divergent and criterion-related validity, reliability and sensitivity) and respondent-related aspects of QX (respondent workload, respondent motivation, questionnaire preference, and questionnaire completion time). An online experiment was carried out (N = 152), in which participants interacted with a smartphone prototype and subsequently completed the verbal SUS together with the H-SUS. Results indicate good psychometric properties of the H-SUS. Compared to the SUS, the H-SUS showed similar workload levels for questionnaire completion, higher levels of respondent motivation, but longer questionnaire completion time. Overall, the H-SUS is considered a promising alternative for the evaluation of perceived usability. Finally, QX can be considered a useful concept for identifying potential problems of psychometric instruments in a respondent-centred way, which may help improve the quality of future scales.

We have developed a pictorial multi-item scale, called P-SUS (Pictorial System Usability Scale), which aims to measure the perceived usability of mobile devices. The scale is based on the established verbal usability questionnaire SUS (System Usability Scale). A user-centred design process was employed to develop and refine its 10 pictorial items. The scale was tested in a first validation study (N=60) using student participants. Psychometric properties (convergent validity, criterion-related validity, sensitivity, and reliability), as well as the motivation to fill in the scale were assessed. The results indicated satisfactory convergent validity for about two-thirds of the items. Furthermore, strong correlations were obtained for the sum scores between verbal and pictorial SUS, and the pictorial scale was perceived as more motivating than the verbal questionnaire. The P-SUS represents a first attempt to provide a pictorial usability scale for the evaluation of (mobile) devices.

Tools that provide visual feedback about emotions to the user in the form of an avatar or an emoticon have become increasingly important. While a great deal of effort has already been put into the reliable and accurate automatic detection of emotions, only very little is known about how this information about affective states should be displayed in a comprehensible way to the user. In the present study, three newly developed feedback tools were evaluated. The tools were developed on the basis of an existing non-verbal questionnaire to represent two dimensions of emotion (i.e. valence and arousal) based on the circumplex model of affect. A total number of 826 participants were tested, using different vignettes that describe situations with specific affective content. Employing three newly developed affective feedback tools (AniSAM, AniAvatar and MergedSAM), the ratings obtained were compared to ratings using the original SAM instrument, a well-established questionnaire to measure affect. Results indicated that the animated feedback increased the accuracy of the arousal representation. Furthermore, valence feedback was more accurate when provided with an animated manikin-based tool rather than an avatar-based tool. This provided first evidence for the usefulness of animated tools offering visual feedback on user emotion. All instruments need to undergo further development. AniSAM and AniAvatar can be downloaded for purposes of practical applications and further research.

An experimental study examined whether the effects of aesthetic appeal on usability test outcomes are moderated by usage domain. The aesthetic appeal of a cell phone was experimentally manipulated in both home- and work-based usage domains. The two usage domains were modeled in a usability laboratory. 60 participants completed a series of typical cell phone user tasks. Dependent measures such as performance, perceived usability, and emotion were taken. The results showed that aesthetic appeal had a positive effect on perceived usability but a negative effect on performance. The effects of aesthetic appeal on usability test outcomes were not moderated by usage domain. The results of this study imply that it may be sufficient to test dual-domain products in only one of their usage domains.

This study examined the utility of the concept of expressive aesthetics by testing websites that did or did not match this concept. A website scoring highly on this concept was created and was then compared to websites that were either non-aesthetic or corresponded to the concept of classical aesthetics. Sixty website users of a broad age range (18–60 years) were allocated to three experimental groups (expressive, classical, and non-aesthetic) and asked to complete a series of information search tasks. During the experiment, measures were taken of performance, perceived usability, perceived aesthetics, emotion, and trustworthiness. The results showed that expressive aesthetics can be considered a distinct concept. It also emerged that the website scoring high on expressive aesthetics shows a similar pattern of results to classical aesthetics. Both aesthetically appealing websites received higher ratings of perceived usability and trustworthiness than the non-aesthetic website. The effects of website aesthetics on subjective measures were not moderated by age.

This paper examines how emotion feedback influences emotion awareness and gaze behavior. Simulating a videoconference setup, 36 participants watched 12 emotional video sequences that were selected from the SEMAINE database. All participants wore an eye-tracker to measure gaze behavior and were asked to rate the perceived emotion for each video sequence. 3 conditions were tested: (c1) no feedback, i.e., the original video-sequences, (c2) correct feedback, i.e., an emoticon is integrated in the video to show the emotion depicted by the person in the video and (c3) random feedback, i.e., the emoticon displays at random an emotional state that may or may not correspond to the one of the person. The results showed that emotion feedback had a significant influence on gaze behavior, e.g., over time random feedback led to a decrease in the frequency of episodes of gaze. No effect of emotion display was observed for emotion recognition. However, experiments on the automatic emotion recognition using gaze behavior provided good performance, with better score on arousal than valence, and a very good performance was obtained in the automatic recognition of the correctness of the emotion feedback.

The aim of the study was to investigate the influence of perceived prototype fidelity in usability tests by comparing two prototypes that differed with respect to their perceived proximity to the final system. The impact of the perceived developmental stage of the product was examined for participants' performance, perceived usability, emotions and psychophysiology. Eighty participants were tested, operating an electronic city guide on a mobile phone. In a 2×2×2 mixed design, the system was either presented as an early prototype or as the final system. In addition, observer presence (no observers vs. three observers) and task difficulty (high vs. low) were experimentally manipulated. Overall, the findings did not indicate major differences for perceived prototype fidelity. However, an interaction between the observer presence and prototype fidelity indicated that the observer presence had a more negative impact on the performance when testing a final system than an early prototype. Furthermore, the observer presence resulted in a psychophysiological stress response. The findings suggest that test outcomes are quite robust against different prototype perceptions but that the observer presence needs careful consideration

An empirical study examined the effects of two influencing factors in usability tests on user performance and usability ratings. Product aesthetics (high vs. low) as the main independent factor and prior usage event (positive vs. negative) as a subsidiary independent factor were varied in a between-participants design. 60 participants took part in the experiment, completing a number of typical tasks of mobile phone users. The results showed that increased product aesthetics had a positive effect on perceived usability and led to longer task completion times. After a negative usage event had been experimentally induced, usability ratings dropped as expected but user performance on subsequent tasks remained unaffected. An important implication of the study is that the aesthetic properties of a product may have multiple effects that go beyond perceived product attractiveness.