Understanding how users interact with systems and experience usability issues is vital in Human-Computer Interaction (HCI). Existing approaches often focus on isolated data types, such as interaction logs or subjective reports, providing only a partial view of the user experience. This work introduces AMUSED, a multimodal dataset that integrates user interaction logs, physiological signals, facial emotion features, and self-assessment reports. The dataset includes expert annotations of eleven types of usability smells, creating a rich resource for investigating relationships between usability problems, user behavior, and emotional states. We conducted experiments with 70 participants interacting with three social networks containing usability issues. We then evaluated various machine learning models to assess the feasibility of automatically detecting these issues. The dataset comprises 24 h of user recordings, with over 20,000 user events, such as clicks, scrolls, and input changes. Our analysis reveals that (i) usability smells frequently co-occur, (ii) negative emotions predominate when severe usability issues arise, and (iii) Gradient Boosting models achieve up to 92% accuracy in detecting usability smells, demonstrating the potential for computational methods in automated usability evaluation. Our findings emphasize the value of emotional metrics in HCI research and highlight promising uses of machine learning to automatically detect usability issues.

Abstract Therapeutic digital games aim to support rehabilitation treatments and help patients understand, motivate and persist in therapeutic practices. Although there are several therapeutic digital games available, their development highly depends on the involvement of computer science professionals. This paper’s contribution is about how RUFUS was designed as a platform to provide EUP (End-User Programming) for healthcare professionals. This platform enables these professionals to author digital games aiming at performing therapeutic activities with their patients and/or caregivers. The platform has a web interface for game authoring, currently composed of three different game mechanics, as well as the possibility of viewing interaction reports. It also encompasses a mobile app that displays the games created for patients. Participatory design techniques were used during the design and evaluation of the three games to engage different stakeholders and also empower healthcare providers. A preliminary evaluation of the RUFUS point to an easy-to-use design solution, but which still needs further testing to ensure intuitiveness for users who have no experience with games or with authoring interfaces.

To address diverse interaction needs of heterogeneous users' groups, user interfaces must be flexible to accommodate for customization that are specific to each user profile. Although, existing web interfaces provide some flexibility, some problems still remain: a) manual adjustments carried out by end users are required for each web application; b) the flexibility provided by current web interfaces is insufficient to address diverse interaction needs of various users' profiles and c) few users are aware about such options to customize the presentation of web interfaces. To contribute to the customization of user interface according to the needs of diverse users, in this work we asses the suitability of a tool that customize web interfaces based on the needs and preferences of end users. UIFlex is a web-based browser plugin that enables users to define their interaction profile. In this task, users are supported by fifteen web-based design rules that were extracted from the literature and the knowledge of authorities. To customize the presentation of web interfaces, UIFlex relies on a set of rules defined for each individual user and "injects" JavaScript codes, Cascading Style Sheets (CSS) and in some cases HyperText Markup Language (HTML) codes in any page that follows W3C standards. UIFlex was evaluated by 104 users of diverse interaction profiles. The results obtained are promising and suggest that the solution improves the perception that the interactive system performs as desired by users.

The Experience Sampling Method (ESM) has been proposed as a method for collecting data about people's experiences in their everyday and natural environments. ESM-based systems offer limited authoring for interactive documents designed to collect text-based responses offered as answers to text-based questions, and integrated with the non-intrusive data collection from sensors. From a document engineering perspective, ESM brings new requirements with respect to the authoring of non-trivial interaction and navigation workflow, in particular when multiple media and collaborative tasks are concerned. Tackling existing challenges, we modeled the Experience Sampling and Programmed Intervention Method (ESPIM) by combining ESM, individualized teaching procedures and ubiquitous computing toward producing interactive personalized multimedia documents applied in data collection.

Ubiquitous systems can be characterized as providing transparency, while being context-sensitive and dynamically interactive. Evaluating ubiquitous environments involves challenges ranging from multidisciplinary areas to environmental simulations. However, few studies have been carried out on the user's feelings and moods in these environments, i.e. within mental contexts. This paper undertakes an observational case study which involves evaluating emotional responses in ubiquitous environments. A number of volunteers were exposed to stimuli with the aim of changing their mood through colors and images, in accordance with their initial emotional state. Emotional responses were evaluated by means of a hybrid approach that entailed employing different evaluation instruments at different periods of the interaction and which took account of different stakeholders. The results suggest that the approach is appropriate, although some adjustments are required for the evaluation instruments, as well for the interaction time with the environment so that it is possible to collect further data about the user's experience.